CN107844405A - Log processing method and device, server - Google Patents
Log processing method and device, server Download PDFInfo
- Publication number
- CN107844405A CN107844405A CN201710948597.6A CN201710948597A CN107844405A CN 107844405 A CN107844405 A CN 107844405A CN 201710948597 A CN201710948597 A CN 201710948597A CN 107844405 A CN107844405 A CN 107844405A
- Authority
- CN
- China
- Prior art keywords
- log
- sliding window
- daily record
- write
- log space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Computer Hardware Design (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Debugging And Monitoring (AREA)
Abstract
A kind of log processing method of offer of the embodiment of the present invention and device, server, this method, including：Application date book section, daily record section include the M log spaces arranged in order；N number of daily record is concurrently write to M log space, wherein, N≤M, and N number of daily record is produced by external equipment；Slided using end log space of the sliding window from the origin log space of daily record section to daily record section, wherein, sliding window covers K log space, K<M；The first confirmation is sent respectively to external equipment for each log space that write-in completion status is in sliding window, so that each log space in write-in completion status in sliding window is in write-in success status, the first confirmation is not sent to external equipment for each log space in write-in completion status outside sliding window.By the way that with upper type, the efficiency of log read can be ensured, and reduce the loss for writing concurrency.
Description
Technical field
The application is related to areas of information technology, more particularly to a kind of log processing method and device, server.
Background technology
In the log system of prior art, log system application date book section in disk, daily record section includes multiple orders
The log space of arrangement, external equipment send the daily record write request for carrying daily record to log system, and log system will receive
The log space at the daily record deposit daily record end arrived.
In the prior art, external equipment sends multiple daily record write requests to log system, and log system is by multiple days
The daily record that will write request carries respectively is concomitantly written in the log space of daily record section, for any one of daily record section
For log space, if after the daily record write-in of the log space is completed, log system is directed to before the log space
Other log spaces do not send confirmation (Acknowledgement, ACK) to external equipment, then log system is for being somebody's turn to do
Log space can not send ACK to external equipment, and log system must wait until that other daily records before the log space are empty
Between send confirmation ACK to external equipment, the log space could be directed to and send confirmation ACK to external equipment.
In the prior art, arrangement above is passed through, it is ensured that when being subsequently read out to daily record section, lost reading first write-in
The daily record lost just stops reading other log spaces of this daily record section, so as to guarantee efficiently to read daily record section.
But conventionally, as log system must wait until other daily records before particular log space
Space sends confirmation ACK to external equipment, could be directed to the particular log space and send confirmation ACK to outside
Equipment, which results in the loss for writing concurrency, have impact on the performance for writing daily record, if the dependence between this ACK
To remove, then this daily record section may have the daily record entanglement write successful daily record and write failure and be stored in daily record section,
So when daily record section is read, if to obtain effective daily record, that just has to travel through whole daily record section, and this results in day
Will reads poorly efficient.
The content of the invention
To solve problem of the prior art, the embodiment of the present invention provides a kind of log processing method and device, server, energy
Enough ensure the efficiency of log read, and reduce the loss for writing concurrency.
In a first aspect, the embodiment of the present invention provides a kind of log processing method, including：
Application date book section, daily record section include the M log spaces arranged in order；
N number of daily record is concurrently write to M log space, wherein, N≤M, and N number of daily record is produced by external equipment；
Slided using end log space of the sliding window from the origin log space of daily record section to daily record section, wherein, it is sliding
Dynamic window covers K log space, K<M；
The first confirmation is sent respectively to outer for each log space that write-in completion status is in sliding window
Portion's equipment so that each log space in write-in completion status in sliding window is in write-in success status, for sliding
Each log space in write-in completion status outside window does not send the first confirmation to external equipment；
Wherein, the first confirmation is used to notify external equipment daily record to write successfully.
To sum up, in embodiments of the present invention, due to having used sliding window, and limit for being in write-in in sliding window
The log space of completion status sends the first confirmation to external equipment, for being in write-in completion status outside sliding window
Log space does not send the first confirmation to external equipment, by sliding window is covered for single log space
Scope it is bigger, therefore the loss for writing concurrency can be reduced.Further, because the log space before sliding window is in
Success status is write, therefore the efficiency of log read can be ensured.
In a kind of implementation of the embodiment of the present invention, using sliding window from the origin log space of daily record section to end
First day of the lunar year will dimensional slip, is specifically included：
When write-in success status is in the origin log space of sliding window, the end day by sliding window to daily record section
Line slip is entered in will space, until the new origin log space of sliding window is in other states of write-not success status.
Sliding window with upper type by being slided, it is ensured that the log space before sliding window is in being written to
Work(state.
In another implementation of the embodiment of the present invention, K is variable, and K=Kn, wherein n is the cunning of sliding window
Dynamic number, KnRepresent that sliding window covers K after carrying out n-th slipnIndividual log space, wherein, K0=M/10, and：
Work as N>When=4,
Work as N<When 4,
By under different load pressures, dynamic adjusts sliding window size, can effectively improve log system and
Write performance is sent out, while lifts the efficiency of log read.
In another implementation of the embodiment of the present invention, this method also includes：
The origin log space of sliding window be in write-in status of fail when, sliding window origin log spatially
Invalid data is write, after invalid data is write, the second confirmation is sent extremely for the origin log space of sliding window
External equipment, and wait other log spaces in sliding window in addition to the origin log space of sliding window to complete log write
Enter, wherein, the second confirmation is used to notify external equipment daily record write-in failure.
In another implementation of the embodiment of the present invention, this method also includes：
Read each daily record of daily record section one by one from the origin log space of daily record section to the end log space of daily record section
Space；
When reading invalid data first, K-1 be located at after the log space with invalid data are read one by one
Log space.
Pass through mode read above, it is ensured that the daily record of K-1 log space in sliding window that last time is slided
It can be read into, so as to ensure that the data for reading daily record are complete.
Second aspect, the embodiment of the present invention provide a kind of log processing device, including：
Daily record section application module, for application date book section, daily record section includes the M log spaces arranged in order；
Daily record writing module, for concurrently writing N number of daily record to M log space, wherein, N≤M, and N number of daily record is by outer
Portion's equipment produces；
Sliding window module, for using end daily record of the sliding window from the origin log space of daily record section to daily record section
Dimensional slip, wherein, sliding window covers K log space, K<M；
Confirmation sending module, for distinguishing for each log space that write-in completion status is in sliding window
The first confirmation is sent to external equipment so that each log space in write-in completion status in sliding window, which is in, to be write
Enter success status, the first confirmation is not sent to outer for each log space in write-in completion status outside sliding window
Portion's equipment；
Wherein, the first confirmation is used to notify external equipment daily record to write successfully.
Second aspect or second aspect any one be achieved in that first aspect or any one realization side of first aspect
System corresponding to formula realizes, the description in first aspect or first aspect any one implementation is applied to second aspect or the
Any one implementation in terms of two, will not be repeated here.
In a kind of implementation of the embodiment of the present invention, sliding window module, it is specifically used for：
When write-in success status is in the origin log space of sliding window, the end day by sliding window to daily record section
Line slip is entered in will space, until the new origin log space of sliding window is in other states of write-not success status.
In another implementation of the embodiment of the present invention, K is variable, and K=Kn, wherein n is the cunning of sliding window
Dynamic number, KnRepresent that sliding window covers K after carrying out n-th slipnIndividual log space, wherein, K0=M/10, and：
Work as N>When=4,
Work as N<When 4,
In another implementation of the embodiment of the present invention, confirmation sending module, it is additionally operable to：
The origin log space of sliding window be in write-in status of fail when, sliding window origin log spatially
Invalid data is write, after invalid data is write, the second confirmation is sent extremely for the origin log space of sliding window
External equipment, and wait other log spaces in sliding window in addition to the origin log space of sliding window to complete log write
Enter, wherein, the second confirmation is used to notify external equipment daily record write-in failure.
In another implementation of the embodiment of the present invention, log processing device also includes log read module, daily record
Read module, it is used for：
Read each daily record of daily record section one by one from the origin log space of daily record section to the end log space of daily record section
Space；
When reading invalid data first, K-1 be located at after the log space with invalid data are read one by one
Log space.
The third aspect, the embodiment of the present invention provide a kind of server, including memory and processor, memory storage have journey
Sequence instructs, and computing device programmed instruction is to cause server to perform the function of above-mentioned log processing device.
Fourth aspect, the application provide a kind of computer-readable recording medium, deposited in the computer-readable recording medium
Instruction is contained, when run on a computer so that computer performs any possibility of above-mentioned first aspect or first aspect
Implementation described in method.
5th aspect, the application provide a kind of computer program product for including instruction, when run on a computer,
So that computer performs the method described in any possible implementation of above-mentioned first aspect or first aspect.
Brief description of the drawings
Fig. 1 is the structural representation of log processing system according to embodiments of the present invention；
Fig. 2 is the flow chart of log processing method according to embodiments of the present invention；
Fig. 3 is the schematic diagram of the daily record section in first state according to embodiments of the present invention；
Fig. 4 is the schematic diagram according to embodiments of the present invention that daily record is concurrently write in daily record section；
Fig. 5 is the schematic diagram of the daily record section in the second state according to embodiments of the present invention；
Fig. 6 is the schematic diagram of the daily record section in the third state according to embodiments of the present invention；
Fig. 7 is the schematic diagram of the daily record section in the 4th state according to embodiments of the present invention；
Fig. 8 is the schematic diagram of the daily record section in the 5th state according to embodiments of the present invention；
Fig. 9 is the schematic diagram of the daily record section in the 6th state according to embodiments of the present invention；
Figure 10 is another flow chart of log processing method according to embodiments of the present invention；
Figure 11 is the apparatus structure schematic diagram of log processing device according to embodiments of the present invention；
Figure 12 is the apparatus structure schematic diagram of server according to embodiments of the present invention.
Embodiment
Referring firstly to Fig. 1, Fig. 1 is the structural representation of log processing system according to embodiments of the present invention, such as Fig. 1 institutes
Show, log processing system includes server 10, network 20 and external equipment 201,202,203, and server 10 passes through network 20
It is connected with external equipment 201,202,203.
Server 10 specifically includes log processing device 100, operating system 102 and hardware 101, and hardware 101 specifically wraps
Include disk 1011, network interface card 1012, CPU1013 and internal memory 1014.Wherein, log processing device 100 operates in operating system 102
On, and control network interface card 1012 to carry day from what the reception external equipment 201,202,203 of network 20 was sent by operating system 102
The daily record write request of will, and confirmation can be sent via network 20 to external equipment 201,202,203.
Also, log processing device 100 can by operating system 102 to disk 1011 apply the space of disk 1011 using as
Daily record section, wherein, daily record section includes multiple tactic log spaces, and log space is used to store daily record.
Alternatively, a log space stores the daily record entrained by a daily record write request.
Fig. 2 is referred to below, Fig. 2 is the flow chart of log processing method according to embodiments of the present invention, as shown in Fig. 2
Log processing method according to embodiments of the present invention comprises the following steps：
Step S101：The application date book section 300 of log processing device 100, and sliding window 200 is set in daily record section 300
Original position, wherein, daily record section 300 includes the M log spaces arranged in order, and it is empty that sliding window 200 covers K daily record
Between, K<M.
For example, reference can be made to Fig. 3, Fig. 3 are showing for the daily record section 300 in first state according to embodiments of the present invention
It is intended to, as shown in figure 3, daily record section 300, and M=16 are arrived in the application of log processing device 100, sliding window 200 covers 7 daily records
Space, K=7.
In embodiments of the present invention, the origin log space of sliding window 200 and the origin log space weight of daily record section 300
Close, so that sliding window 200 is arranged at the original position of daily record section 300.
In this step, because log processing device 100 does not carry out daily record write-in, therefore daily record section 300 in daily record section 300
Each log space be in unwritten state.
Step S102：Log processing device 100 concurrently writes N number of daily record to M log space, wherein, N≤M, and it is N number of
Daily record is produced by external equipment 201.
Alternatively, in embodiments of the present invention, one daily record of the corresponding write-in of each log space, and N number of daily record synchronous write
Enter into the top n log space of M log space.
For example, reference can be made to Fig. 4, Fig. 4 are the signals for concurrently writing daily record in daily record section 300 according to embodiments of the present invention
Figure, as shown in figure 4, N=13, i.e. log processing device 100 are respectively written into daily record 1 to 13 in log space 1 to 13 simultaneously.
Step S103：Log processing device 100 is empty for each daily record in write-in completion status in sliding window 200
Between send the first confirmation respectively to external equipment 201, for each day in write-in completion status outside sliding window 200
Will space does not send the first confirmation to external equipment 201, wherein, the first confirmation is used to notify external equipment 201
Will writes successfully.
It can be referred to reference to Fig. 5 and Fig. 6, wherein, Fig. 5 is the daily record in the second state according to embodiments of the present invention
The schematic diagram of section 300, Fig. 6 is the schematic diagram of the daily record section 300 in the third state according to embodiments of the present invention.
As shown in figure 5, log space 1,2,3,5,8,10,11 and 13 be in write completion status, log space 4,
6th, 7,9,12,14,15 and 16 it is in unwritten state.
Wherein, log space 4,6,7,9,12 is in during the reason for unwritten state is that daily record is currently written into, or
The reason for daily record write-in fails, and log space 14,15 and 16 is in unwritten state is not to be directed to those log spaces
Carry out daily record write-in.
In this step, log processing device 100 is for each daily record in write-in completion status in sliding window 200
Space 1,2,3 and 5 sends the first confirmation to external equipment 201 respectively, for being completed outside sliding window 200 in write-in
Each log space 8,10,11 and 13 of state does not send the first confirmation to external equipment 201.
In embodiments of the present invention, log processing device 100 is directed to the log space in write-in completion status and sends the
After one confirmation to external equipment 201, represent to write successfully in the daily record that the log space is carried out, now the log space
In write-in success status.
Fig. 6 is for details, reference can be made to, log processing device 100 is for being in each of write-in completion status in sliding window 200
After log space 1,2,3 and 5 sends the first confirmation to external equipment 201 respectively, at log space 1,2,3 and 5
In write-in success status, and log space 8 outside sliding window 200,10,11 and 13 still in write-in completion status.
Step S104：Log processing device 100 judges whether the origin log space of sliding window 200 is in and write successfully
State, if it is, step S105 is performed, if not, performing step S106.
For shown in Fig. 6, the origin log space 1 of sliding window 200 is in write-in success status, log processing device
100 perform step S105.
Step S105：Log processing device 100 is slided sliding window 200 to the end log space of daily record section 300
It is dynamic, until the new origin log space of sliding window 200 is in other states of write-not success status, or until sliding window
The end log space of mouth 200 and the terminal position space coincidence of daily record section 300.
For example, reference can be made to Fig. 7, Fig. 7 are showing for the daily record section 300 in the 4th state according to embodiments of the present invention
It is intended to, as shown in fig. 7, sliding window 200 enters line slip to the end log space 16 at daily record end, until sliding window 200
New origin log space 4 is in other states of write-not success status, and sliding window 200 stops sliding, wherein, in this cunning
The new origin log space 4 of dynamic window 200 is in unwritten state.
In other examples, other states of write-not success status can also be write-in completion status.
After sliding window 200 stops sliding, log processing device 100 jumps to step S103, for new slip
Log space in window 200 carries out similar process.
Specifically, log processing device 100 is for each day in write-in completion status in new sliding window 200
Will space 8 and 10 sends the first confirmation to external equipment 201 respectively, for being completed outside sliding window 200 in write-in
Each log space 11 and 13 of state does not send the first confirmation to external equipment 201.
Fig. 8 can be combined, Fig. 8 is the schematic diagram of the daily record section 300 in the 5th state according to embodiments of the present invention, is such as schemed
Shown in 8, after above-mentioned processing is carried out, the log space 5,8 and 10 in new sliding window 200 is in writing successfully shape
State.
Now, log processing device 100 can jump to step S104, judge that the origin log of new sliding window 200 is empty
Between 4 whether in write-in success status, and as shown in figure 8, log space 4 is in unwritten state, now judged result is no,
Log processing device 100 jumps to step S106.
Step S106：Log processing device 100 spatially writes invalid data, pin in the origin log of sliding window 200
Second confirmation is sent to external equipment 201 to the origin log space of sliding window 200, and waited in sliding window 200
Other log spaces in addition to the origin log space of sliding window 200 complete daily record write-in.Wherein, the second confirmation is used for
Notify the daily record of external equipment 201 write-in failure.
It can be referred to reference to Fig. 9, Fig. 9 is showing for the daily record section 300 in the 6th state according to embodiments of the present invention
It is intended to, as shown in figure 9, log processing device 100 can write invalid number on the new origin log space 4 of sliding window 200
According to so that log space 4 is in write-in status of fail, and waits the origin log in sliding window 200 except sliding window 200 empty
Between other log spaces 6,7 and 9 outside 4 complete daily record write-in.
Alternatively, for the log space in unwritten state, log processing device 100 in sliding window 200
After daily record is write to it, preset time is may wait for, if exceeding the preset time, writes direct invalid data.
Such as log space 4, the timing after daily record write-in is carried out to log space 4 of log processing device 100, if super
Go out preset time, then the log space 4 directly in sliding window 200 writes invalid data so that log space 4 is in write-in
Status of fail, and can also do similar process for log space 6,7,9.
In embodiments of the present invention, the log space that log processing device 100 can be further after sliding window 200 is equal
Write invalid data so that the log space after sliding window 200 is in writing status of fail.
In this, log processing device 100 completes the daily record write operation of current log section 300, the day of current log section 300
Will space 1,2,3,5,8,10 have recorded effective daily record, and other log spaces then have recorded invalid data.
In the embodiment of the present invention, due to having used sliding window 200, and limit for being in write-in in sliding window 200
The log space of completion status sends the first confirmation to external equipment 201, for being completed outside sliding window 200 in write-in
The log space of state does not send the first confirmation to external equipment 201, it can be ensured that the log space before sliding window 200
It is in writing success status, the daily record entanglement for avoiding writing successful daily record and writing failure is stored in daily record section 300
Situation, the efficiency of log read can be ensured.
Figure 10 is referred to below, and Figure 10 is another flow chart of log processing method according to embodiments of the present invention.Figure 10
Show the concrete mode for the data segment for reading Fig. 2 generations.
As shown in Figure 10, the log processing method implemented according to the present invention also includes：
Step S201：Log processing device 100 reads the origin log space of daily record section 300.
Step S202：Log processing device 100 judges whether it is to read invalid data first in the daily record section 300,
If it is, step S203 is performed, if not, performing step S204.
Step S203：Log processing device 100 reads K-1 after the log space with invalid data one by one
Log space.
Step S204：Log processing device 100 reads next log space in current log space, and jumps to step
S202。
To sum up, end daily record of the log processing device 100 from the origin log space of daily record section 300 to daily record section 300 is empty
Between read each log space of daily record section 300 one by one, when reading invalid data first, read one by one be located at have it is invalid
K-1 log space after the log space of data, read so as to complete daily record end.
It can be understood with reference to Fig. 9, in step S201, log processing device 100 reads the origin log at daily record end first
Space 1, and whether the data for judging to read in origin log space 1 are invalid datas, it can be seen from Fig. 9, log space 1
Data are effective daily record, therefore jump to step S204, and next daily record that log processing device 100 reads log space 1 is empty
Between 2, and similar cycle is carried out, so as to read the log space 1 to 3 for storing effective daily record respectively, until reading daily record
Space 4, because what is write in log space 4 is invalid data, therefore log processing device 100 jumps to step S203, daily record
Processing unit 100 reads K-1 log space after the log space with invalid data one by one, due to the reality shown in Fig. 9
Apply in example, K=7, therefore, log processing device 100 reads 6 daily records after the log space 4 with invalid data one by one
Space 5 to 10, when reading invalid data in log space 4,6,7 and 9, the invalid data read is directly abandoned, when
When log space 5,8 and 10 reads effective daily record, effective daily record is recorded, so as in daily record section 300
All effective daily records are read in log space 1,2,3,5,8 and 10.
To sum up, after the embodiment of the present invention concurrently writes daily record into daily record section 300 based on sliding window 200, due to cunning
Log space before dynamic window 200 is in writing success status, therefore efficiently can be read into from daily record section 300
Whole daily records of work(write-in, can effectively ensure reading efficiency.
It is worth noting that, in the present embodiment, used sliding window 200 is the sliding window of regular length, i.e. K
For constant, the big I of K values is set according to being actually needed.
However, it is understood that in other example, used sliding window 200 can be elongated sliding window.
Specifically, K is variable, and K=Kn, wherein n are the slip number of sliding window 200, and Kn represents sliding window 200
Kn log space is covered after carrying out n-th slip, wherein, K0=M/10, and：
Work as N>When=4,
Work as N<When 4,
It is worth noting that, in other example, can also be rightWithUsing to
On round.
By the way that under different load pressures, dynamic adjusts the size of sliding window 200, can effectively improve log system
Concurrent write performance, while lift the efficiency of log read.
To sum up, in embodiments of the present invention, due to having used sliding window 200, and limit for locating in sliding window 200
The first confirmation is sent to external equipment 201 in the log space of write-in completion status, is write for being in outside sliding window 200
The log space for entering completion status does not send the first confirmation to external equipment 201, because sliding window is relative to single day
The scope covered for will space is bigger, therefore can reduce the loss for writing concurrency.Further, due to sliding window 200
Log space before is in writing success status, therefore can ensure the efficiency of log read.
Figure 11 is referred to below, Figure 11 is the apparatus structure schematic diagram of log processing device according to embodiments of the present invention,
As shown in figure 11, log processing device 100, including：
Daily record section application module 1001, for application date book section 300, daily record section 300 includes the M daily records arranged in order
Space；
Daily record writing module 1002, for concurrently writing N number of daily record to M log space, wherein, N≤M, and N number of daily record
Produced by external equipment 201；
Sliding window module 1003, for using sliding window 200 from the origin log space of daily record section 300 to daily record section
300 end log space slides, wherein, sliding window 200 covers K log space, K<M；
Confirmation sending module 1004, for for each daily record in write-in completion status in sliding window 200
Space sends the first confirmation to external equipment 201 respectively so that in each of write-in completion status in sliding window 200
Log space is in write-in success status, is not sent out for each log space in write-in completion status outside sliding window 200
The first confirmation is sent to external equipment 201；
Wherein, the first confirmation is used to notify the daily record of external equipment 201 to write successfully.
Alternatively, sliding window module 1003, is specifically used for：
When write-in success status is in the origin log space of sliding window 200, by sliding window 200 to daily record section
300 end log space enters line slip, until the new origin log space of sliding window 200 is in write-not success status
Other states.
Alternatively, K is variable, and K=Kn, wherein n be sliding window 200 slip number, KnRepresent sliding window 200
K is covered after carrying out n-th slipnIndividual log space, wherein, K0=M/10, and：
Work as N>When=4,
Work as N<When 4,
Alternatively, confirmation sending module 1004, is additionally operable to：
When being in other states of write-not success status in the origin log space of sliding window 200, in sliding window
200 origin log spatially writes invalid data, after invalid data is write, for the origin log of sliding window 200
Space sends the second confirmation to external equipment 201, and waits the origin log that sliding window 200 is removed in sliding window 200
Other log spaces outside space complete daily record write-in, wherein, the second confirmation is used to notify the daily record of external equipment 201 to write
Failure.
Alternatively, log processing device 100 also includes log read module, log read module, is used for：
Daily record section 300 is read one by one from the origin log space of daily record section 300 to the end log space of daily record section 300
Each log space；
When reading invalid data first, K-1 be located at after the log space with invalid data are read one by one
Log space.
Figure 12 is referred to below, and Figure 12 is the apparatus structure schematic diagram of server according to embodiments of the present invention, such as Figure 12
Shown, server 10, including processor 1021, memory 1022 and bus 1023, processor 1021 and memory 1022 divide
It is not connected with bus 1023, memory 1022 has program stored therein instruction, and computing device programmed instruction is make it that server 10 is real
The function of existing above-mentioned disclosed log processing device 100.
To sum up, in embodiments of the present invention, due to having used sliding window 200, and limit for locating in sliding window 200
The first confirmation is sent to external equipment 201 in the log space of write-in completion status, is write for being in outside sliding window 200
The log space for entering completion status does not send the first confirmation to external equipment 201, because sliding window is relative to single day
The scope covered for will space is bigger, therefore can reduce the loss for writing concurrency.Further, due to sliding window 200
Log space before is in writing success status, therefore can ensure the efficiency of log read.
It should be noted that the processor in the embodiment of the present application can be central processing unit (English：central
Processing unit, abbreviation：CPU), network processing unit (English：Network processor, abbreviation：NP) or CPU and
NP combination.Processor can further include hardware chip.Above-mentioned hardware chip can be application specific integrated circuit (English：
Application-specific integrated circuit, abbreviation：ASIC), PLD (English：
Programmable logic device, abbreviation：PLD) or it is combined.Above-mentioned PLD can be CPLD
(English：Complex programmable logic device, abbreviation：CPLD), field programmable gate array (English：
Field-programmable gate array, abbreviation：FPGA), GAL (English：generic array
Logic, abbreviation：GAL) or it is combined.
It should be noted that the memory in the embodiment of the present application can include volatile memory (English：volatile
Memory), such as random access memory is (English：Random-access memory, abbreviation：RAM)；Memory can also wrap
Include nonvolatile memory (English：Non-volatile memory), such as read-only storage (English：read-only
Memory, abbreviation：ROM), flash memory (English：Flash memory), hard disk (English：Hard disk drive, contracting
Write：HDD) or solid state hard disc is (English：Solid-state drive, abbreviation：SSD)；Memory can also include mentioned kind
The combination of memory.
Based on identical inventive concept, the application provides a kind of computer-readable recording medium, described computer-readable to deposit
Instruction is stored with storage media, when run on a computer so that computer performs any possible in above-described embodiment
Method described in implementation.
Based on identical inventive concept, the application provides a kind of computer program product for including instruction, when it is being calculated
When being run on machine so that computer performs the method described in any possible implementation in above-described embodiment.
It should be understood by those skilled in the art that, embodiments herein can be provided as method, system or computer program
Product.Therefore, the application can use the reality in terms of complete hardware embodiment, complete software embodiment or combination software and hardware
Apply the form of example.Moreover, the application can use the computer for wherein including computer usable program code in one or more
The shape for the computer program product implemented on storage channel (including but is not limited to magnetic disk storage and optical memory etc.) can be used
Formula.
The application is with reference to the flow according to the method for the embodiment of the present application, equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that can be by every first-class in computer program instructions implementation process figure and/or block diagram
Journey and/or the flow in square frame and flow chart and/or block diagram and/or the combination of square frame.These computer programs can be provided
The processors of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing devices is instructed to produce
A raw machine instruction so that use is produced by the instruction of computer or the computing device of other programmable data processing devices
In the dress for realizing the function of being specified in one flow of flow chart or multiple flows and/or one square frame of block diagram or multiple square frames
Put.
These computer program instructions, which may be alternatively stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that the instruction being stored in the computer-readable memory, which produces, to be included referring to
Make the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one square frame of block diagram or
The function of being specified in multiple square frames.
These computer program instructions can be also loaded into computer or other programmable data processing devices so that counted
Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented processing, so as in computer or
The instruction performed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in individual square frame or multiple square frames.
Obviously, those skilled in the art can carry out the essence of various changes and modification without departing from the application to the application
God and scope.So, if these modifications and variations of the application belong to the scope of the application claim and its equivalent technologies
Within, then the application is also intended to comprising including these changes and modification.
Claims (11)
- A kind of 1. log processing method, it is characterised in that including：Application date book section, the daily record section include the M log spaces arranged in order；N number of daily record is concurrently write to the M log space, wherein, N≤M, and N number of daily record is produced by external equipment；Slided using end log space of the sliding window from the origin log space of the daily record section to the daily record section, its In, the sliding window covers K log space, K<M；The first confirmation is sent respectively to institute for each log space that write-in completion status is in the sliding window State external equipment so that each log space in write-in completion status in the sliding window is in write-in success status, The first confirmation is not sent to the outside for each log space in write-in completion status outside the sliding window Equipment；Wherein, first confirmation is used to notify the external equipment daily record to write successfully.
- 2. according to the method for claim 1, it is characterised in that the starting date using sliding window from the daily record section Will space terminad log space slides, and specifically includes：When write-in success status is in the origin log space of the sliding window, by the sliding window to the daily record section End log space enter line slip, until the new origin log space of the sliding window is in write-not success status Other states.
- 3. method according to claim 1 or 2, it is characterised in that the K is variable, and the K=Kn, wherein n is institute State the slip number of sliding window, KnRepresent that the sliding window covers K after carrying out n-th slipnIndividual log space, wherein, K0 =M/10, and：Work as N>When=4,Work as N<When 4,
- 4. according to the method described in any one of claims 1 to 3, it is characterised in that methods described also includes：When being in other states of write-not success status in the origin log space of the sliding window, in the sliding window Origin log spatially write invalid data, after the invalid data is write, for the starting date of the sliding window Will space sends the second confirmation to the external equipment, and waits the starting that the sliding window is removed in the sliding window Other log spaces outside log space complete daily record write-in, wherein, second confirmation is used to notify the outside to set Standby daily record write-in failure.
- 5. according to the method for claim 4, it is characterised in that methods described also includes：End log space from from the origin log space of the daily record section to the daily record section reads the daily record section one by one Each log space；When reading invalid data first, K-1 be located at after the log space with the invalid data are read one by one Log space.
- A kind of 6. log processing device, it is characterised in that including：Daily record section application module, for application date book section, the daily record section includes the M log spaces arranged in order；Daily record writing module, for concurrently writing N number of daily record to the M log space, wherein, N≤M, and N number of daily record Produced by external equipment；Sliding window module, for using sliding window from the origin log space of the daily record section to the end of the daily record section Log space slides, wherein, the sliding window covers K log space, K<M；Confirmation sending module, for distinguishing for each log space that write-in completion status is in the sliding window The first confirmation is sent to the external equipment so that each daily record in write-in completion status in the sliding window is empty Between in write-in success status, do not send first for each log space in write-in completion status outside the sliding window Confirmation is to the external equipment；Wherein, first confirmation is used to notify the external equipment daily record to write successfully.
- 7. device according to claim 6, it is characterised in that the sliding window module, be specifically used for：When write-in success status is in the origin log space of the sliding window, by the sliding window to the daily record section End log space enter line slip, until the new origin log space of the sliding window is in write-not success status Other states.
- 8. the device according to claim 6 or 7, it is characterised in that the K is variable, and the K=Kn, wherein n is institute State the slip number of sliding window, KnRepresent that the sliding window covers K after carrying out n-th slipnIndividual log space, wherein, K0 =M/10, and：Work as N>When=4,Work as N<When 4,
- 9. according to the device described in any one of claim 6 to 8, it is characterised in that the confirmation sending module, also use In：When being in other states of write-not success status in the origin log space of the sliding window, in the sliding window Origin log spatially write invalid data, after the invalid data is write, for the starting date of the sliding window Will space sends the second confirmation to the external equipment, and waits the starting that the sliding window is removed in the sliding window Other log spaces outside log space complete daily record write-in, wherein, second confirmation is used to notify the outside to set Standby daily record write-in failure.
- 10. device according to claim 9, it is characterised in that also including log read module, the log read mould Block, it is used for：End log space from from the origin log space of the daily record section to the daily record section reads the daily record section one by one Each log space；When reading invalid data first, K-1 be located at after the log space with the invalid data are read one by one Log space.
- 11. a kind of server, it is characterised in that including memory and processor, the memory storage has programmed instruction, described Computing device described program is instructed to cause the step described in the server perform claim requirement 1 to 5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710948597.6A CN107844405B (en) | 2017-10-12 | 2017-10-12 | Log processing method and device and server |
| PCT/CN2018/109880 WO2019072212A1 (en) | 2017-10-12 | 2018-10-11 | Log processing method and device and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710948597.6A CN107844405B (en) | 2017-10-12 | 2017-10-12 | Log processing method and device and server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107844405A true CN107844405A (en) | 2018-03-27 |
| CN107844405B CN107844405B (en) | 2021-11-19 |
Family
ID=61661291
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710948597.6A Active CN107844405B (en) | 2017-10-12 | 2017-10-12 | Log processing method and device and server |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107844405B (en) |
| WO (1) | WO2019072212A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019072212A1 (en) * | 2017-10-12 | 2019-04-18 | 华为技术有限公司 | Log processing method and device and server |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1499868A (en) * | 2002-11-08 | 2004-05-26 | 深圳市中兴通讯股份有限公司 | Method for downloading data based on control instructions along with roate |
| CN101179529A (en) * | 2007-12-12 | 2008-05-14 | 北京亿企通信息技术有限公司 | Method and system for reminding user to inscribe log in enterprise instant communication instrument |
| CN102521405A (en) * | 2011-12-26 | 2012-06-27 | 中国科学院计算技术研究所 | Massive structured data storage and query methods and systems supporting high-speed loading |
| CN103309767A (en) * | 2012-03-08 | 2013-09-18 | 阿里巴巴集团控股有限公司 | Method and device for processing client log |
| CN103347086A (en) * | 2013-07-11 | 2013-10-09 | 南京大学 | Synergy kernel construction method based on distributed type negotiation algorithm |
| CN103645859A (en) * | 2013-11-19 | 2014-03-19 | 华中科技大学 | Disk array caching method for virtual SSD and SSD isomerous mirror image |
| US20140081983A1 (en) * | 2004-03-09 | 2014-03-20 | Versata Development Group, Inc. | Session-Based Processing Method and System |
| CN104156432A (en) * | 2014-08-08 | 2014-11-19 | 四川九成信息技术有限公司 | File access method |
| CN104391679A (en) * | 2014-11-18 | 2015-03-04 | 浪潮电子信息产业股份有限公司 | GPU (graphics processing unit) processing method for high-dimensional data stream in irregular stream |
| CN104484295A (en) * | 2014-12-16 | 2015-04-01 | 中国人民解放军国防科学技术大学 | Receiver sliding window-based data transmission method in parallel computer system |
| CN104571956A (en) * | 2014-12-29 | 2015-04-29 | 成都致云科技有限公司 | Data writing method and splitting device |
| CN104657366A (en) * | 2013-11-18 | 2015-05-27 | 深圳市腾讯计算机系统有限公司 | Method and device for writing mass logs in database and log disaster-tolerant system |
| US9088808B1 (en) * | 2008-02-08 | 2015-07-21 | Google Inc. | User interaction based related videos |
| CN106484565A (en) * | 2016-09-22 | 2017-03-08 | 华为数字技术（成都）有限公司 | Method of data synchronization between multiple data centers and relevant device |
| CN107193898A (en) * | 2017-05-09 | 2017-09-22 | 中国科学院计算技术研究所 | The inquiry sharing method and system of log data stream based on stepped multiplexing |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107844405B (en) * | 2017-10-12 | 2021-11-19 | 华为技术有限公司 | Log processing method and device and server |
-
2017
- 2017-10-12 CN CN201710948597.6A patent/CN107844405B/en active Active
-
2018
- 2018-10-11 WO PCT/CN2018/109880 patent/WO2019072212A1/en active Application Filing
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1499868A (en) * | 2002-11-08 | 2004-05-26 | 深圳市中兴通讯股份有限公司 | Method for downloading data based on control instructions along with roate |
| US20140081983A1 (en) * | 2004-03-09 | 2014-03-20 | Versata Development Group, Inc. | Session-Based Processing Method and System |
| CN101179529A (en) * | 2007-12-12 | 2008-05-14 | 北京亿企通信息技术有限公司 | Method and system for reminding user to inscribe log in enterprise instant communication instrument |
| US9088808B1 (en) * | 2008-02-08 | 2015-07-21 | Google Inc. | User interaction based related videos |
| CN102521405A (en) * | 2011-12-26 | 2012-06-27 | 中国科学院计算技术研究所 | Massive structured data storage and query methods and systems supporting high-speed loading |
| CN103309767A (en) * | 2012-03-08 | 2013-09-18 | 阿里巴巴集团控股有限公司 | Method and device for processing client log |
| CN103347086A (en) * | 2013-07-11 | 2013-10-09 | 南京大学 | Synergy kernel construction method based on distributed type negotiation algorithm |
| CN104657366A (en) * | 2013-11-18 | 2015-05-27 | 深圳市腾讯计算机系统有限公司 | Method and device for writing mass logs in database and log disaster-tolerant system |
| CN103645859A (en) * | 2013-11-19 | 2014-03-19 | 华中科技大学 | Disk array caching method for virtual SSD and SSD isomerous mirror image |
| CN104156432A (en) * | 2014-08-08 | 2014-11-19 | 四川九成信息技术有限公司 | File access method |
| CN104391679A (en) * | 2014-11-18 | 2015-03-04 | 浪潮电子信息产业股份有限公司 | GPU (graphics processing unit) processing method for high-dimensional data stream in irregular stream |
| CN104484295A (en) * | 2014-12-16 | 2015-04-01 | 中国人民解放军国防科学技术大学 | Receiver sliding window-based data transmission method in parallel computer system |
| CN104571956A (en) * | 2014-12-29 | 2015-04-29 | 成都致云科技有限公司 | Data writing method and splitting device |
| CN106484565A (en) * | 2016-09-22 | 2017-03-08 | 华为数字技术（成都）有限公司 | Method of data synchronization between multiple data centers and relevant device |
| CN107193898A (en) * | 2017-05-09 | 2017-09-22 | 中国科学院计算技术研究所 | The inquiry sharing method and system of log data stream based on stepped multiplexing |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019072212A1 (en) * | 2017-10-12 | 2019-04-18 | 华为技术有限公司 | Log processing method and device and server |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107844405B (en) | 2021-11-19 |
| WO2019072212A1 (en) | 2019-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI604308B (en) | Data storage device and data maintenance method thereof | |
| JP5002201B2 (en) | Memory system | |
| TWI375962B (en) | Data writing method for flash memory and storage system and controller using the same | |
| KR101366960B1 (en) | Multi way nand flash controller and method using input/output pins | |
| US20130311712A1 (en) | Control apparatus, storage device, and storage control method | |
| US20080109627A1 (en) | Nonvolatile Memory Device And Method For Accessing Nonvolatile Memory Device | |
| CN114661223A (en) | Managing write removals for solid state drives | |
| US20150149741A1 (en) | Storage System and Control Method Thereof | |
| JP6391061B2 (en) | How to write a file on tape | |
| US20210034301A1 (en) | System and method for flash storage management using multiple open page stripes | |
| US7657697B2 (en) | Method of controlling a semiconductor memory device applied to a memory card | |
| EP1313025A2 (en) | Direct memory access transfer of data and check information to and from a data storage device | |
| CN103389942A (en) | Control device, storage device, and storage control method | |
| US20090307427A1 (en) | Memory card and method of writing data | |
| US10365834B2 (en) | Memory system controlling interleaving write to memory chips | |
| TWI608488B (en) | Data storage device and data accessing method thereof | |
| CN107844405A (en) | Log processing method and device, server | |
| CN104598171B (en) | Array method for reconstructing and device based on metadata | |
| DE112019005506T5 (en) | SYSTEMS AND METHODS FOR FINDING A LAST GOOD PAGE IN AN OPEN NAND BLOCK | |
| TW201227294A (en) | Dada writing method, memory controller and memory storage apparatus | |
| CN115933994A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN103389943A (en) | Control device, storage device, and storage control method | |
| CN112433672A (en) | Solid state disk reading method and device | |
| CN103577113B (en) | A kind of memory unit access method | |
| CN105706067A (en) | Microcomputer with built-in flash memory, method for writing data to built-in flash memory of microcomputer, and program for writing data to flash memory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |