CN106648467A - Log generation method and system - Google Patents
Log generation method and system Download PDFInfo
- Publication number
- CN106648467A CN106648467A CN201611239218.8A CN201611239218A CN106648467A CN 106648467 A CN106648467 A CN 106648467A CN 201611239218 A CN201611239218 A CN 201611239218A CN 106648467 A CN106648467 A CN 106648467A
- Authority
- CN
- China
- Prior art keywords
- file
- daily record
- log
- binary
- character
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a log generation method. The log generation method comprises the following steps: using a log pre-processing tool to scan a preset sound code file, and detecting whether a WriteLog key word exists; when the WriteLog key word is detected, extracting a character in a corresponding log descriptor parameter and storing the extracted characters in a log character data mapping file, simultaneously generating a unique ID value and substituting the ID value to the character in the log descriptor parameter, and forming a new sound code file; compiling the new sound code file to generate an object code for releasing operation, and generating a binary system log file. According to the method, the character constant in the program is extracted in advance when the log is written, and subsequently the operation for character data can be reduced, so that the performance can be improved, and the size of the log file is reduced. The invention also discloses a log generation system, and the system has the above beneficial effects.
Description
Technical field
The present invention relates to technical field of data processing, more particularly to a kind of daily record generation method and system.
Background technology
Application program or serve end program operationally, due to the complicated and problem investigation or business diary record etc. of business
It is required in real time special log information being write in file.So the log recording of program is necessarily used in production environment
The basic fundamental for arriving.It contributes to the effect such as displaying of the investigation of problem and system mode.
It is how all substantially enough for small lot daily record output.But once program is complicated, and
Performance requirement is high, when must export a large amount of log informations again.That conventional log recording can affect on the contrary normal business journey
Sort run.
At present, log recording method can include conventional method, by network push daily record, install additional solid state hard disc method this
Three kinds.Wherein, conventional method：Exactly in the code line for needing output journal, disk I/O is directly operated, by program
Specific information constitute a character string be written in disk.By network push daily record：By network such as udp protocol, by day
To after one or more computer, asynchronous writes daily record data in disk to will data-pushing.Install solid state hard disc method additional：Pass through
Solid state hard disc is installed additional to lift IO performances, the lifting for writing daily record performance is reached.
But all there is corresponding shortcoming in above-mentioned various methods.Wherein, conventional method：Task optimization measure is not taken,
When system carries out a large amount of daily records outputs, can be because write daily record performance affects slowly normal service process performance excessively.On the contrary can
Production is affected because daily record amount is excessive.So often carrying out daily record according to condition to simplify output, but simplify after output, it is past
It is past to cannot get detailed daily record data.Once go wrong.Investigation problem is extremely difficult.And substantial amounts of log recording then can be occupied too
Many resources.By network push daily record：When system carries out a large amount of daily records outputs, network congestion can be caused.Affect normal net
Network is communicated, or even log information can be lost due to network busy.Install solid state hard disc method additional：The method is needed by hardware
Support, expense is expensive, and capacity is little, under the scene for needing to record a large amount of daily records, meeting lose using value because capacity is little.
Therefore, how under conditions of existing hardware is not changed, a large amount of daily records and not shadow are outwards exported with very high performance
The performance of program is rung, is those skilled in the art's technical issues that need to address.
The content of the invention
It is an object of the invention to provide a kind of daily record generation method and system, when daily record is write by the character constant in program
Extract in advance, can subsequently reduce the operation to character data, with improving performance and log file size can be reduced.
To solve above-mentioned technical problem, the present invention provides a kind of daily record generation method, including：
Predetermined sound code file is scanned using log integrity instrument, detects whether there is WriteLog keywords；
When WriteLog keywords are detected, extract the character in corresponding daily record description information parameter and store to day
In will character data mapped file, while generating a unique ID value and the ID values being substituted into the daily record description information
Character in parameter, forms new sound code file；
The new sound code file is compiled into generation object code carries out issue operation, generates binary log text
Part.
Optionally, the generation binary log file, including：
Enter binary streamization according to the incoming reference orders of WriteLog in the new sound code file to process, generate two
Binary data stream；
Binary log file is generated according to the binary data stream.
Optionally, binary log file is generated according to the binary data stream, including：
The binary data stream is directly inserted into buffer queue；
Daily record write thread takes out one by one binary data stream in buffer queue, and is written into disk and generates two
System journal file.
Optionally, the parameter of the WriteLog includes daily record level parameters, daily record description information parameter and variable array
Parameter.
Optionally, this programme also includes：
Increase the predetermined sound code file version identifier, and to the corresponding daily record character of the predetermined sound code file
Data mapped file increases identical version identifier.
Optionally, this programme also includes：
According to the binary log file and the daily record character data mapped file, using journal file interpreter solution
The readable journal file of analysis output.
Optionally, according to the binary log file and the daily record character data mapped file, using journal file
The readable journal file of interpreter parsing output, including：
Obtain the binary log file and the corresponding daily record character data mapped file；
The data being successively read in the binary log file；
Parameters value in the data is parsed, and according to the daily record description information parameter value from the daily record number of characters
According to obtaining corresponding character string in mapped file；
Using Format functions, according to the character string and each described parameter value, target journaling character information is generated,
Form readable journal file and export.
The present invention also provides a kind of log generating system, including：
Scan module, for being scanned to predetermined sound code file using log integrity instrument, detects whether exist
WriteLog keywords；
Replacement module, for when WriteLog keywords are detected, in extracting corresponding daily record description information parameter
Character is simultaneously stored in daily record character data mapped file, while generating a unique ID value and the ID values being substituted into institute
The character in daily record description information parameter is stated, new sound code file is formed；
Collector, for the new sound code file to be compiled into generation object code issue operation is carried out, and is generated
Binary log file.
Optionally, the collector, including：
Binary system fluidizes processing unit, for being entered according to the incoming reference orders of WriteLog in the new sound code file
Binary streamization process, generates binary data stream；
Binary log file generating unit, for generating binary log file according to the binary data stream.
Optionally, this programme also includes：
Parsing module, for according to the binary log file and the daily record character data mapped file, using day
The readable journal file of will command interpreter parsing output.
One kind provided by the present invention discloses daily record generation method, including：Using log integrity instrument to predetermined origin
Code file is scanned, and detects whether there is WriteLog keywords；When WriteLog keywords are detected, extract corresponding
Character and storage in daily record description information parameter is in daily record character data mapped file, while one unique ID value of generation
And ID values are substituted into the character in daily record description information parameter, form new sound code file；New sound code file is compiled
Translating generation object code carries out issue operation, generates binary log file；It can be seen that, the method is when daily record is write by program
Character constant is extracted in advance, can subsequently reduce the operation to character data, with improving performance and can reduce journal file
Size, and alleviate IO burdens；Present invention also offers a kind of log generating system, with above-mentioned beneficial effect, here is no longer
Repeat.
Description of the drawings
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
The accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings in the following description are only this
Inventive embodiment, for those of ordinary skill in the art, on the premise of not paying creative work, can be with basis
The accompanying drawing of offer obtains other accompanying drawings.
The flow chart of the daily record generation method that Fig. 1 is provided by the embodiment of the present invention；
The schematic flow sheet of the daily record generation method that Fig. 2 is provided by the embodiment of the present invention；
The structured flowchart of the log generating system that Fig. 3 is provided by the embodiment of the present invention.
Specific embodiment
The core of the present invention is to provide a kind of daily record generation method and system, when daily record is write by the character constant in program
Extract in advance, can subsequently reduce the operation to character data, with improving performance and log file size can be reduced.
To make purpose, technical scheme and the advantage of the embodiment of the present invention clearer, below in conjunction with the embodiment of the present invention
In accompanying drawing, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is
The a part of embodiment of the present invention, rather than the embodiment of whole.Based on the embodiment in the present invention, those of ordinary skill in the art
The every other embodiment obtained under the premise of creative work is not made, belongs to the scope of protection of the invention.
The method that the present embodiment is provided can outwards export a large amount of daily records and not shadow in program operation with very high performance
Ring the performance of program.And can be significantly reduced journal file than traditional logs recording method again under same log information content
Size.Specifically refer to Fig. 1, the flow chart of the daily record generation method that Fig. 1 is provided by the embodiment of the present invention；The method can be wrapped
Include：
S100, predetermined sound code file is scanned using log integrity instrument, detects whether there is WriteLog passes
Key word；
S110, when WriteLog keywords are detected, extract the character in corresponding daily record description information parameter and deposit
In storing up daily record character data mapped file, while generate a unique ID value and the ID values are substituted into the daily record retouching
The character in information parameter is stated, new sound code file is formed；
Specifically, when the two steps mainly realize writing daily record, the operation to character data is reduced as far as possible, such that it is able to carry
High daily record generates performance.I.e. after the completion of coding, by the daily record description information parameter correspondence in code (i.e. predetermined sound code file)
Character all extract and be stored in independent daily record character data mapped file, while each character to extracting is carried out only
One coding, and this yard is substituted into the character string position of log code, complete whole WriteLog in predetermined sound code file crucial
After replacement at word, new sound code file is formed.
Wherein, the parameter of WriteLog includes daily record level parameters, daily record description information parameter and variable array parameter.
Before recompilating predetermined sound code file, need to carry out some specification peace treaties to WriteLog functions (i.e. WriteLog)
Beam.For example：
Daily record output function is arranged such as：WriteLog(iLeve:integer；const Format:string；const
Args:array of const)。
Wherein, iLeve parameters represent that journal stage not, this journey day is determined by it according to the current setting rank of systematic parameter
Whether will needs is recorded.Format parameters are that the description information of this journey daily record, i.e. a line can be by the string arguments of Format.
Type parameters, such as %d, %f, %s can be included.Args is a variable array, i.e., its inside can have multiple parameters, and
And each parameter can be with difference.
After the completion of written in code, usage log pretreating tool is needed to pre-process the WriteLog in code.Will
SExplain incoming character generates daily record character data mapped file.
Said process is illustrated below by programmed process citing：
Programmer needs at output journal in code, and using the WriteLog functions and Parameter specifications of agreement daily record is carried out
Output.
Such as code below fragment is predetermined sound code file：
Next it is exactly usage log pretreating tool to relating in code after the completion of programmer is all of written in code
And the carrying out exported to daily record pre-processes.The process logic of log integrity instrument is very simple, i.e., all of sound code file is entered
Row scanning (can scan one by one), when WriteLog keywords are detected, automatically by the word in daily record description information parameter
Symbol is extracted and is appended in daily record character data mapped file, while generating a unique ID value is substituted into daily record in current code
At character in description information parameter.Code more than for example can define as follows new sound code file after being processed：
WriteLog (1,1, Stream.Size) // daily record output 1
if Stream.Size>0then
begin
WriteLog (1,2, SizeOf (ZlibCount)) // daily record output 2
Stream.Position:=0；
Stream.ReadBuffer(ZlibCount,SizeOf(ZlibCount))；
GetMem(Buffer,ZlibCount)；
UnZipStream:=TDecompressionStream.Create (Stream)；
Try
UnZipStream.ReadBuffer(Buffer^,ZlibCount)；
Result.WriteBuffer(Buffer^,ZlibCount)；
Result.Position:=0；
Finally
WriteLog (1,3, UnZipStream.SizeOf) // daily record output 3
UnZipStream.Free；
FreeMem(Buffer)；
end；
end；
Can obtain being included in daily record character data mapped file by adding boldface type in code：
" size=%d of the current Stream streams of 1=", " 2=obtains primary flow size=%d ", " 3=is disposed
UnZipStream=%d ".
S120, the new sound code file is compiled generation object code carries out issue operation, generate binary system day
Will file.
Specifically, when daily record is exported, the character constant in program is extracted in advance, then can be lifted in follow-up process
Performance simultaneously reduces log file size and (because in the operation of program, can be exported repeatedly with a line daily record, but longer word
Symbol has been extracted, so reducing file size in very big program and improving performance).
Because daily record output function conventional at present all be unable to do without similar Format functions, need to spell character string
Switch through to change into and be just saved in disk file after a string of character strings.And the operation of character string be in the calculation it is relatively time consuming, it
It is the main cause for affecting program feature.Therefore in order to evade the string operation of daily record output.By the character of time-consuming consumption energy
String operation give miscellaneous equipment on demand go process.So as to further improve the performance of daily record generation.Log information do not entered
Row directly splicing, for example：Writelog (" output journal %d ", 123) traditional method be in code, by 123 value replace into
To daily record data " output journal %d " %d positions.But the present embodiment will no longer do this operation, but by log parameters by suitable
Sequence is preserved.
Because the daily record data being written in program in disk is all directly to be stored using binary system fluidisation, that is, obtain
Be binary log file.It does not have as traditional method, needs the word for all switching to daily record data to understand
Symbol string is stored.It is the content that cannot understand internal record so directly opening journal file.So need the later stage by
This log analysis tool come complete log analysis and problem investigation etc. work.Realize time-consuming daily record character splicing and the row of generation
Except in current operating environment.And can be to go to parse journal file using other machine resources or random time.To calculating money
Source takes has accomplished cleverly to shift.The time-consuming string operation of similar Format when daily record is exported is evaded.(the present invention
When daily record data is stored, it is not necessary to operationally carry out the splicing character string of time-consuming expense performance).It is preferred, therefore, that the life
Can include into binary log file：
Enter binary streamization according to the incoming reference orders of WriteLog in the new sound code file to process, generate two
Binary data stream；
Binary log file is generated according to the binary data stream.
Specifically, fluidization treatment：They are directly entered binary streamization according to incoming reference order to process.For example it is every
The storage format of individual parameter is as follows：
Types of variables (1：Integer 2：Floating type 3：Character type)
Length
Variate-value
For example:WriteLog (1,2, SizeOf (ZlibCount)) assumes that the value of SizeOf (ZlibCount) is 100
Inside is processed into binary stream as shown in Table 1, can be given birth to according to binary data in above-mentioned binary system flow table
Into binary log file.
The binary system flow table of table 1
| Offset address | The value being stored in | Byte length | Implication explanation |
| 0000H | 3 | 4 bytes | The number of parameters of WriteLog |
| 0004H | 1 | 4 bytes | Types of variables |
| 0008H | 4 | 4 bytes | The length information of current variable values |
| 000cH | 1 | 4 bytes | The value of the parameter 1 being stored in WriteLog |
| 0010H | 1 | 4 bytes | Types of variables |
| 0014H | 4 | 4 bytes | The length information of current variable values |
| 0018H | 2 | 4 bytes | The value of the parameter 2 being stored in WriteLog |
| 001cH | 1 | 4 bytes | Types of variables |
| 0020H | 4 | 4 bytes | The length information of current variable values |
| 0024H | 100 | 4 bytes | The value of the parameter 3 being stored in WriteLog |
Further for IO efficiency is improved, the present embodiment writes direct disk when daily record is write, or not due to the speed of disk I/O
Degree is slow, so be first written in the internal memory for having pre-allocated, then asynchronous write disk.Specifically, enter according to described two
Data flow processed generates binary log file can be included：
The binary data stream is directly inserted into buffer queue；
Daily record write thread takes out one by one binary data stream in buffer queue, and is written into disk and generates two
System journal file.
Specifically, initial parameter is simply directly entered binary stream preservation when daily record is exported, is spelled without the need for extra character
Connect etc. and to process, while also without the low Format functions for the treatment of effeciency.So in binary log file record, performance
There is greatly lifting.Fig. 2 is refer to, Fig. 2 gives multiple predetermined sound code files (Fig. 2 middle finger service codes) and generates daily record
Schematic flow sheet.Can be with parallel running between each service code.All respective service code is carried out into WriteLog crucial
Word is scanned, and the object code to generating carries out fluidization treatment, obtains binary stream, and by the asynchronous reading of cache queue
Binary stream data are in disk.
Based on above-mentioned technical proposal, daily record generation method provided in an embodiment of the present invention, when daily record is write by program
Character constant is extracted in advance, can subsequently reduce the operation to character data, with improving performance and can reduce journal file
Size, and alleviate IO burdens.
Due to daily record character data can be formed in the process that the sound code file to form new is compiled to predetermined sound code file
Mapped file.This daily record character data mapped file and predetermined sound code file can be seen according to the description of the various embodiments described above
It is one-to-one.It is therefore necessary to ensure the unification of both of these documents.Generation identical code name can be passed through, and (such as a pair literary
The corresponding code digit of part is identical), it is also possible to recognize identical file etc. by increasing mark.The present embodiment does not enter to this
Row is limited.Further for raising reliability, it is preferred that based on above-mentioned any embodiment, the method can also include：
Increase the predetermined sound code file version identifier, and to the corresponding daily record character of the predetermined sound code file
Data mapped file increases identical version identifier.
Specifically, current goal code is combined because " the daily record character data mapped file " that generate needs, therefore
Needs do corresponding version function, it is impossible to obscure mapped file.Here can be directly version mark to be increased to predetermined sound code file
Know, so as to the object code after the new sound code file that especially generates and corresponding compiling can all have the version identifier.Or
Person is directly to increase version identifier to object code.
It is not final log information due to what is preserved in binary log file, it is similar to being a kind of unprocessed form
Daily record data.Therefore it can not be by direct reading.So needing to provide a journal file interpreter.Will according to special algorithm
Binary log file combines daily record character data mapped file, carries out final daily record output.And then it is literary to obtain readable daily record
Part.In order to improve system effectiveness, this resolving live need not be done.Therefore it does not affect actual production environment
Service operation.Above-mentioned any embodiment is based on, the method can also include：According to the binary log file and the day
Will character data mapped file, using the readable journal file of journal file interpreter parsing output.
Specifically, due to recompilating to predetermined sound code file in the various embodiments described above after form new sound code file
And daily record character data mapped file, therefore the journal file for ultimately generating has certain confidentiality, it must get day
Will character data mapped file can be parsed to binary log file, generate readable journal file.Source that will be new
Code file is compiled the final object code of generation and carries out issue operation, and the daily record character data mapped file of generation is carried
Supply log analysis tool is used.
Optionally, according to the binary log file and the daily record character data mapped file, using journal file
The readable journal file of interpreter parsing output can include：
Obtain the binary log file and the corresponding daily record character data mapped file；
The data being successively read in the binary log file；
Parameters value in the data is parsed, and according to the daily record description information parameter value from the daily record number of characters
According to obtaining corresponding character string in mapped file；
Using Format functions, according to the character string and each described parameter value, target journaling character information is generated,
Form readable journal file and export.
Specifically, from journal file stream, by bar parameter is taken out.By the ID values in parameter (Format), you can from day
The information of daily record output original in program is got in will character data mapped file.Then further according to other parameters,
The type parameters that can be updated in current string, ultimately form a log information understood that can be read.
Unify extraction due to character string of the present embodiment when coding stage just exporting daily record to go out, just subtract naturally
Time-consuming string operation is lacked, while in program code and time-consuming Format sentences need not be called to spell character string
Connect operation.And the output of daily record is directly to be written in memory cache using two influent stream modes.Therefore daily record in the present embodiment
A large amount of output performances be high, even if a large amount of daily records export in the case of also hardly affect service code fortune
OK, due to journal file in a large amount of character strings do not exist, so the file of output is more much smaller than original, reduce
The I/O operation of disk.Daily record simultaneously is also equipped with certain confidentiality.
Based on above-mentioned technical proposal, the daily record generation method that the embodiment of the present invention is carried, in the condition for not changing existing hardware
And when needing to export a large amount of log informations in the program that high-performance is concurrently run greatly, can be reached with few resource occupation
High performance log recording.Simultaneously the disk space shared by it can less and on business procedure code runnability affect
It is minimum.Meanwhile, the log information of generation also has certain confidentiality.
Log generating system provided in an embodiment of the present invention is introduced below, log generating system described below with
Above-described daily record generation method can be mutually to should refer to.
Refer to Fig. 3, the structured flowchart of the log generating system that Fig. 3 is provided by the embodiment of the present invention；The system can be with
Including：
Scan module 100, for being scanned to predetermined sound code file using log integrity instrument, detects whether exist
WriteLog keywords；
Replacement module 200, for when WriteLog keywords are detected, in extracting corresponding daily record description information parameter
Character and store in daily record character data mapped file, while generate a unique ID value and the ID values are substituted into
Character in the daily record description information parameter, forms new sound code file；
Collector 300, for the new sound code file to be compiled into generation object code issue operation is carried out, raw
Into binary log file.
Optionally, the collector 300 can include：
Binary system fluidizes processing unit, for being entered according to the incoming reference orders of WriteLog in the new sound code file
Binary streamization process, generates binary data stream；
Binary log file generating unit, for generating binary log file according to the binary data stream.
Based on above-mentioned any embodiment, the system can also include：
Mark module, for increasing the predetermined sound code file version identifier, and to the predetermined sound code file correspondence
The daily record character data mapped file increase identical version identifier.
Based on above-mentioned any embodiment, the system can also include：
Parsing module, for according to the binary log file and the daily record character data mapped file, using day
The readable journal file of will command interpreter parsing output.
Each embodiment is described by the way of progressive in specification, and what each embodiment was stressed is and other realities
Apply the difference of example, between each embodiment identical similar portion mutually referring to.For device disclosed in embodiment
Speech, because it is corresponded to the method disclosed in Example, so description is fairly simple, related part is referring to method part illustration
.
Professional further appreciates that, with reference to the unit of each example of the embodiments described herein description
And algorithm steps, can with electronic hardware, computer software or the two be implemented in combination in, in order to clearly demonstrate hardware and
The interchangeability of software, according to function has generally described the composition and step of each example in the above description.These
Function is performed with hardware or software mode actually, depending on the application-specific and design constraint of technical scheme.Specialty
Technical staff can use different methods to realize described function to each specific application, but this realization should not
Think beyond the scope of this invention.
The step of method described with reference to the embodiments described herein or algorithm, directly can be held with hardware, processor
Capable software module, or the combination of the two is implementing.Software module can be placed in random access memory (RAM), internal memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In field in known any other form of storage medium.
A kind of daily record generation method provided by the present invention and system are described in detail above.It is used herein
Specific case is set forth to the principle and embodiment of the present invention, and the explanation of above example is only intended to help and understands this
The method and its core concept of invention.It should be pointed out that for those skilled in the art, without departing from this
On the premise of bright principle, some improvement and modification can also be carried out to the present invention, these are improved and modification also falls into power of the present invention
In the protection domain that profit is required.
Claims (10)
1. a kind of daily record generation method, it is characterised in that include：
Predetermined sound code file is scanned using log integrity instrument, detects whether there is WriteLog keywords；
When WriteLog keywords are detected, extract the character in corresponding daily record description information parameter and store to daily record word
In symbol data mapped file, while generating a unique ID value and the ID values being substituted into the daily record description information parameter
In character, form new sound code file；
The new sound code file is compiled into generation object code carries out issue operation, generates binary log file.
2. daily record generation method according to claim 1, it is characterised in that the generation binary log file, including：
Enter binary streamization according to the incoming reference orders of WriteLog in the new sound code file to process, generate binary system
Data flow；
Binary log file is generated according to the binary data stream.
3. daily record generation method according to claim 2, it is characterised in that generate two according to the binary data stream and enter
Journal file processed, including：
The binary data stream is directly inserted into buffer queue；
Daily record write thread takes out one by one binary data stream in buffer queue, and is written into disk and generates binary system
Journal file.
4. daily record generation method according to claim 3, it is characterised in that the parameter of the WriteLog includes journal stage
Other parameter, daily record description information parameter and variable array parameter.
5. daily record generation method according to claim 4, it is characterised in that also include：
Increase the predetermined sound code file version identifier, and to the corresponding daily record character data of the predetermined sound code file
Mapped file increases identical version identifier.
6. the daily record generation method according to any one of claim 1-5, it is characterised in that also include：
According to the binary log file and the daily record character data mapped file, parsed using journal file interpreter defeated
Go out readable journal file.
7. daily record generation method according to claim 6, it is characterised in that according to the binary log file and described
Daily record character data mapped file, using the parsing of journal file interpreter readable journal file is exported, including：
Obtain the binary log file and the corresponding daily record character data mapped file；
The data being successively read in the binary log file；
Parameters value in the data is parsed, and is reflected from the daily record character data according to the daily record description information parameter value
Penetrate in file and obtain corresponding character string；
Using Format functions, according to the character string and each described parameter value, target journaling character information is generated, formed
Readable journal file is simultaneously exported.
8. a kind of log generating system, it is characterised in that include：
Scan module, for being scanned to predetermined sound code file using log integrity instrument, detects whether exist
WriteLog keywords；
Replacement module, for when WriteLog keywords are detected, extracting the character in corresponding daily record description information parameter
And store in daily record character data mapped file, while generating a unique ID value and the ID values being substituted into the day
Character in will description information parameter, forms new sound code file；
Collector, for the new sound code file to be compiled into generation object code issue operation is carried out, and is generated two and is entered
Journal file processed.
9. log generating system according to claim 8, it is characterised in that the collector, including：
Binary system fluidizes processing unit, for carrying out two according to the incoming reference orders of WriteLog in the new sound code file
System fluidization treatment, generates binary data stream；
Binary log file generating unit, for generating binary log file according to the binary data stream.
10. log generating system according to claim 8 or claim 9, it is characterised in that also include：
Parsing module, for according to the binary log file and the daily record character data mapped file, using daily record text
The readable journal file of part interpreter parsing output.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611239218.8A CN106648467A (en) | 2016-12-28 | 2016-12-28 | Log generation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611239218.8A CN106648467A (en) | 2016-12-28 | 2016-12-28 | Log generation method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106648467A true CN106648467A (en) | 2017-05-10 |
Family
ID=58836480
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611239218.8A Pending CN106648467A (en) | 2016-12-28 | 2016-12-28 | Log generation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106648467A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108021368A (en) * | 2017-12-11 | 2018-05-11 | 北京奇虎科技有限公司 | Characterize the method, apparatus and terminal device of operating system nucleus |
| CN108958652A (en) * | 2018-06-25 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of method, apparatus and computer readable storage medium of record log information |
| CN110309050A (en) * | 2019-05-22 | 2019-10-08 | 深圳壹账通智能科技有限公司 | Detection method, device, server and the storage medium of code specification |
| CN110474807A (en) * | 2019-08-16 | 2019-11-19 | 北京云中融信网络科技有限公司 | A kind of log processing method and device |
| CN111881012A (en) * | 2020-08-10 | 2020-11-03 | 深圳大势智能科技有限公司 | Log recording method, device, equipment and storage medium |
| CN111935237A (en) * | 2020-07-15 | 2020-11-13 | 展讯通信（上海）有限公司 | Log processing method and system, electronic device and storage medium |
| CN112100018A (en) * | 2019-06-17 | 2020-12-18 | 腾讯科技（深圳）有限公司 | Log information generation method and related device |
| CN112381519A (en) * | 2020-11-20 | 2021-02-19 | 北京云族佳科技有限公司 | Method and device for processing work logs and readable storage medium |
| CN112559458A (en) * | 2020-12-11 | 2021-03-26 | 深圳金信诺高新技术股份有限公司 | Method and system for reducing log size |
| CN112749139A (en) * | 2020-12-30 | 2021-05-04 | 咪咕文化科技有限公司 | Log file processing method, electronic device and storage medium |
| CN112749061A (en) * | 2021-01-16 | 2021-05-04 | 苏州浪潮智能科技有限公司 | Log recording method and system |
| CN113076060A (en) * | 2021-03-10 | 2021-07-06 | 杭州又拍云科技有限公司 | Single-point log storage method |
| CN113497787A (en) * | 2020-03-20 | 2021-10-12 | 阿里巴巴集团控股有限公司 | File transmission method, equipment and system |
| CN113535529A (en) * | 2021-07-22 | 2021-10-22 | 中国银联股份有限公司 | Service log analysis method and device and computer readable storage medium |
| CN113760839A (en) * | 2020-11-25 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Log data compression processing method and device, electronic equipment and storage medium |
| CN113836031A (en) * | 2021-09-29 | 2021-12-24 | 长江存储科技有限责任公司 | System, method, apparatus, and medium for embedded system testing |
| CN114116367A (en) * | 2020-08-25 | 2022-03-01 | 荣耀终端有限公司 | Method for generating log and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101178709A (en) * | 2006-11-08 | 2008-05-14 | 中兴通讯股份有限公司 | Diary record system and method |
| CN102508778A (en) * | 2011-11-16 | 2012-06-20 | 山东浪潮齐鲁软件产业股份有限公司 | Method for detecting whether log is directly output without log output level judgment |
| CN103092742A (en) * | 2011-10-31 | 2013-05-08 | 国际商业机器公司 | Optimization method and system of program logging |
-
2016
- 2016-12-28 CN CN201611239218.8A patent/CN106648467A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101178709A (en) * | 2006-11-08 | 2008-05-14 | 中兴通讯股份有限公司 | Diary record system and method |
| CN103092742A (en) * | 2011-10-31 | 2013-05-08 | 国际商业机器公司 | Optimization method and system of program logging |
| CN102508778A (en) * | 2011-11-16 | 2012-06-20 | 山东浪潮齐鲁软件产业股份有限公司 | Method for detecting whether log is directly output without log output level judgment |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108021368A (en) * | 2017-12-11 | 2018-05-11 | 北京奇虎科技有限公司 | Characterize the method, apparatus and terminal device of operating system nucleus |
| CN108021368B (en) * | 2017-12-11 | 2021-09-07 | 北京奇虎科技有限公司 | Method and device for representing kernel of operating system and terminal equipment |
| CN108958652A (en) * | 2018-06-25 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of method, apparatus and computer readable storage medium of record log information |
| CN110309050A (en) * | 2019-05-22 | 2019-10-08 | 深圳壹账通智能科技有限公司 | Detection method, device, server and the storage medium of code specification |
| CN112100018A (en) * | 2019-06-17 | 2020-12-18 | 腾讯科技（深圳）有限公司 | Log information generation method and related device |
| CN112100018B (en) * | 2019-06-17 | 2023-10-13 | 腾讯科技（深圳）有限公司 | Log information generation method and related device |
| CN110474807A (en) * | 2019-08-16 | 2019-11-19 | 北京云中融信网络科技有限公司 | A kind of log processing method and device |
| CN110474807B (en) * | 2019-08-16 | 2022-06-21 | 北京云中融信网络科技有限公司 | Log processing method and device |
| CN113497787A (en) * | 2020-03-20 | 2021-10-12 | 阿里巴巴集团控股有限公司 | File transmission method, equipment and system |
| CN111935237A (en) * | 2020-07-15 | 2020-11-13 | 展讯通信（上海）有限公司 | Log processing method and system, electronic device and storage medium |
| CN111935237B (en) * | 2020-07-15 | 2023-02-21 | 展讯通信（上海）有限公司 | Log processing method and system, electronic device and storage medium |
| CN111881012A (en) * | 2020-08-10 | 2020-11-03 | 深圳大势智能科技有限公司 | Log recording method, device, equipment and storage medium |
| CN114116367A (en) * | 2020-08-25 | 2022-03-01 | 荣耀终端有限公司 | Method for generating log and electronic equipment |
| CN112381519A (en) * | 2020-11-20 | 2021-02-19 | 北京云族佳科技有限公司 | Method and device for processing work logs and readable storage medium |
| CN113760839A (en) * | 2020-11-25 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Log data compression processing method and device, electronic equipment and storage medium |
| CN112559458B (en) * | 2020-12-11 | 2021-11-30 | 深圳金信诺高新技术股份有限公司 | Method and system for reducing log size |
| CN112559458A (en) * | 2020-12-11 | 2021-03-26 | 深圳金信诺高新技术股份有限公司 | Method and system for reducing log size |
| CN112749139A (en) * | 2020-12-30 | 2021-05-04 | 咪咕文化科技有限公司 | Log file processing method, electronic device and storage medium |
| CN112749139B (en) * | 2020-12-30 | 2024-04-19 | 咪咕文化科技有限公司 | Log file processing method, electronic equipment and storage medium |
| CN112749061A (en) * | 2021-01-16 | 2021-05-04 | 苏州浪潮智能科技有限公司 | Log recording method and system |
| CN113076060A (en) * | 2021-03-10 | 2021-07-06 | 杭州又拍云科技有限公司 | Single-point log storage method |
| CN113535529A (en) * | 2021-07-22 | 2021-10-22 | 中国银联股份有限公司 | Service log analysis method and device and computer readable storage medium |
| CN113836031A (en) * | 2021-09-29 | 2021-12-24 | 长江存储科技有限责任公司 | System, method, apparatus, and medium for embedded system testing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106648467A (en) | Log generation method and system | |
| CN105447099B (en) | Log-structuredization information extracting method and device | |
| CN110383319A (en) | Large scale scale heterogeneous data intake and user's parsing | |
| US20110087669A1 (en) | Composite locality sensitive hash based processing of documents | |
| US20130124796A1 (en) | Storage method and apparatus which are based on data content identification | |
| US7958133B2 (en) | Application conversion of source data | |
| CN102804168A (en) | Data Compression For Reducing Storage Requirements In A Database System | |
| CN113312108B (en) | SWIFT message verification method and device, electronic equipment and storage medium | |
| CN104572781A (en) | Method and device for generating transaction log | |
| CN105706092A (en) | Methods and systems of four-valued simulation | |
| CN102880615A (en) | Data storage method and device | |
| CN106970939A (en) | A kind of database audit method and its system | |
| US9830326B2 (en) | Identifying data offsets using binary masks | |
| US8131728B2 (en) | Processing large sized relationship-specifying markup language documents | |
| CN100527131C (en) | Interdynamic access method and tool of IMS data base | |
| CN109446167A (en) | A kind of storage of daily record data, extracting method and device | |
| US20220198133A1 (en) | System and method for validating tabular summary reports | |
| US8463759B2 (en) | Method and system for compressing data | |
| CN111930708A (en) | Extension system and method of object tag based on Ceph object storage | |
| CN114186233A (en) | Code anti-obfuscation method, device, electronic equipment and storage medium | |
| US11556840B2 (en) | High-speed scanning parser for scalable collection of statistics and use in preparing data for machine learning | |
| CN112905462A (en) | Method and device for processing data in simulation | |
| CN107506156B (en) | Io optimization method of block device | |
| CN101553800A (en) | Migration apparatus which convert SAM/VSAM files of mainframe system into SAM/VSAM files of open system and method for thereof | |
| Indig et al. | Gut, besser, chunker–selecting the best models for text chunking with voting |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170510 |
|
| RJ01 | Rejection of invention patent application after publication |