CN106055450A - Binary log analysis method and apparatus - Google Patents
Binary log analysis method and apparatus Download PDFInfo
- Publication number
- CN106055450A CN106055450A CN201610342231.XA CN201610342231A CN106055450A CN 106055450 A CN106055450 A CN 106055450A CN 201610342231 A CN201610342231 A CN 201610342231A CN 106055450 A CN106055450 A CN 106055450A
- Authority
- CN
- China
- Prior art keywords
- field
- bodies
- output
- instruction
- daily record
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present invention discloses a binary log analysis method and apparatus. According to the technical scheme of embodiments, users can select field bodies from a preset field body set to form an analysis template according to the preset field body set, the analysis template comprises at least one field body, and each field body comprises an output field name, position information representing a position of a field value of the output field name, length information representing the bit occupied by the field value, and an output field type of the field value; and then binary logs can be analyzed through the analysis template. According to the binary log analysis method, due to the fact that the analysis template is randomly configured by the users according to the field set, log analysis application program is not required to be developed and maintained for specific binary logs, maintenance workload of the analysis template is smaller than that of the analysis application program, and maintenance is convenient.
Description
Technical field
The present invention relates to daily record analytic technique field, particularly relate to a kind of binary log analysis method and device.
Background technology
Along with the development of massive logs analysis mining technology, how the daily record of analyzing device or system enjoys industry to pay attention to.
The purpose that daily record resolves, is the number that the data in daily record to be resolved (i.e. original log) are made into formatting by certain logical groups
According to, in order to it is supplied to down-stream system and is analyzed and excavates.
Original log includes character string daily record and binary log, the character string that compares daily record, when data volume is identical, and two
System daily record volume is little, compressible, network transmission of being more convenient for, but while bringing many advantages, also resolves band to daily record
Carry out a lot of inconvenience.
At present, binary log is varied, is generally directed to a kind of binary log and develops the daily record parsing of a set of correspondence
Application program.Thus, there is following defect in prior art:
1), daily record resolve application program and can only resolve the binary log of correspondence, and each parsing application program
It is required for safeguarding, causes exploitation and maintenance workload big.
2), when data change of format after the parsing of binary log, need developer again to develop daily record and resolve
Application program.
So, the existing binary log analytic method scope of application is little, and binary log resolves the exploitation of application program
Efficiency is low, and the cycle is long, and maintenance workload is big.
Summary of the invention
Embodiments provide a kind of binary log analysis method and device, existing in order to solve to presently, there are
The binary log analytic method scope of application little, binary log resolve application program development efficiency low, the cycle is long, safeguard
The problems such as workload is big.
Embodiments provide a kind of binary log analytic method, including:
Obtain daily record to be resolved；And the parsing template being pre-configured with；Described parsing template includes at least one field bodies,
Each field bodies includes: output field name, represents the positional information of the position of the field value of this output field name, represent this field
The length information of the number of bits shared by value and the output field type of this field value；
For each field bodies, according to positional information and the length information of this field bodies, from described daily record to be resolved
Obtain the binary number that the field value of this field bodies is corresponding；
Described binary number is resolved according to the output field type of this field bodies, obtains field value；
The field value that parsing is obtained output corresponding with output field name.
Further, the embodiment of the present invention additionally provides a kind of binary log resolver, including:
Acquisition module, is used for obtaining daily record to be resolved；And the parsing template being pre-configured with；Described parsing template include to
A few field bodies, each field bodies includes: output field name, represent the position letter of the position of the field value of this output field name
Breath, the length information representing number of bits shared by this field value and the output field type of this field value；
Parsing module, for for each field bodies, according to positional information and the length information of this field bodies, from described
Daily record to be resolved obtains the binary number that the field value of this field bodies is corresponding；
Field value acquisition module, for described binary number is resolved according to the output field type of this field bodies,
Obtain field value；
Output module, the field value output corresponding with output field name for parsing is obtained.
Invention has the beneficial effect that: in technical scheme described in the embodiment of the present invention, and user can be according to preset field body
Set, therefrom selects field bodies to constitute and resolves template, and this parsing template includes at least one field bodies, each field bodies bag
Include: output field name, represent the positional information of the position of the field value of this output field name, represent the bit shared by this field value
The output field type of the length information of figure place and this field value；May then pass through parsing template binary log is entered
Row resolves.It is that user arbitrarily can configure according to set of fields owing to resolving template, so the two of embodiment of the present invention offer
System daily record analytic method, it is not necessary to develop for specific binary log and safeguard that daily record resolves application program, additionally it is possible to
It is suitable for the parsing of the arbitrarily binary log of output data form, safeguards relative to resolving application program resolving template
Maintenance workload is little, and easy to maintenance.
Accompanying drawing explanation
For the technical scheme being illustrated more clearly that in the embodiment of the present invention, in embodiment being described below required for make
Accompanying drawing briefly introduce, it should be apparent that, below describe in accompanying drawing be only some embodiments of the present invention, for this
From the point of view of the those of ordinary skill in field, on the premise of not paying creative work, it is also possible to obtain other according to these accompanying drawings
Accompanying drawing.
Fig. 1 show the schematic flow sheet of binary log analytic method described in the embodiment of the present invention one；
Fig. 2 show the structural representation of field bodies described in the embodiment of the present invention one；
Fig. 3 show the structural representation resolving template described in the embodiment of the present invention one；
Fig. 4 show described in the embodiment of the present invention one another structural representation resolving template；
Fig. 5 show the structural representation of binary log resolver described in the embodiment of the present invention two.
Detailed description of the invention
Embodiments provide a kind of binary log analysis method and device.In technology described in the embodiment of the present invention
In scheme, user can therefrom select field bodies to constitute resolve template according to preset field body set, and this parsing template includes
At least one field bodies, each field bodies includes: output field name, represent the position of the position of the field value of this output field name
Information, the length information representing number of bits shared by this field value and the output field type of this field value；The most permissible
By resolving template, binary log is resolved.It is that user arbitrarily can configure according to set of fields owing to resolving template
, so the binary log analytic method that the embodiment of the present invention provides, it is not necessary to develop for specific binary log and safeguard
Daily record resolves application program, additionally it is possible to be suitable for the parsing of the arbitrarily binary log of output data form, to resolving template
Safeguard relative to resolve application program maintenance workload little and easy to maintenance.
In order to make the object, technical solutions and advantages of the present invention clearer, below in conjunction with accompanying drawing the present invention made into
One step ground describes in detail, it is clear that described embodiment is only a part of embodiment of the present invention rather than whole enforcement
Example.Based on the embodiment in the present invention, those of ordinary skill in the art are obtained under not making creative work premise
All other embodiments, broadly fall into the scope of protection of the invention.
Embodiment one:
As it is shown in figure 1, it is the schematic flow sheet of binary log analytic method described in the embodiment of the present invention one, described
Binary log analytic method can comprise the following steps that
Step 101: obtain daily record to be resolved；And the parsing template being pre-configured with；Described parsing template includes at least one
Individual field bodies, each field bodies includes: output field name, represent the position of the field value of this output field name positional information,
Represent length information and the output field type of this field value of number of bits shared by this field value.
Wherein, output field type includes data type in one embodiment, in network communication protocol definition type etc.
At least one.Any one during data type is e.g. following: int (Integer, integer type), String (character string),
Long (integer), boolean (Boolean type), float (floating type), double (double-precision floating point type), date (date type) etc..
Network communication protocol definition type is e.g. applicable to represent MAC Address (Media Access Control, physical address)
Field type, it is adaptable to represent IPV4 (Internet Protocol Version 4) or the field type of IPV6 (IPv6), the suitableeest
For representing the field type of IP address, field type can also apply to the field type of express time stamp.It is embodied as
Time, any field type having specific format requirement is all applicable to the embodiment of the present invention, and this is not limited by the present invention.
Step 102: for each field bodies, according to positional information and the length information of this field bodies, solve from described waiting
Analysis daily record obtains the binary number that the field value of this field bodies is corresponding.
Wherein, in one embodiment, positional information can be the original position of the binary number that field value is corresponding, terminates
Any one in position or centre position, this is not limited by the present invention.Only can determine field value pair according to positional information
The position of the binary number answered, and determine that according to length information the method for the figure place of binary number is all applicable to the present invention and implements
Example, this is not limited by the present invention.
Step 103: described binary number is resolved according to the output field type of this field bodies, obtains field value.
Step 104: the field value that parsing is obtained output corresponding with output field name.
Wherein, in one embodiment, the daily record to be resolved after parsing, need defeated according to specific data interchange format
Go out, so, in the embodiment of the present invention, resolving in template and can also include preset data interchange format, step 104 specifically can perform
According to resolve the preset data interchange format in template, field value parsing obtained is corresponding with output field name to be exported.Its
In, preset data DIF e.g. XML (Extensible Markup Language, extensible markup language), JSON
(JavaScript Object Notation, JavaScript object representation), YAML (Yet Another Markup
Language, another kind of markup language) etc. in one of which.It should be noted that any data interchange format is all applicable to
The embodiment of the present invention, this is not limited by the present invention.
For ease of understanding, the daily record analytic method provided the embodiment of the present invention below is described further, specifically
, including herein below:
1), about field bodies
Wherein, in one embodiment, in order to can Command Line Parsing template according to the actual requirements so that Command Line Parsing template
Easy to operate, the embodiment of the present invention can pre-define field bodies formed field bodies set, then according to field bodies
Set generates and resolves template.Concrete:
(1), can be according to following methods generation field bodies:
Step A1: receive the establishment instruction for creating field bodies；Described establishment instruction includes field bodies to be created
Output field name, represent the positional information of the position of the field value of this output field name, represent the bit shared by this field value
The length information of number and the output field type of this field value.
Wherein, in one embodiment, as in figure 2 it is shown, wherein, Z1 represents output field to the structural representation of field bodies
Name, Z2 represents that positional information, Z3 represent that length information, Z4 represent output field type, and 201,202,203 all represent decollator.
Such as, if creating the entitled U of output field that instruction includes, the start bit phase of the binary number that positional information field value is corresponding
Side-play amount for preset reference position represents, and this side-play amount is 2, and length information is 3 (represents that field is used by oneself 3 binary systems
Position represents), output field type string, with "；" representing decollator, the field bodies of the most final generation can be expressed as
U；2；3；String
It should be noted that output field name, positional information, length information and the elder generation of output field type in field bodies
Rear order does not limits, and when being embodied as, can set according to actual needs.
When resolving daily record to be resolved with specific reference to field bodies, each information in field bodies can be determined according to separator
(i.e. output field name, positional information, length information and output field type).
Step A2: create the instruction described field bodies to be created of establishment according to described.
Such as, in the embodiment of the present invention, it is provided that the editing interface of User Defined field bodies, user can be by interface
Operation, by the establishment field bodies button in interface, start to create field bodies.After user selects to create field bodies button,
Display requires user's input and output field name, positional information, length information and the dialog box of output field type, and user is permissible
Input corresponding informance by this dialog box, thus complete the establishment of field bodies.
So, in the embodiment of the present invention, user can be according to self being actually needed establishment field bodies, and developer can
Programming language need not be understood, it is also possible to complete to resolve the configuration of template, specifically resolve application relative to prior art is developed
Program, the configuration of parsing template is more simple to operate, hommization.
(2), can be according to the method described parsing template of configuration:
Step B1: receive the selection to the field bodies in preset field body set and instruct.
Step B2: generate described parsing template according to the set of the field bodies selected.
Wherein, in one embodiment, can be using the set of the field bodies selected as resolving template.
So, the configuration resolving template has only to by selecting field bodies to complete, easy and simple to handle.
Wherein, in one embodiment, binary log includes daily record head and daily record body, for the ease of understanding which field
Body belongs to daily record head, which field bodies belong to daily record body with, for each field of described parsing template in the embodiment of the present invention
Body, also includes in described parsing template representing that this field bodies is under the jurisdiction of daily record head and is still under the jurisdiction of the field bodies title of daily record body,
Then step B2 also may particularly include following steps:
Step B21: receive the name instruction that the field bodies selected is given described field bodies title, wrap in this name instruction
Include field bodies entitled daily record leader to know or the instruction information of daily record body mark.
Wherein, instruction information can include that daily record leader is known and daily record body identifies, i.e. if name instruction includes daily record head
Mark, then it represents that the field bodies entitled daily record leader of field bodies is known, and this field bodies is under the jurisdiction of daily record head；If name instruction is wrapped
Include daily record body mark, then it represents that the field bodies entitled daily record body mark of field bodies, this field bodies is under the jurisdiction of daily record body.So,
It is assured that field bodies is under the jurisdiction of daily record head or daily record body according to resolving template.
Step B22: give described field bodies title according to name instruction to the field bodies of selection, and generate described parsing mould
Plate.
Final parsing template will include field head and field bodies two parts content, and concrete parsing template is represented by
As it is shown on figure 3, in Fig. 3, U1 represents that daily record leader is known, 301 is decollator, is used for distinguishing daily record head and daily record body, and U2 represents day
Will body identifies.
Such as, preset field body set is as shown in table 1, and certainly, table 1 is merely to illustrate the embodiment of the present invention and is not used to limit
Determine the embodiment of the present invention.
Table 1 preset field body set example
| Field bodies |
| M1 |
| M2 |
| M3 |
| M4 |
| M5 |
If selecting field bodies M1 and M2 and giving the knowledge of entitled daily record leader, then explanation daily record head includes M1 and M2 two
Field bodies, if selecting field bodies M3, M4, M5, and gives entitled daily record body mark, then it represents that daily record body includes M3, M4, M5
Totally 3 field bodies.The most final parsing template is represented by:
U1 [M1, M2], U2 [M3, M4, M5]
If also including preset data interchange format in parsing template, the most final parsing template is represented by as shown in Figure 4.
In Fig. 4 identical with the implication of same-sign in Fig. 3, do not repeat them here, in Fig. 4 302 represent be used for distinguishing data interchange format
Decollator with U1.
After can distinguishing the field bodies belonging to daily record head and daily record body, in the embodiment of the present invention, step 105 (will resolve
The field value obtained output corresponding with output field name) may particularly include field value that parsing is obtained and output field name, with
And the output of field bodies title correspondence.
Such as table 2 show and just resolves the field value obtained output corresponding with output field name and field bodies title
Example, it should be noted that table 2 is merely to illustrate the embodiment of the present invention, be not intended to limit the present invention embodiment.
Table 2
(3) preset field body set, for the ease of user is safeguarded, in the embodiment of the present invention, it is also possible to receive preset word
Edit instruction in segment body set, described edit instruction include following in any one: the deletion of cancel (CANCL) segment body instruction, in advance
Put the amendment instruction adding instruction, amendment field bodies adding field bodies in field bodies set；And, perform phase according to edit instruction
Should operate.
So, in example of the present invention, can safeguard field bodies set with edit instruction, operation is simple in maintenance.
(4), wherein, in one embodiment, under default situations, a kind of template that resolves can resolve a kind of daily record.In order to really
Protecting and resolving template is to resolve corresponding daily record to be resolved, thus obtains correct analysis result, in the embodiment of the present invention,
Before step 102, it is also possible to determine the bit sum of described daily record to be resolved, with all field values of described parsing template
The length information sum of shared number of bits is identical.If identical, illustrate that daily record to be resolved is the parsing corresponding with resolving template
Daily record, can carry out resolving operation.Wherein, length information sum calculates when can be to load this parsing template, so only meter
Calculating once, certainly, when being embodied as, it is also possible to calculate before resolving each daily record to be resolved, the present invention implements
This is not limited by example.
Underneath with the binary log solution as a example by the more commonly used netflow v5 daily record, the embodiment of the present invention provided
Analysis method illustrates: netflow v5 journal format is known format, and includes herein below, as shown in table 3.In table 3, B
Table 3
| Bytes | Contents |
| 0-1 | version |
| 2-3 | count |
| 4-7 | Sys_uptime |
| 8-11 | unix_secs |
| 12-15 | unix_nsecs |
| 16-19 | flow_sequence |
| 20 | engine_type |
| 21 | engine_id |
| 22-23 | Sampling_interval |
Then the parsing masterplate of the daily record head of netflow v5 is represented by:
head:{length:24,
fields:[{version:INT；2；0},{count:INT；2；2},{sys_uptime:LONG；4；4},
{unix_secs:U NIXTIME；4；8},{unix_nsecs:UNIXTIME；4；12},{flow_sequence:LONG；4；
16},{eng ine_type:SHORT；1；20},{engine_id:SHORT；1；21},{sampling_interval:INT；
2；22}]}
Wherein, length:24 represents that daily record head takies 24 bits.
The method for expressing of the daily record body of the daily record of netflow v5 is identical with daily record, repeats no more here.
When resolving daily record to be resolved, can carry out according to following step, the execution sequence of certain each step is permissible
According to being actually needed setting:
The first step: calculate the bit sum length resolving template: wherein
Length=head.length+body.length
Wherein head.length represents the number of bits shared by daily record head, and body.length represents the ratio that daily record body takies
Special figure place.
Second step: the bit sum of relatively daily record to be resolved is the most equal with length, if equal, represent and resolves mould
Plate is corresponding with daily record to be resolved.
3rd step: analyze and resolve template, obtain all of field bodies and (include field bodies and the field of daily record body of daily record head
Body) form field bodies set, this set is represented by
[M1,M2,…,Mn]
4th step: each field bodies in traversal field bodies set, looks for correspondence according to positional information and length information
The binary number of field value, then resolve binary number and draw field value, field value is assigned to field output name；Then may be used
With the daily record to be resolved after resolving according to preset data interchange format output.
To sum up, in technical scheme described in the embodiment of the present invention, user therefrom can select according to preset field body set
Field bodies constitutes parsing template, and this parsing template includes that at least one field bodies, each field bodies include: output field name,
Represent the positional information of the position of the field value of this output field name, represent that the length of the number of bits shared by this field value is believed
Breath and the output field type of this field value；May then pass through parsing template binary log is resolved.Owing to solving
Analysis template is that user can be according to the set arbitrarily configuration that certainly stews, so the binary log that the embodiment of the present invention provides resolves
Method, it is not necessary to develop for specific binary log and safeguard that daily record resolves application program.Additionally it is possible to be suitable for arbitrarily output
The parsing of the binary log of data form, safeguards relative to the maintenance workload resolving application program resolving template
Little and easy to maintenance.
Embodiment two
Based on identical inventive concept, the embodiment of the present invention also provides for a kind of binary log resolver, such as Fig. 5 institute
Show, for the structural representation of this device, including:
Acquisition module 501, is used for obtaining daily record to be resolved；And the parsing template being pre-configured with；Described parsing template bag
Including at least one field bodies, each field bodies includes: output field name, represent the position of the position of the field value of this output field name
Confidence breath, the length information representing number of bits shared by this field value and the output field type of this field value；
Parsing module 502, for for each field bodies, according to positional information and the length information of this field bodies, from
Described daily record to be resolved obtains the binary number that the field value of this field bodies is corresponding；
Field value acquisition module 503, for solving described binary number according to the output field type of this field bodies
Analysis, obtains field value；
Output module 504, the field value output corresponding with output field name for parsing is obtained.
Wherein, in one embodiment, described device also includes:
Resolve template configuration module, for according to the following methods described parsing template of configuration:
Receive the selection to the field bodies in preset field body set to instruct；
Set according to the field bodies selected generates described parsing template.
Wherein, in one embodiment, for each field bodies of described parsing template, described parsing template also includes
Represent that this field bodies is under the jurisdiction of daily record head and is still under the jurisdiction of the field bodies title of daily record body；Resolve template configuration module, specifically use
In:
Receiving the name instruction that the field bodies selected gives described field bodies title, this name instruction includes field bodies
Entitled daily record leader is known or the instruction information of daily record body mark；
Give described field bodies title according to the name instruction field bodies to selecting, and generate described parsing template.
Wherein, in one embodiment, output module, specifically for:
The field value that parsing is obtained output corresponding with output field name and field bodies title.
Wherein, in one embodiment, described device also includes:
Edit instruction receiver module, for receiving the edit instruction in preset field body set, described edit instruction bag
Include following in any one: cancel (CANCL) segment body delete instruction, add in preset field body set field bodies interpolation instruction,
The amendment instruction of amendment field bodies；
Edit instruction performs module, for performing corresponding operating according to edit instruction.
Wherein, in one embodiment, described device also includes:
Field bodies generation module, for according to following methods generation field bodies:
Receive the establishment instruction for creating field bodies；Described establishment instructs the output field including field bodies to be created
Name, represent the positional information of the position of the field value of this output field name, represent the length of number of bits shared by this field value
The output field type of information and this field value；
The instruction described field bodies to be created of establishment is created according to described.
Wherein, in one embodiment, described device also includes:
Determine module, for determining the bit sum of described daily record to be resolved, with all fields of described parsing template
The length information sum of the number of bits shared by value is identical.
In technical scheme described in the embodiment of the present invention, user can therefrom select field bodies according to preset field body set
Constituting and resolve template, this parsing template includes that at least one field bodies, each field bodies include: output field name, expression should
The positional information of the position of the field value of output field name, represent number of bits shared by this field value length information and
The output field type of this field value；May then pass through parsing template binary log is resolved.Owing to resolving template
It is that user arbitrarily can configure according to the set that certainly stews, so the binary log analytic method that the embodiment of the present invention provides,
Without developing for specific binary log and safeguarding that daily record resolves application program.Data are arbitrarily exported additionally it is possible to be suitable for
The parsing of the binary log of form, safeguards little relative to the maintenance workload resolving application program to resolving template, and
Easy to maintenance.
About the device in above-described embodiment, wherein modules performs the concrete mode of operation in relevant the method
Embodiment in be described in detail, explanation will be not set forth in detail herein.
Those skilled in the art are it should be appreciated that embodiments of the invention can be provided as method, device, system or calculate
Machine program product.Therefore, the present invention can use complete hardware embodiment, complete software implementation or combine software and hardware side
The form of the embodiment in face.And, the present invention can use and wherein include computer usable program code one or more
The upper computer implemented of computer-usable storage medium (including but not limited to disk memory, CD-ROM, optical memory etc.)
The form of program product.
The present invention is with reference to method, device (device) and the flow chart of computer program according to embodiments of the present invention
And/or block diagram describes.It should be understood that can be by each flow process in computer program instructions flowchart and/or block diagram
And/or the flow process in square frame and flow chart and/or block diagram and/or the combination of square frame.These computer programs can be provided to refer to
Order arrives the processor of general purpose computer, special-purpose computer, Embedded Processor or other programmable data processing meanss to produce
One machine so that the instruction performed by the processor of computer or other programmable data processing meanss is produced and is used for realizing
The device of the function specified in one flow process of flow chart or multiple flow process and/or one square frame of block diagram or multiple square frame.
These computer program instructions may be alternatively stored in and computer or other programmable data processing meanss can be guided with spy
Determine in the computer-readable memory that mode works so that the instruction being stored in this computer-readable memory produces and includes referring to
Make the manufacture of device, this command device realize at one flow process of flow chart or multiple flow process and/or one square frame of block diagram or
The function specified in multiple square frames.
These computer program instructions also can be loaded in computer or other programmable data processing meanss so that at meter
Perform sequence of operations step on calculation machine or other programmable devices to produce computer implemented process, thus at computer or
The instruction performed on other programmable devices provides for realizing at one flow process of flow chart or multiple flow process and/or block diagram one
The step of the function specified in individual square frame or multiple square frame.
Although preferred embodiments of the present invention have been described, but those skilled in the art once know basic creation
Property concept, then can make other change and amendment to these embodiments.So, claims are intended to be construed to include excellent
Select embodiment and fall into all changes and the amendment of the scope of the invention.
Obviously, those skilled in the art can carry out various change and the modification essence without deviating from the present invention to the present invention
God and scope.So, if these amendments of the present invention and modification belong to the scope of the claims in the present invention and equivalent technologies thereof
Within, then the present invention is also intended to comprise these change and modification.
Claims (14)
1. a binary log analytic method, it is characterised in that including:
Obtain daily record to be resolved；And the parsing template being pre-configured with；Described parsing template includes at least one field bodies, each
Field bodies includes: output field name, represents the positional information of the position of the field value of this output field name, represent this field value institute
The length information of the number of bits accounted for and the output field type of this field value；
For each field bodies, according to positional information and the length information of this field bodies, obtain from described daily record to be resolved
The binary number that the field value of this field bodies is corresponding；
Described binary number is resolved according to the output field type of this field bodies, obtains field value；
The field value that parsing is obtained output corresponding with output field name.
Method the most according to claim 1, it is characterised in that according to the following methods described parsing template of configuration:
Receive the selection to the field bodies in preset field body set to instruct；
Set according to the field bodies selected generates described parsing template.
Method the most according to claim 2, it is characterised in that for each field bodies of described parsing template, described solution
Analysis template also includes representing that this field bodies is under the jurisdiction of daily record head and is still under the jurisdiction of the field bodies title of daily record body；According to select
The set of field bodies generates described parsing template, specifically includes:
Receiving the name instruction that the field bodies selected gives described field bodies title, this name instruction includes field bodies title
Know for daily record leader or the instruction information of daily record body mark；
Give described field bodies title according to the name instruction field bodies to selecting, and generate described parsing template.
Method the most according to claim 3, it is characterised in that field value parsing obtained is corresponding with output field name defeated
Go out, specifically include:
The field value that parsing is obtained output corresponding with output field name and field bodies title.
Method the most according to claim 2, it is characterised in that described method also includes:
Receive to the edit instruction in preset field body set, described edit instruction include following in any one: delete field
Deleting of body instructs, adds in preset field body set the interpolation instruction of field bodies, the amendment of amendment field bodies instructs；
Corresponding operating is performed according to edit instruction.
Method the most according to claim 1, it is characterised in that according to following methods generation field bodies:
Receive the establishment instruction for creating field bodies；Described create instruction include field bodies to be created output field name,
Represent the positional information of the position of the field value of this output field name, represent that the length of the number of bits shared by this field value is believed
Breath and the output field type of this field value；
The instruction described field bodies to be created of establishment is created according to described.
7. according to described method arbitrary in claim 1-6, it is characterised in that described for each field bodies, according to this word
The positional information of segment body and length information, obtain the binary system that the field value of this field bodies is corresponding from described daily record to be resolved
Before number, described method also includes:
Determine the bit sum of described daily record to be resolved, with the number of bits shared by all field values of described parsing template
Length information sum is identical.
8. a binary log resolver, it is characterised in that including:
Acquisition module, is used for obtaining daily record to be resolved；And the parsing template being pre-configured with；Described parsing template includes at least one
Individual field bodies, each field bodies includes: output field name, represent the position of the field value of this output field name positional information,
Represent length information and the output field type of this field value of number of bits shared by this field value；
Parsing module, for for each field bodies, according to positional information and the length information of this field bodies, solves from described waiting
Analysis daily record obtains the binary number that the field value of this field bodies is corresponding；
Field value acquisition module, for being resolved according to the output field type of this field bodies by described binary number, obtains
Field value；
Output module, the field value output corresponding with output field name for parsing is obtained.
Device the most according to claim 8, it is characterised in that described device also includes:
Resolve template configuration module, for according to the following methods described parsing template of configuration:
Receive the selection to the field bodies in preset field body set to instruct；
Set according to the field bodies selected generates described parsing template.
Device the most according to claim 9, it is characterised in that for each field bodies of described parsing template, described solution
Analysis template also includes representing that this field bodies is under the jurisdiction of daily record head and is still under the jurisdiction of the field bodies title of daily record body；Parsing template is joined
Put module, specifically for:
Receiving the name instruction that the field bodies selected gives described field bodies title, this name instruction includes field bodies title
Know for daily record leader or the instruction information of daily record body mark；
Give described field bodies title according to the name instruction field bodies to selecting, and generate described parsing template.
11. devices according to claim 10, it is characterised in that output module, specifically for:
The field value that parsing is obtained output corresponding with output field name and field bodies title.
12. devices according to claim 9, it is characterised in that described device also includes:
Edit instruction receiver module, for receive to the edit instruction in preset field body set, described edit instruction include with
Any one in Xia: deleting of cancel (CANCL) segment body instructs, adds in preset field body set the interpolation instruction of field bodies, amendment
The amendment instruction of field bodies；
Edit instruction performs module, for performing corresponding operating according to edit instruction.
13. devices according to claim 8, it is characterised in that described device also includes:
Field bodies generation module, for according to following methods generation field bodies:
Receive the establishment instruction for creating field bodies；Described create instruction include field bodies to be created output field name,
Represent the positional information of the position of the field value of this output field name, represent that the length of the number of bits shared by this field value is believed
Breath and the output field type of this field value；
The instruction described field bodies to be created of establishment is created according to described.
14. arbitrary described devices in-13 according to Claim 8, it is characterised in that described device also includes:
Determine module, for determining the bit sum of described daily record to be resolved, with all field value institutes of described parsing template
The length information sum of the number of bits accounted for is identical.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610342231.XA CN106055450B (en) | 2016-05-20 | 2016-05-20 | A kind of binary log analysis method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610342231.XA CN106055450B (en) | 2016-05-20 | 2016-05-20 | A kind of binary log analysis method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106055450A true CN106055450A (en) | 2016-10-26 |
| CN106055450B CN106055450B (en) | 2019-07-02 |
Family
ID=57176621
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610342231.XA Active CN106055450B (en) | 2016-05-20 | 2016-05-20 | A kind of binary log analysis method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106055450B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106682099A (en) * | 2016-12-01 | 2017-05-17 | 北京奇虎科技有限公司 | Data storage method and device |
| CN106682097A (en) * | 2016-12-01 | 2017-05-17 | 北京奇虎科技有限公司 | Method and device for processing log data |
| CN106815306A (en) * | 2016-12-16 | 2017-06-09 | 中铁程科技有限责任公司 | Daily record analysis method and device |
| CN108052590A (en) * | 2017-12-11 | 2018-05-18 | 四川新网银行股份有限公司 | The dynamic application method and system of a kind of structural data |
| CN109995468A (en) * | 2018-01-03 | 2019-07-09 | 凌群电脑股份有限公司 | The variable information composing method of high-effect data length and its system |
| CN110120945A (en) * | 2019-04-23 | 2019-08-13 | 苏州凯迪泰医学科技有限公司 | A kind of method of data structured encapsulation |
| CN110309113A (en) * | 2018-03-06 | 2019-10-08 | 阿里巴巴集团控股有限公司 | Log analytic method, system and equipment |
| CN110362547A (en) * | 2018-04-02 | 2019-10-22 | 阿里巴巴集团控股有限公司 | Coding, parsing, storage method and the device of journal file |
| CN111563123A (en) * | 2020-05-07 | 2020-08-21 | 北京首汽智行科技有限公司 | Live warehouse metadata real-time synchronization method |
| CN114785604A (en) * | 2022-04-28 | 2022-07-22 | 北京安博通金安科技有限公司 | Dynamic log analysis method, device, equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1645336A (en) * | 2005-01-20 | 2005-07-27 | 上海复旦光华信息科技股份有限公司 | Automatic extraction and analysis for formwork based on heterogenerous logbook |
| JP4331440B2 (en) * | 2002-05-10 | 2009-09-16 | キヤノン電子株式会社 | Information management server, information processing apparatus, information management system, control method therefor, and program |
| US7661032B2 (en) * | 2007-01-06 | 2010-02-09 | International Business Machines Corporation | Adjusting sliding window parameters in intelligent event archiving and failure analysis |
| JP2011065440A (en) * | 2009-09-17 | 2011-03-31 | Mitsubishi Denki Information Technology Corp | Log data analysis device and log data analysis method of the same, and log data analysis program |
| CN103544076A (en) * | 2012-07-13 | 2014-01-29 | 阿里巴巴集团控股有限公司 | Data backup method and device |
| CN103929321A (en) * | 2013-01-15 | 2014-07-16 | 腾讯科技（深圳）有限公司 | Log processing method and device |
| CN104615736A (en) * | 2015-02-10 | 2015-05-13 | 上海创景计算机系统有限公司 | Quick analysis and storage method of big data based on database |
| CN105447099A (en) * | 2015-11-11 | 2016-03-30 | 中国建设银行股份有限公司 | Log structured information extraction method and apparatus |
-
2016
- 2016-05-20 CN CN201610342231.XA patent/CN106055450B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4331440B2 (en) * | 2002-05-10 | 2009-09-16 | キヤノン電子株式会社 | Information management server, information processing apparatus, information management system, control method therefor, and program |
| CN1645336A (en) * | 2005-01-20 | 2005-07-27 | 上海复旦光华信息科技股份有限公司 | Automatic extraction and analysis for formwork based on heterogenerous logbook |
| US7661032B2 (en) * | 2007-01-06 | 2010-02-09 | International Business Machines Corporation | Adjusting sliding window parameters in intelligent event archiving and failure analysis |
| JP2011065440A (en) * | 2009-09-17 | 2011-03-31 | Mitsubishi Denki Information Technology Corp | Log data analysis device and log data analysis method of the same, and log data analysis program |
| CN103544076A (en) * | 2012-07-13 | 2014-01-29 | 阿里巴巴集团控股有限公司 | Data backup method and device |
| CN103929321A (en) * | 2013-01-15 | 2014-07-16 | 腾讯科技（深圳）有限公司 | Log processing method and device |
| CN104615736A (en) * | 2015-02-10 | 2015-05-13 | 上海创景计算机系统有限公司 | Quick analysis and storage method of big data based on database |
| CN105447099A (en) * | 2015-11-11 | 2016-03-30 | 中国建设银行股份有限公司 | Log structured information extraction method and apparatus |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106682099A (en) * | 2016-12-01 | 2017-05-17 | 北京奇虎科技有限公司 | Data storage method and device |
| CN106682097A (en) * | 2016-12-01 | 2017-05-17 | 北京奇虎科技有限公司 | Method and device for processing log data |
| CN106815306A (en) * | 2016-12-16 | 2017-06-09 | 中铁程科技有限责任公司 | Daily record analysis method and device |
| CN108052590A (en) * | 2017-12-11 | 2018-05-18 | 四川新网银行股份有限公司 | The dynamic application method and system of a kind of structural data |
| CN109995468A (en) * | 2018-01-03 | 2019-07-09 | 凌群电脑股份有限公司 | The variable information composing method of high-effect data length and its system |
| CN110309113A (en) * | 2018-03-06 | 2019-10-08 | 阿里巴巴集团控股有限公司 | Log analytic method, system and equipment |
| CN110309113B (en) * | 2018-03-06 | 2023-05-26 | 阿里巴巴集团控股有限公司 | Log analysis method, system and equipment |
| CN110362547A (en) * | 2018-04-02 | 2019-10-22 | 阿里巴巴集团控股有限公司 | Coding, parsing, storage method and the device of journal file |
| CN110362547B (en) * | 2018-04-02 | 2023-10-03 | 杭州阿里巴巴智融数字技术有限公司 | Method and device for encoding, analyzing and storing log file |
| CN110120945A (en) * | 2019-04-23 | 2019-08-13 | 苏州凯迪泰医学科技有限公司 | A kind of method of data structured encapsulation |
| CN111563123A (en) * | 2020-05-07 | 2020-08-21 | 北京首汽智行科技有限公司 | Live warehouse metadata real-time synchronization method |
| CN111563123B (en) * | 2020-05-07 | 2023-08-22 | 北京首汽智行科技有限公司 | Real-time synchronization method for hive warehouse metadata |
| CN114785604A (en) * | 2022-04-28 | 2022-07-22 | 北京安博通金安科技有限公司 | Dynamic log analysis method, device, equipment and storage medium |
| CN114785604B (en) * | 2022-04-28 | 2023-11-07 | 北京安博通金安科技有限公司 | Dynamic log analysis method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106055450B (en) | 2019-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106055450A (en) | Binary log analysis method and apparatus | |
| CN104484216B (en) | Service interface document and on-line testing instrument generation method, device | |
| CN106055585A (en) | Log analysis method and apparatus | |
| CN106250104B (en) | A kind of remote operating system for server, method and device | |
| CN109522018A (en) | Page processing method, device and storage medium | |
| CN102163223A (en) | Self-service query method and device thereof for background data | |
| CN106970820A (en) | Code storage method and code storage | |
| Hoeller et al. | Efficient XML usage within wireless sensor networks | |
| CN109522013A (en) | A kind of code generating method and device of business function | |
| CN115567607A (en) | Processing method, device and system for calling link, electronic equipment and storage medium | |
| De Almeida et al. | Exploring perturbation based testing for web services | |
| Cohen et al. | Dynamic Expression Trees | |
| CN105793842B (en) | Conversion method and device between serialized message | |
| CN104090895B (en) | Obtain the method for radix, device, server and system | |
| CN111783391B (en) | Online artificial text marking system and method | |
| CN110928540A (en) | Page generation method and device | |
| CN114764330A (en) | Data blood margin analysis method and device, electronic equipment and computer readable storage medium | |
| CN114089987A (en) | Visual script execution method and device and electronic equipment | |
| CN116244164A (en) | Test data packet generation method and device, electronic equipment and storage medium | |
| Li et al. | Automatic service composition based on process ontology | |
| CN114860566A (en) | Source code testing method and device, electronic equipment and storage medium | |
| CN110888883A (en) | Data storage method, device, system and storage medium | |
| CN112394972A (en) | Cloud application publishing method, device, equipment and storage medium | |
| CN112394912B (en) | Robot application data dynamic extraction method based on ROS (reactive oxygen species) calculation graph | |
| CN113094040B (en) | Applet compiling method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 100089 Beijing city Haidian District Road No. 4 North wa Yitai three storey building Patentee after: NSFOCUS Technologies Group Co.,Ltd. Patentee after: NSFOCUS TECHNOLOGIES Inc. Address before: 100089 Beijing city Haidian District Road No. 4 North wa Yitai three storey building Patentee before: NSFOCUS INFORMATION TECHNOLOGY Co.,Ltd. Patentee before: NSFOCUS TECHNOLOGIES Inc. |
|
| CP01 | Change in the name or title of a patent holder |