CN101931562B - Web log processing method and device - Google Patents
Web log processing method and device Download PDFInfo
- Publication number
- CN101931562B CN101931562B CN 201010296462 CN201010296462A CN101931562B CN 101931562 B CN101931562 B CN 101931562B CN 201010296462 CN201010296462 CN 201010296462 CN 201010296462 A CN201010296462 A CN 201010296462A CN 101931562 B CN101931562 B CN 101931562B
- Authority
- CN
- China
- Prior art keywords
- daily record
- log types
- log
- types
- resolve
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a web log processing method and a web log processing device. The web log processing method comprises the following steps of: configuring rule files in network management equipment; storing field information of many log types in the rule files; and searching the rule files for a matched log type according to the field information of the received logs and resolving the logs. When the method is used for resolving the logs, if new log types are added, the logs of the newly added log types can be resolved by simply adding the field information of the new log types into the rule files, so that the problem of long upgrading period of a log processing software edition is solved.
Description
Technical field
The present invention relates to the communications field, relate in particular to a kind of processing method and equipment of network log.
Background technology
In network management system, Network Management Equipment obtains the network log of each managed device, and the network behavior of each managed device is monitored.For the network equipment that different vendor produces, the type of its network log can have nothing in common with each other.A lot of instrument factory commercial cities be its network equipment network log statistical definition a kind of even multiple journal format.
For Network Management Equipment, may get access to various types of network logs and analyze.Existing Network Management Equipment normally carries out code parsing and processing at every kind of different daily records of different vendor.When occurring a kind of new journal format or new daily record template on certain managed device, must come adaptive at the new log processing software of Network Management Equipment configuration.As shown in Figure 1, when usage log form on the managed device or the one issue daily record of daily record template, Network Management Equipment usage log process software version one carries out log processing; When the newly-increased journal format of managed device or the two issue daily records of daily record template, Network Management Equipment needs two pairs of journal formats of usage log process software version or daily record template one and journal format or daily record template two to carry out log processing; In like manner, for can adaptive journal format or daily record template N, must provide new log processing software version N again, the log processing software of different editions can only adaptive limited journal format or daily record template.But the upgrade cycle of log processing software version is long, can't in time satisfy user's demand; And the log analysis software that repeats to buy different vendor has also increased user's investment.
Summary of the invention
The invention provides a kind of processing method and equipment of network log, solve the long problem of upgrade cycle of log processing software version in the prior art.
The invention provides a kind of processing method of network log, be applied to the Network Management Equipment of network management system, comprising:
Steps A, configuration rule file on the described Network Management Equipment, the field information of the multiple Log Types of storage in this rule file;
Step B, described Network Management Equipment obtains the field information of the daily record that receives, and searches the Log Types that mates with the field information that obtains, and uses the Log Types that finds to resolve daily record;
The Log Types of the described field information coupling of searching and obtaining specifically comprises:
When receiving the daily record of set form, described Network Management Equipment is resolved daily record according to the current Log Types in the rule file, reads the Log Types identification field value in the daily record; Wherein, when the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, use described current Log Types to resolve daily record; The Log Types identification field value of the Log Types identification field value in daily record and described current Log Types uses next Log Types to resolve daily record not simultaneously;
After described Network Management Equipment received the daily record template, described Network Management Equipment was searched the Log Types of the daily record that can transmit described daily record template correspondence in rule file; When if each field corresponding with described daily record template is included in the described daily record template in the Log Types that described network management equipment judges goes out to find, described Network Management Equipment uses the Log Types that finds to resolve the daily record of described daily record template correspondence.
When the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, uses described current Log Types to resolve daily record and comprise:
During the value of the message total length field in reading daily record, judge whether the value of this field and the message total length field value of described current Log Types mate, if coupling then uses described current Log Types to resolve daily record; Otherwise use next Log Types to resolve daily record.
Also comprise before the step B:
Step C after described Network Management Equipment receives daily record, searches the Log Types corresponding with the transmit leg of described daily record in the corresponding relation of stored log type and daily record transmit leg, if search success then use the Log Types that finds to resolve daily record; Otherwise execution in step B.
Described Network Management Equipment is the corresponding relation of storing daily record type and daily record transmit leg in advance; The perhaps Log Types that finds among the described Network Management Equipment storing step B and the corresponding relation of daily record transmit leg.
A kind for the treatment of facility of network log comprises:
Dispensing unit is used for the configuration rule file, the field information of the multiple Log Types of storage in this rule file;
Resolution unit is connected with described dispensing unit, is used for the field information of the daily record that the equipment that obtains receives, and searches the Log Types with the field information coupling of obtaining, and uses the described Log Types parsing daily record that the unit finds of searching;
Wherein, described resolution unit specifically is used for:
When equipment receives the daily record of set form, resolve daily record according to the current Log Types in the rule file, read the Log Types identification field value in the daily record, wherein, when the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, use described current Log Types to resolve daily record, the Log Types identification field value of the Log Types identification field value in daily record and described current Log Types uses next Log Types to resolve daily record not simultaneously;
After equipment receives the daily record template, in rule file, search the Log Types of the daily record that can transmit described daily record template correspondence; When each field corresponding with described daily record template is included in the described daily record template in judging the Log Types that finds, use the Log Types that finds to resolve the daily record of described daily record template correspondence.
Described resolution unit also is used for:
During the value of the message total length field in reading daily record, judge whether the value of this field and the message total length field value of described current Log Types mate, if coupling then uses described current Log Types to resolve daily record; Otherwise use next Log Types to resolve daily record.
Also comprise:
Memory cell is for the corresponding relation of storing daily record type and daily record transmit leg;
Search the unit, be connected with described memory cell, be used for after equipment receives daily record, searching the Log Types corresponding with the transmit leg of daily record according to described corresponding relation;
Described resolution unit also is connected with the described unit of searching, and is used for describedly when searching the unit and searching successfully, uses the described Log Types parsing daily record that the unit finds of searching.
Described memory cell also is used for:
The corresponding relation of storing daily record type and daily record transmit leg in advance; Perhaps store Log Types that described resolution unit finds and the corresponding relation of daily record transmit leg.
Compared with prior art, the present invention has the following advantages at least:
Configuration rule file on the Network Management Equipment, the field information of the multiple Log Types of storage in this rule file, the Log Types of searching coupling according to the field information of the daily record that receives in rule file carries out the daily record parsing, when carrying out the daily record parsing by this method, if when increasing Log Types newly, then only need in rule file, increase the field information of new Log Types, to realize the daily record parsing to newly-increased Log Types, avoid the long problem of upgrade cycle of log processing software version, in time satisfy user's demand, avoid repeating to buy the log analysis software of different vendor, save user cost.
Description of drawings
Fig. 1 is the log processing software version update schematic diagram of Network Management Equipment in the prior art;
Fig. 2 is to the daily record resolving schematic diagram of first kind network log in the method provided by the invention;
Fig. 3 is to the daily record resolving schematic diagram of the second class network log in the method provided by the invention;
Fig. 4 is the structural representation of the treatment facility of network log provided by the invention.
Embodiment
For the processing method of network log provided by the invention clearly is described, at first introduce the type of existing network log below.
Though the form of network log is a lot, is divided into two big classes substantially:
First kind network log is the set form daily record, comprises V1, V5, V7, the V8 of NetFlow, the V5 of NetStream, V8, and daily records such as NAT, FLOW, ACCESS, these journal formats are all more fixing.The journal format of NetFlow V5 for example, the form of every daily record shown in a daily record as shown in table 1 form and the table 2, wherein the size of each field and implication are determined, can not change.
Table 1
Table?B-3?Version?5?Header?Format
Table 2
Table?B-4?Version?5?Flow?Record?Format
Second class is the daily record based on template, such as NetFlow V9, and NetStream V9, IPFIX etc., such daily record is more flexible, can carry different network logs.Equipment can send a template definition earlier before sending network log to Network Management Equipment, this template definition is told in the daily record of the follow-up transmission of daily record receiver of Network Management Equipment will comprise which information and the position of each information in message and the byte number that takies.
In the processing method of network log provided by the invention, rule file of definition on the Network Management Equipment, the daily record mapping ruler of the multiple Log Types of storage in this rule file, the daily record mapping ruler of each Log Types comprises the field information of this Log Types.Concrete, this rule file is as follows:
<？xml?version＝″1.0″encoding＝″gb2312″？>
<RcvLogMapRule?xmlns:xsi＝″http://www.w3.org/2001/XMLSchema-instance″
xsi:noNamespaceSchemaLocation＝″RcvLogMapRule.xsd″>
＜!--this document has defined the mapping ruler of daily record, comprise position in original log of field name, this field,--
-->
<！---->
＜!--can define a plurality of log in logs, each log can comprise following field:--〉
＜!--1) type: Log Types can be NAT, FLOW, ACCESS, NETSTREAM etc.-->
＜!--2) canApplyNsv9: can this Log Types use NetStreamV9 transmission, 0 expression can not, otherwise can--
＜!--3) fields: define all field informations of a kind of Log Types, can comprise a plurality of field.-->
-->
<！--5)memInd-->
＜!--5) memIndex: the index of this field, wherein-1 expression skips over this field, and memIndex can not equal 0--
＜!--6) length: this field shared byte length in the original log agreement.-->
-->
＜!--1-table--〉
＜!--1-represents source IP address; 2-represents purpose IP address; 3-represents common IP; 4-represents MAC--〉
-->
＜!--10 expression beginning--〉
＜!--10 expression time starteds; 11 expression concluding times; 12-represents total duration;--
-->
＜!--20-represents--〉
＜!--20-represents Inbound message number; 21-represents the outgoing packet number;--
＜!--22-represents the Inbound byte number; 23-expresses the direction byte number;--
＜!--24-represents the Inbound fluxion; 25-expresses the direction fluxion;--
＜!--30-represents reserved field; The general integer of 31-; The general character string of 32-;--
＜!--33-represents the original message field; 34-represents 64 long fields;--
＜!--36-represents signless integer; 35-daily record direction, value are the full Intranet of 1 representative, and value is 0 for other situation;--
＜!--37-represents the daily record version information; 38-daily record bar number--〉
＜!--39-represents percentage (form of numeral+percentage sign); 40-represents that value is empty field--〉
<logs>
<log>
<type>NSV5</type>
<canApplyNsv9>1</canApplyNsv9>
<head>
<field>
<name>version</name>
<memIndex>-2<memIndex>
<length>2</length>
<type>37</type>
<checkValue>5</checkValue>
</field>
<field>
<name>count</name>
<memIndex>-2<memIndex>
<length>2</length>
<type>38</type>
</field>
<field>
<name>sysUptime</name>
<memIndex>-2<memIndex>
<length>4</length>
<type>31</type>
</field>
<field>
<name>unixSecs</name>
<memIndex>-1<memIndex>
<length>4</length>
<type>31</type>
</field>
<field>
<name>unixNSecs</name>
<memIndex>-1<memIndex>
<length>4</length>
<type>31</type>
</field>
<field>
<name>flowSequence</name>
<memIndex>-1<memIndex>
<length>4</length>
<type>31</type>
</field>
<field>
<name>engineType</name>
<memIndex>-1<memIndex>
<length>1</length>
<type>31</type>
</field>
<field>
<name>engineID</name>
<memIndex>-1<memIndex>
<length>1</length>
<type>31</type>
</field>
<field>
<name>samplingInterval</name>
<memIndex>-2<memIndex>
<length>2</length>
<type>31</type>
</field>
</head>
<fields>
<field>
<name>srcIp</name>
<memIndex>1</memIndex>
<Nsv9Index>8</Nsv9Index>
<length>4</length>
<type>1</type>
</field>
<field>
<name>dstIp</name>
<memIndex>2</memIndex>
<Nsv9Index>12</Nsv9Index>
<length>4</length>
<type>2</type>
</field>
<field>
<name>nexthop</name>
<memIndex>3</memIndex>
<Nsv9Index>15</Nsv9Index>
<length>4</length>
<type>3</type>
</field>
<field>
<name>ifindex</name>
<memIndex>4</memIndex>
<Nsv9Index>58</Nsv9Index>
<length>2</length>
<type>31</type>
</field>
<field>
<name>ofindex</name>
<memIndex>5</memIndex>
<Nsv9Index>59</Nsv9Index>
<length>2</length>
<type>31</type>
</field>
<field>
<name>outPackets</name>
<memIndex>6</memIndex>
<Nsv9Index>2</Nsv9Index>
<length>4</length>
<type>21</type>
</field>
<field>
<name>outOctets</name>
<memIndex>7</memIndex>
<Nsv9Index>1</Nsv9Index>
<length>4</length>
<type>23</type>
</field>
<field>
<name>startTime</name>
<memIndex>8</memIndex>
<Nsv9Index>22</Nsv9Index>
<length>4</length>
<exprValue>$CurrentTime-(sysUptime-startTime)/1000</exprValue>
<type>10</type>
</field>
<field>
<name>endTime</name>
<memIndex>9</memIndex>
<Nsv9Index>21</Nsv9Index>
<length>4</length>
<exprValue>$CurrentTime-(sysUptime-endTime)/1000</exprValue>
<type>11</type>
</field>
<field>
<name>srcPort</name>
<memIndex>10</memIndex>
<Nsv9Index>7</Nsv9Index>
<length>2</length>
<type>31</type>
</field>
<field>
<name>dstPort</name>
<memIndex>11</memIndex>
<Nsv9Index>11</Nsv9Index>
<length>2</length>
<type>31</type>
</field>
<field>
<name>pad1</name>
<memIndex>-1</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>1</length>
<type>31</type>
</field>
<field>
<name>tcpFlags</name>
<memIndex>12</memIndex>
<Nsv9Index>6</Nsv9Index>
<length>1</length>
<type>31</type>
</field>
<field>
<name>prot</name>
<memIndex>13</memIndex>
<Nsv9Index>4</Nsv9Index>
<length>1</length>
<type>31</type>
</field>
<field>
<name>tos</name>
<memIndex>14</memIndex>
<Nsv9Index>5</Nsv9Index>
<length>1</length>
<type>31</type>
</field>
<field>
<name>srcAs</name>
<memIndex>15</memIndex>
<Nsv9Index>16</Nsv9Index>
<length>2</length>
<type>31</type>
</field>
<field>
<name>dstAs</name>
<memIndex>16</memIndex>
<Nsv9Index>17</Nsv9Index>
<length>2</length>
<type>31</type>
</field>
<field>
<name>src_Mask</name>
<memIndex>17</memIndex>
<Nsv9Index>9</Nsv9Index>
<length>1</length>
<type>3</type>
</field>
<field>
<name>dst_Mask</name>
<memIndex>18</memIndex>
<Nsv9Index>13</Nsv9Index>
<length>1</length>
<type>3</type>
</field>
<field>
<name>direct</name>
<memIndex>19</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>1</length>
<type>35</type>
</field>
<field>
<name>pad2</name>
<memIndex>-1</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>1</length>
<type>31</type>
</field>
<field>
<name>inPackets</name>
<memIndex>20</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>20</type>
</field>
<field>
<name>inOctets</name>
<memIndex>21</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>22</type>
</field>
<field>
<name>outFlows</name>
<memIndex>22</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>25</type>
</field>
<field>
<name>totaltime</name>
<memIndex>23</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>12</type>
</field>
<field>
<name>hostIP</name>
<memIndex>24</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>5</type>
</field>
<field>
<name>inFlows</name>
<memIndex>25</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>24</type>
</field>
<field>
<name>appId</name>
<memIndex>26</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>40</type>
</field>
</fields>
</log>
</logs>
</RcvLogMapRule>
Below only be the field of a log, similar for the field information of all the other log, can be with reference to above-mentioned form.Wherein, the type under the log label identifies the Log Types of this log correspondence.Whether canApplyNsv9 identifies this daily record can be by NetStream V9 transmission; The head information definition daily record the head form, the fields information definition all field informations of a kind of Log Types, the field label has defined the meaning of each field in the daily record: wherein name indicates the name of field; MemIndex indicates the index of this field in internal memory, if not needing to handle, a field can not use particular value (for example-1) sign, program is read this information and will directly be abandoned, and being illustrated in for particular value-2 (this particular value also can be other) needs to use in the packet parsing process but do not need to be saved in database; Length indicates the byte number that take of this field in the original log message; Type indicates the type of this field, compares and computing for time started, concluding time, informational needs such as source IP, purpose IP, identifies by giving different types; The checkValue field is used for the daily record that verification receives and whether belongs to current Log Types; Nsv9Index identifies the call number (having specified a unique index for each field among the NetStream V9) of this field correspondence in NetStream V9; ExprValue represents that the value of this field need carry out computing, mainly is time started and concluding time in order to calculate daily record, and De $CurrentTime is the valve system current time in the expression formula.
Among the present invention, after Network Management Equipment received daily record, Network Management Equipment obtained the field information of the daily record that receives, and the Log Types that mates with the field information that obtains in the search rule file uses the Log Types that finds to resolve daily record.
Be the processing method that first kind network log (daily record of set form) and the second class network log are introduced network log provided by the invention respectively with this daily record below.
After Network Management Equipment received daily record, if first kind network log, as shown in Figure 2, Network Management Equipment can be searched the Log Types that mates with daily record according to the Log Types identification field, uses the Log Types that finds to resolve daily record.Below be that the checkValue field is that example is introduced with the Log Types identification field, comprising:
Concrete, Network Management Equipment is resolved each bar daily record record one by one according to the resolution rules of rule file and is kept in the assigned indexes of internal memory.Then preserve value after the calculating for the field that expression formula is arranged.
Among the present invention, can also comprise after the step 203:
If the second class network log, Network Management Equipment carries out daily record according to following steps resolves, and as shown in Figure 3, comprising:
Concrete, among the present invention, the value of canApplyNsv9 is to represent that this Log Types can not be by NetStream V9 transmission at 0 o'clock, otherwise represents that this Log Types can be by NetStream V9 transmission.The value of current canApplyNsv9 can be adjusted flexibly, is not limited to 0.
Need to prove, in the method provided by the invention, Network Management Equipment is behind the Log Types of resolving the managed device issue, record the corresponding relation of this managed device and this Log Types, for example Network Management Equipment can be set up the corresponding relation tabulation of Log Types and managed device, after the daily record that receives arbitrary managed device transmission, at first search the Log Types that whether stores this managed device correspondence in this tabulation, if have then use the stored log type to resolve daily record, otherwise handle according to the mode of above-mentioned Fig. 2 or Fig. 3.
In addition, can also set in advance the Log Types of each managed device issue, namely set the Log Types of each the managed device issue under the network management system, and tabulate at the corresponding relation of Network Management Equipment storing daily record type and managed device.
Behind newly-increased Log Types, do not need to develop again new log processing software on the Network Management Equipment, only need in rule file, increase the log field of newly-increased Log Types.
When adopting method provided by the invention, Network Management Equipment can also be handled when resolving daily record, for example is set to ignore the information of some fields, reduces the log processing burden of Network Management Equipment, and for example some field is only resolved and do not stored in the rule file.
The invention provides a kind for the treatment of facility of network log, as shown in Figure 4, comprising:
Dispensing unit 11 is used for the configuration rule file, the field information of the multiple Log Types of storage in this rule file;
Resolution unit 12 is connected with described dispensing unit 11, is used for the field information of the daily record that the equipment that obtains receives, and searches the Log Types with the field information coupling of obtaining, and uses the Log Types that finds to resolve daily record.
Described resolution unit 12 also is used for:
When equipment receives the daily record of set form, resolve daily record according to the current Log Types in the rule file, read the Log Types identification field value in the daily record;
When the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, use described current Log Types to resolve daily record;
The Log Types identification field value of the Log Types identification field value in daily record and described current Log Types uses next Log Types to resolve daily record not simultaneously.
Described resolution unit 12 also is used for:
During the value of the message total length field in reading daily record, judge whether the value of this field and the message total length field value of described current Log Types mate, if coupling then uses described current Log Types to resolve daily record; Otherwise use next Log Types to resolve daily record.
Described resolution unit 12 comprises:
Search subelement 121, be used for after equipment receives the daily record template, in rule file, searching the Log Types of the daily record that can transmit described daily record template correspondence;
Judgment sub-unit 122 is connected with the described subelement 121 of searching, and is used for judging whether Log Types each field corresponding with described daily record template that finds is included in the described daily record template;
Resolve subelement 123, is connected with described judgment sub-unit 122, the judged result that is used for described interpretation subelement is when being, the Log Types that use finds is resolved the daily record of described daily record template correspondence.
This equipment can also comprise:
Memory cell 13 is for the corresponding relation of storing daily record type and daily record transmit leg;
Search unit 14, be connected with described memory cell 13, be used for after equipment receives daily record, searching the Log Types corresponding with the transmit leg of daily record according to described corresponding relation;
Described resolution unit 12 also is connected with the described unit 14 of searching, and is used for describedly when searching the unit and searching successfully, uses the described Log Types parsing daily record that the unit finds of searching.
Described memory cell 13 also is used for: the corresponding relation of storing daily record type and daily record transmit leg in advance; Perhaps store Log Types that described resolution unit finds and the corresponding relation of daily record transmit leg.
Through the above description of the embodiments, those skilled in the art can be well understood to the present invention and can realize by the mode that software adds essential general hardware platform, can certainly pass through hardware, but the former is better execution mode under a lot of situation.Based on such understanding, the part that technical scheme of the present invention contributes to prior art in essence in other words can embody with the form of software product, this computer software product is stored in the storage medium, comprise that some instructions are with so that a computer equipment (can be personal computer, server, the perhaps network equipment etc.) carry out the described method of each embodiment of the present invention.
It will be appreciated by those skilled in the art that accompanying drawing is the schematic diagram of a preferred embodiment, the module in the accompanying drawing or flow process might not be that enforcement the present invention is necessary.
It will be appreciated by those skilled in the art that the module in the device among the embodiment can be distributed in the device of embodiment according to the embodiment description, also can carry out respective change and be arranged in the one or more devices that are different from present embodiment.The module of above-described embodiment can be merged into a module, also can further split into a plurality of submodules.
More than disclosed only be several specific embodiment of the present invention, still, the present invention is not limited thereto, any those skilled in the art can think variation all should fall into protection scope of the present invention.
Claims (8)
1. the processing method of a network log is applied to the Network Management Equipment of network management system, it is characterized in that, comprising:
Steps A, configuration rule file on the described Network Management Equipment, the field information of the multiple Log Types of storage in this rule file;
Step B, described Network Management Equipment obtains the field information of the daily record that receives, and searches the Log Types that mates with the field information that obtains, and uses the Log Types that finds to resolve daily record;
The Log Types of the described field information coupling of searching and obtaining specifically comprises:
When receiving the daily record of set form, described Network Management Equipment is resolved daily record according to the current Log Types in the rule file, reads the Log Types identification field value in the daily record; Wherein, when the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, use described current Log Types to resolve daily record; The Log Types identification field value of the Log Types identification field value in daily record and described current Log Types uses next Log Types to resolve daily record not simultaneously; Wherein, when if the Log Types identification field value of described next Log Types is identical with Log Types identification field value in the described daily record, use described next Log Types to resolve daily record, otherwise, continue to choose Log Types, when the Log Types identification field value of the Log Types of choosing is identical with Log Types identification field value in the described daily record, use the described Log Types of choosing to resolve daily record;
After described Network Management Equipment received the daily record template, described Network Management Equipment was searched the Log Types of the daily record that can transmit described daily record template correspondence in rule file; When if each field corresponding with described daily record template is included in the described daily record template in the Log Types that described network management equipment judges goes out to find, described Network Management Equipment uses the Log Types that finds to resolve the daily record of described daily record template correspondence; Wherein, carry information and the position of each information in message and the byte number that takies that Log Types, daily record comprise in the described daily record template.
2. the method for claim 1 is characterized in that, when the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, uses described current Log Types to resolve daily record and comprises:
During the value of the message total length field in reading daily record, when judging the message total length field value coupling of the value of this field and described current Log Types, use described current Log Types to resolve daily record.
3. method as claimed in claim 1 or 2 is characterized in that, also comprises before the step B:
Step C after described Network Management Equipment receives daily record, searches the Log Types corresponding with the transmit leg of described daily record in the corresponding relation of stored log type and daily record transmit leg, if search success then use the Log Types that finds to resolve daily record; Otherwise execution in step B.
4. method as claimed in claim 3 is characterized in that, described Network Management Equipment is the corresponding relation of storing daily record type and daily record transmit leg in advance; The perhaps Log Types that finds among the described Network Management Equipment storing step B and the corresponding relation of daily record transmit leg.
5. the treatment facility of a network log is characterized in that, comprising:
Dispensing unit is used for the configuration rule file, the field information of the multiple Log Types of storage in this rule file;
Resolution unit is connected with described dispensing unit, is used for the field information of the daily record that the equipment that obtains receives, and searches the Log Types with the field information coupling of obtaining, and uses the Log Types that finds to resolve daily record;
Wherein, described resolution unit specifically is used for:
When equipment receives the daily record of set form, resolve daily record according to the current Log Types in the rule file, read the Log Types identification field value in the daily record, wherein, when the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, use described current Log Types to resolve daily record, the Log Types identification field value of the Log Types identification field value in daily record and described current Log Types uses next Log Types to resolve daily record not simultaneously; Wherein, when if the Log Types identification field value of described next Log Types is identical with Log Types identification field value in the described daily record, use described next Log Types to resolve daily record, otherwise, continue to choose Log Types, when the Log Types identification field value of the Log Types of choosing is identical with Log Types identification field value in the described daily record, use the described Log Types of choosing to resolve daily record;
After equipment receives the daily record template, in rule file, search the Log Types of the daily record that can transmit described daily record template correspondence; When each field corresponding with described daily record template is included in the described daily record template in judging the Log Types that finds, use the Log Types that finds to resolve the daily record of described daily record template correspondence; Wherein, carry information and the position of each information in message and the byte number that takies that Log Types, daily record comprise in the described daily record template.
6. equipment as claimed in claim 5 is characterized in that, described resolution unit also is used for:
During the value of the message total length field in reading daily record, when judging the message total length field value coupling of the value of this field and described current Log Types, use described current Log Types to resolve daily record.
7. as claim 5 or 6 described equipment, it is characterized in that, also comprise:
Memory cell is for the corresponding relation of storing daily record type and daily record transmit leg;
Search the unit, be connected with described memory cell, be used for after equipment receives daily record, searching the Log Types corresponding with the transmit leg of daily record according to described corresponding relation;
Described resolution unit also is connected with the described unit of searching, and is used for describedly when searching the unit and searching successfully, uses the described Log Types parsing daily record that the unit finds of searching.
8. equipment as claimed in claim 7 is characterized in that, described memory cell also is used for:
The corresponding relation of storing daily record type and daily record transmit leg in advance; Perhaps store Log Types that described resolution unit finds and the corresponding relation of daily record transmit leg.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010296462 CN101931562B (en) | 2010-09-29 | 2010-09-29 | Web log processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010296462 CN101931562B (en) | 2010-09-29 | 2010-09-29 | Web log processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101931562A CN101931562A (en) | 2010-12-29 |
| CN101931562B true CN101931562B (en) | 2013-08-28 |
Family
ID=43370496
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010296462 Active CN101931562B (en) | 2010-09-29 | 2010-09-29 | Web log processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101931562B (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102164050B (en) * | 2011-05-16 | 2014-01-22 | 北京星网锐捷网络技术有限公司 | Log parsing method and log parsing node device |
| CN103929321A (en) * | 2013-01-15 | 2014-07-16 | 腾讯科技（深圳）有限公司 | Log processing method and device |
| CN103368782B (en) * | 2013-07-30 | 2016-08-10 | 浙江中烟工业有限责任公司 | A kind of network status analysis method |
| CN104580310A (en) * | 2013-10-21 | 2015-04-29 | 腾讯科技（深圳）有限公司 | Log processing method and server |
| CN104933077B (en) * | 2014-03-20 | 2018-08-14 | 上海宝信软件股份有限公司 | Rule-based multifile information analysis method |
| CN105447099B (en) * | 2015-11-11 | 2018-12-14 | 中国建设银行股份有限公司 | Log-structuredization information extracting method and device |
| CN106126383B (en) * | 2016-06-01 | 2019-03-19 | 新华三技术有限公司 | A kind of log processing method and device |
| CN106201848A (en) * | 2016-06-30 | 2016-12-07 | 北京奇虎科技有限公司 | The log processing method of a kind of real-time calculating platform and device |
| CN108268471A (en) * | 2016-12-30 | 2018-07-10 | 北京国双科技有限公司 | The read method and device of journal file |
| CN113688006B (en) * | 2017-10-16 | 2024-03-29 | 创新先进技术有限公司 | Log data verification method and device |
| CN107919981A (en) * | 2017-10-31 | 2018-04-17 | 江苏省未来网络创新研究院 | A kind of analysis method of multi-vendor log cache |
| CN108804106B (en) * | 2018-04-28 | 2022-04-19 | 北京机械设备研究所 | Data analysis software optimization method based on configuration table management |
| CN109408479B (en) * | 2018-09-19 | 2023-05-30 | 平安科技（深圳）有限公司 | Log data adding method, system, computer device and storage medium |
| CN110263009B (en) * | 2019-06-21 | 2024-01-16 | 深圳前海微众银行股份有限公司 | Method, device and equipment for generating log classification rule and readable storage medium |
| CN110826299B (en) * | 2019-10-25 | 2023-05-23 | 上海工业自动化仪表研究院有限公司 | General template log analysis method based on classification |
| CN111078657A (en) * | 2019-12-26 | 2020-04-28 | 北京思特奇信息技术股份有限公司 | Service log query method, system, medium and equipment of distributed system |
| CN113810242A (en) * | 2020-06-16 | 2021-12-17 | 中盈优创资讯科技有限公司 | System log analysis method and device |
| CN113407421B (en) * | 2021-08-19 | 2021-11-30 | 北京江融信科技有限公司 | Dynamic log record management method and system for micro-service gateway |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1917445A (en) * | 2006-09-07 | 2007-02-21 | 上海交通大学 | Method for auditing log event of fire wall, and teaching experimental system |
| CN101453378A (en) * | 2008-12-30 | 2009-06-10 | 杭州华三通信技术有限公司 | Method and system for log damp and audit |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1955159B1 (en) * | 2005-11-12 | 2012-07-04 | Logrhythm, Inc. | Log collection, structuring and processing |
-
2010
- 2010-09-29 CN CN 201010296462 patent/CN101931562B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1917445A (en) * | 2006-09-07 | 2007-02-21 | 上海交通大学 | Method for auditing log event of fire wall, and teaching experimental system |
| CN101453378A (en) * | 2008-12-30 | 2009-06-10 | 杭州华三通信技术有限公司 | Method and system for log damp and audit |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101931562A (en) | 2010-12-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101931562B (en) | Web log processing method and device | |
| US9681204B2 (en) | Methods and apparatus to validate a tag for media | |
| US20130191890A1 (en) | Method and system for user identity recognition based on specific information | |
| CN100489879C (en) | Method, system and server for checking page data | |
| US20100333172A1 (en) | Method, apparatus and system for monitoring database security | |
| CN101614781B (en) | Intelligent diagnosis method of radio and television equipment based on spatial rule index | |
| CN107329888A (en) | Intelligent contract command code coverage rate computational methods and system | |
| WO2019161774A1 (en) | Methods, application server, block chain node and media for logistics tracking and source tracing | |
| CN105321108A (en) | System and method for creating a list of shared information on a peer-to-peer network | |
| CN103297267B (en) | A kind of methods of risk assessment of network behavior and system | |
| CN101188561A (en) | Communication method and system for universal service data based on self-defined template | |
| Tan et al. | An end-to-end covert channel via packet dropout for mobile networks | |
| HK1126052A1 (en) | Aggregated resource reservation for data flows | |
| US8700632B2 (en) | Managing heterogeneous data | |
| CN109379326A (en) | XML message rule method of calibration, equipment and storage medium | |
| SE0201315D0 (en) | A method and system of rating in a charging system | |
| KR100817562B1 (en) | Method for indexing a large scaled logfile, computer readable medium for storing program therein, and system for the preforming the same | |
| CN106255082A (en) | The recognition methods of a kind of refuse messages and system | |
| CN109614417B (en) | Data flow-based report index display method and device and terminal | |
| CN100555957C (en) | Method and device thereof that a kind of incident is synthetic | |
| CN101140581A (en) | SQL statement construct method and apparatus of preprocess special-character | |
| CN109522528A (en) | A kind of word document is converted to the method that can calculate automatically html document | |
| US20140337069A1 (en) | Deriving business transactions from web logs | |
| CN100568955C (en) | A kind of VOD method and system | |
| CN102609482A (en) | Packaging method of JSON (JavaScript Object Notiation) data format |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 310052 Binjiang District Changhe Road, Zhejiang, China, No. 466, No. Patentee after: Xinhua three Technology Co., Ltd. Address before: 310053 Hangzhou hi tech Industrial Development Zone, Zhejiang province science and Technology Industrial Park, No. 310 and No. six road, HUAWEI, Hangzhou production base Patentee before: Huasan Communication Technology Co., Ltd. |
|
| CP03 | Change of name, title or address |