CN101931562A - Web log processing method and device - Google Patents
Web log processing method and device Download PDFInfo
- Publication number
- CN101931562A CN101931562A CN 201010296462 CN201010296462A CN101931562A CN 101931562 A CN101931562 A CN 101931562A CN 201010296462 CN201010296462 CN 201010296462 CN 201010296462 A CN201010296462 A CN 201010296462A CN 101931562 A CN101931562 A CN 101931562A
- Authority
- CN
- China
- Prior art keywords
- daily record
- log types
- log
- field
- types
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a web log processing method and a web log processing device. The web log processing method comprises the following steps of: configuring rule files in network management equipment; storing field information of many log types in the rule files; and searching the rule files for a matched log type according to the field information of the received logs and resolving the logs. When the method is used for resolving the logs, if new log types are added, the logs of the newly added log types can be resolved by simply adding the field information of the new log types into the rule files, so that the problem of long upgrading period of a log processing software edition is solved.
Description
Technical field
The present invention relates to the communications field, relate in particular to a kind of processing method and equipment of network log.
Background technology
In network management system, Network Management Equipment obtains the network log of each managed device, and the network behavior of each managed device is monitored.For the network equipment that different vendor produces, the type of its network log can have nothing in common with each other.A lot of instrument factory commercial cities be its network equipment network log statistical definition a kind of even multiple journal format.
For Network Management Equipment, may get access to various types of network logs and analyze.Existing Network Management Equipment normally carries out code parsing and processing at every kind of different daily records of different vendor.When occurring a kind of new journal format or new daily record template on certain managed device, it is adaptive dispose new log processing software on Network Management Equipment.As shown in Figure 1, when usage log form on the managed device or the one issue daily record of daily record template, Network Management Equipment usage log process software version one carries out log processing; When newly-increased journal format of managed device or the two issue daily records of daily record template, Network Management Equipment needs two pairs of journal formats of usage log process software version or daily record template one and journal format or daily record template two to carry out log processing; In like manner, for can adaptive journal format or daily record template N, must provide new log processing software version N again, the log processing software of different editions can only adaptive limited journal format or daily record template.But the upgrade cycle of log processing software version is long, can't in time satisfy user's demand; And the log analysis software that repeats to buy different vendor has also increased user's investment.
Summary of the invention
The invention provides a kind of processing method and equipment of network log, solve the long problem of upgrade cycle of log processing software version in the prior art.
The invention provides a kind of processing method of network log, be applied to the Network Management Equipment of network management system, comprising:
Steps A, configuration rule file on the described Network Management Equipment, the field information of the multiple Log Types of storage in this rule file;
Step B, described Network Management Equipment obtains the field information of the daily record that receives, and the Log Types of the field information coupling of searching and obtaining uses the Log Types that finds to resolve daily record.
The Log Types of the described field information coupling of searching and obtaining comprises:
When receiving the daily record of set form, described Network Management Equipment is resolved daily record according to the current Log Types in the rule file, reads the Log Types identification field value in the daily record;
When the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, use described current Log Types to resolve daily record;
The Log Types identification field value of Log Types identification field value in daily record and described current Log Types uses next Log Types to resolve daily record not simultaneously.
When the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, uses described current Log Types to resolve daily record and comprise:
During the value of the message total length field in reading daily record, judge whether the value of this field and the message total length field value of described current Log Types mate, if coupling then uses described current Log Types to resolve daily record; Otherwise use next Log Types to resolve daily record.
The Log Types of the described field information coupling of searching and obtaining comprises:
After described Network Management Equipment received the daily record template, described Network Management Equipment was searched the Log Types of the daily record that can transmit described daily record template correspondence in rule file;
Whether each field corresponding with described daily record template is included in the described daily record template in the Log Types that described network management equipment judges finds;
If judged result is for being, described Network Management Equipment uses the Log Types that finds to resolve the daily record of described daily record template correspondence.
Also comprise before the step B:
Step C after described Network Management Equipment receives daily record, searches the Log Types corresponding with the transmit leg of described daily record in the corresponding relation of stored log type and daily record transmit leg, if search success then use the Log Types that finds to resolve daily record; Otherwise execution in step B.
Described Network Management Equipment is the corresponding relation of storing daily record type and daily record transmit leg in advance; The Log Types that finds among the perhaps described Network Management Equipment storing step B and the corresponding relation of daily record transmit leg.
A kind of treatment facility of network log comprises:
Dispensing unit is used for the configuration rule file, the field information of the multiple Log Types of storage in this rule file;
Resolution unit is connected with described dispensing unit, is used to the field information of the daily record that the equipment that obtains receives, and the Log Types of the field information of searching and obtaining coupling uses the described Log Types parsing daily record that the unit finds of searching.
Described resolution unit also is used for:
When equipment receives the daily record of set form, resolve daily record according to the current Log Types in the rule file, read the Log Types identification field value in the daily record;
When the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, use described current Log Types to resolve daily record;
The Log Types identification field value of Log Types identification field value in daily record and described current Log Types uses next Log Types to resolve daily record not simultaneously.
Described resolution unit also is used for:
During the value of the message total length field in reading daily record, judge whether the value of this field and the message total length field value of described current Log Types mate, if coupling then uses described current Log Types to resolve daily record; Otherwise use next Log Types to resolve daily record.
Described resolution unit comprises:
Search subelement, be used for after equipment receives the daily record template, in rule file, search the Log Types of the daily record that can transmit described daily record template correspondence;
Judgment sub-unit is connected with the described subelement of searching, and is used for judging whether Log Types each field corresponding with described daily record template that finds is included in the described daily record template;
Resolve subelement, is connected with described interpretation subelement, the judged result that is used for described interpretation subelement is when being, the Log Types that use finds is resolved the daily record of described daily record template correspondence.
Also comprise:
Memory cell is used for the corresponding relation of storing daily record type and daily record transmit leg;
Search the unit, be connected, be used for after equipment receives daily record, search the Log Types corresponding with the transmit leg of daily record according to described corresponding relation with described memory cell;
Described resolution unit also is connected with the described unit of searching, and is used for describedly when searching the unit and searching successfully, uses and describedly searches the Log Types that the unit finds and resolve daily record.
Described memory cell also is used for:
The corresponding relation of storing daily record type and daily record transmit leg in advance; Perhaps store the Log Types that described resolution unit finds and the corresponding relation of daily record transmit leg.
Compared with prior art, the present invention has the following advantages at least:
Configuration rule file on the Network Management Equipment, the field information of the multiple Log Types of storage in this rule file, the Log Types of searching coupling according to the field information of the daily record that receives in rule file carries out the daily record parsing, when carrying out the daily record parsing by this method, if when increasing Log Types newly, then only need in rule file, increase the field information of new Log Types, to realize daily record parsing to newly-increased Log Types, avoid the long problem of upgrade cycle of log processing software version, in time satisfy user's demand, avoid repeating to buy the log analysis software of different vendor, save user cost.
Description of drawings
Fig. 1 is the log processing software version update schematic diagram of Network Management Equipment in the prior art;
Fig. 2 is to the daily record resolving schematic diagram of first kind network log in the method provided by the invention;
Fig. 3 is to the daily record resolving schematic diagram of the second class network log in the method provided by the invention;
Fig. 4 is the structural representation of the treatment facility of network log provided by the invention.
Embodiment
For the processing method of network log provided by the invention clearly is described, at first introduce the type of existing network log below.
Though the form of network log is a lot, is divided into two big classes substantially:
First kind network log is the set form daily record, comprises V1, V5, V7, the V8 of NetFlow, the V5 of NetStream, V8, and daily records such as NAT, FLOW, ACCESS, these journal formats are all more fixing.The journal format of NetFlow V5 for example, the form of every daily record shown in a daily record as shown in table 1 form and the table 2, wherein the size of each field and implication are all determined, can not change.
Table 1
Table?B-3?Version?5?Header?Format
Table 2
Table?B-4?Version?5?Flow?Record?Format
Second class is the daily record based on template, such as NetFlow V9, and NetStream V9, IPFIX etc., such daily record is more flexible, can carry different network logs.Equipment can send a template definition earlier before sending network log to Network Management Equipment, this template definition is told in the daily record of the follow-up transmission of daily record receiver of Network Management Equipment will comprise which information and the position of each information in message and the byte number that takies.
In the processing method of network log provided by the invention, rule file of definition on the Network Management Equipment, the daily record mapping ruler of the multiple Log Types of this rule file stored, the daily record mapping ruler of each Log Types comprises the field information of this Log Types.Concrete, this rule file is as follows:
<？xml?version＝″1.0″encoding＝″gb2312″？>
<RcvLogMapRule?xmlns:xsi＝″http://www.w3.org/2001/XMLSchema-instance″
xsi:noNamespaceSchemaLocation＝″RcvLogMapRule.xsd″>
＜!--this document has defined the mapping ruler of daily record, comprise position in original log of field name, this field,--
-->
<！---->
＜!--can define a plurality of log in logs, each log can comprise following field:--〉
＜!--1) type: Log Types can be NAT, FLOW, ACCESS, NETSTREAM etc.-->
＜!--2) canApplyNsv9: can this Log Types use NetStreamV9 transmission, 0 expression can not, otherwise can--
＜!--3) fields: define all field informations of a kind of Log Types, can comprise a plurality of field.-->
-->
<！--5)memInd-->
＜!--5) memIndex: the index of this field, wherein-1 expression skips over this field, and memIndex can not equal 0--
＜!--6) length: this field shared byte length in the original log agreement.-->
-->
＜!--1-table--〉
＜!--1-represents source IP address; 2-represents purpose IP address; 3-represents common IP; 4-represents MAC--〉
-->
＜!--10 expression beginning--〉
＜!--10 expression time starteds; 11 expression concluding times; 12-represents total duration;--
-->
＜!--20-represents--〉
＜!--20-represents Inbound message number; 21-represents the outgoing packet number;--
＜!--22-represents the Inbound byte number; 23-expresses the direction byte number;--
＜!--24-represents the Inbound fluxion; 25-expresses the direction fluxion;--
＜!--30-represents reserved field; The general integer of 31-; The general character string of 32-;--
＜!--33-represents the original message field; 34-represents 64 long fields;--
＜!--36-represents signless integer; 35-daily record direction, value are the full Intranet of 1 representative, and value is 0 for other situation;--
＜!--37-represents the daily record version information; 38-daily record bar number--〉
＜!--39-represents percentage (form of numeral+percentage sign); 40-represents that value is empty field--〉
<logs>
<log>
<type>NSV5</type>
<canApplyNsv9>1</canApplyNsv9>
<head>
<field>
<name>version</name>
<memIndex>-2<memIndex>
<length>2</length>
<type>37</type>
<checkValue>5</checkValue>
</field>
<field>
<name>count</name>
<memIndex>-2<memIndex>
<length>2</length>
<type>38</type>
</field>
<field>
<name>sysUptime</name>
<memIndex>-2<memIndex>
<length>4</length>
<type>31</type>
</field>
<field>
<name>unixSecs</name>
<memIndex>-1<memIndex>
<length>4</length>
<type>31</type>
</field>
<field>
<name>unixNSecs</name>
<memIndex>-1<memIndex>
<length>4</length>
<type>31</type>
</field>
<field>
<name>flowSequence</name>
<memIndex>-1<memIndex>
<length>4</length>
<type>31</type>
</field>
<field>
<name>engineType</name>
<memIndex>-1<memIndex>
<length>1</length>
<type>31</type>
</field>
<field>
<name>engineID</name>
<memIndex>-1<memIndex>
<length>1</length>
<type>31</type>
</field>
<field>
<name>samplingInterval</name>
<memIndex>-2<memIndex>
<length>2</length>
<type>31</type>
</field>
</head>
<fields>
<field>
<name>srcIp</name>
<memIndex>1</memIndex>
<Nsv9Index>8</Nsv9Index>
<length>4</length>
<type>1</type>
</field>
<field>
<name>dstIp</name>
<memIndex>2</memIndex>
<Nsv9Index>12</Nsv9Index>
<length>4</length>
<type>2</type>
</field>
<field>
<name>nexthop</name>
<memIndex>3</memIndex>
<Nsv9Index>15</Nsv9Index>
<length>4</length>
<type>3</type>
</field>
<field>
<name>ifindex</name>
<memIndex>4</memIndex>
<Nsv9Index>58</Nsv9Index>
<length>2</length>
<type>31</type>
</field>
<field>
<name>ofindex</name>
<memIndex>5</memIndex>
<Nsv9Index>59</Nsv9Index>
<length>2</length>
<type>31</type>
</field>
<field>
<name>outPackets</name>
<memIndex>6</memIndex>
<Nsv9Index>2</Nsv9Index>
<length>4</length>
<type>21</type>
</field>
<field>
<name>outOctets</name>
<memIndex>7</memIndex>
<Nsv9Index>1</Nsv9Index>
<length>4</length>
<type>23</type>
</field>
<field>
<name>startTime</name>
<memIndex>8</memIndex>
<Nsv9Index>22</Nsv9Index>
<length>4</length>
<exprValue>$CurrentTime-(sysUptime-startTime)/1000</exprValue>
<type>10</type>
</field>
<field>
<name>endTime</name>
<memIndex>9</memIndex>
<Nsv9Index>21</Nsv9Index>
<length>4</length>
<exprValue>$CurrentTime-(sysUptime-endTime)/1000</exprValue>
<type>11</type>
</field>
<field>
<name>srcPort</name>
<memIndex>10</memIndex>
<Nsv9Index>7</Nsv9Index>
<length>2</length>
<type>31</type>
</field>
<field>
<name>dstPort</name>
<memIndex>11</memIndex>
<Nsv9Index>11</Nsv9Index>
<length>2</length>
<type>31</type>
</field>
<field>
<name>pad1</name>
<memIndex>-1</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>1</length>
<type>31</type>
</field>
<field>
<name>tcpFlags</name>
<memIndex>12</memIndex>
<Nsv9Index>6</Nsv9Index>
<length>1</length>
<type>31</type>
</field>
<field>
<name>prot</name>
<memIndex>13</memIndex>
<Nsv9Index>4</Nsv9Index>
<length>1</length>
<type>31</type>
</field>
<field>
<name>tos</name>
<memIndex>14</memIndex>
<Nsv9Index>5</Nsv9Index>
<length>1</length>
<type>31</type>
</field>
<field>
<name>srcAs</name>
<memIndex>15</memIndex>
<Nsv9Index>16</Nsv9Index>
<length>2</length>
<type>31</type>
</field>
<field>
<name>dstAs</name>
<memIndex>16</memIndex>
<Nsv9Index>17</Nsv9Index>
<length>2</length>
<type>31</type>
</field>
<field>
<name>src_Mask</name>
<memIndex>17</memIndex>
<Nsv9Index>9</Nsv9Index>
<length>1</length>
<type>3</type>
</field>
<field>
<name>dst_Mask</name>
<memIndex>18</memIndex>
<Nsv9Index>13</Nsv9Index>
<length>1</length>
<type>3</type>
</field>
<field>
<name>direct</name>
<memIndex>19</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>1</length>
<type>35</type>
</field>
<field>
<name>pad2</name>
<memIndex>-1</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>1</length>
<type>31</type>
</field>
<field>
<name>inPackets</name>
<memIndex>20</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>20</type>
</field>
<field>
<name>inOctets</name>
<memIndex>21</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>22</type>
</field>
<field>
<name>outFlows</name>
<memIndex>22</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>25</type>
</field>
<field>
<name>totaltime</name>
<memIndex>23</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>12</type>
</field>
<field>
<name>hostIP</name>
<memIndex>24</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>5</type>
</field>
<field>
<name>inFlows</name>
<memIndex>25</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>24</type>
</field>
<field>
<name>appId</name>
<memIndex>26</memIndex>
<Nsv9Index>0</Nsv9Index>
<length>0</length>
<type>40</type>
</field>
</fields>
</log>
</logs>
</RcvLogMapRule>
Below only be the field of a log, similar for the field information of all the other log, can be with reference to above-mentioned form.Wherein, the type under the log label identifies the Log Types of this log correspondence.Whether canApplyNsv9 identifies this daily record can be by NetStream V9 transmission; The head information definition daily record the head form, the fields information definition all field informations of a kind of Log Types, the field label has defined the meaning of each field in the daily record: wherein name indicates the name of field; MemIndex indicates the index of this field in internal memory, if not needing to handle, a field can not use particular value (for example-1) sign, program is read this information and will directly be abandoned, and being illustrated in for particular value-2 (this particular value also can be other) needs to use in the packet parsing process but do not need to be saved in database; Length indicates the byte number that take of this field in the original log message; Type indicates the type of this field, compares and computing for time started, concluding time, informational needs such as source IP, purpose IP, identifies by giving different types; The checkValue field is used for the daily record that verification receives and whether belongs to current Log Types; Nsv9Index identifies the call number (having specified a unique index for each field among the NetStream V9) of this field correspondence in NetStream V9; ExprValue represents that the value of this field need carry out computing, mainly is time started and concluding time in order to calculate daily record, and De $CurrentTime is the valve system current time in the expression formula.
Among the present invention, after Network Management Equipment received daily record, Network Management Equipment obtained the field information of the daily record that receives, and the Log Types that mates with the field information that obtains in the search rule file uses the Log Types that finds to resolve daily record.
Be the processing method that the first kind network log (daily record of set form) and the second class network log are introduced network log provided by the invention respectively with this daily record below.
After Network Management Equipment received daily record, if first kind network log, as shown in Figure 2, Network Management Equipment can be searched the Log Types that mates with daily record according to the Log Types identification field, uses the Log Types that finds to resolve daily record.Below be that the checkValue field is that example is introduced with the Log Types identification field, comprising:
Concrete, Network Management Equipment is resolved each bar log record one by one according to the resolution rules of rule file and is kept in the assigned indexes of internal memory.Then preserve value after the calculating for the field that expression formula is arranged.
Among the present invention, can also comprise after the step 203:
If the second class network log, Network Management Equipment carries out daily record according to following steps resolves, and as shown in Figure 3, comprising:
Concrete, among the present invention, the value of canApplyNsv9 is to represent that this Log Types can not be by NetStream V9 transmission at 0 o'clock, otherwise represents that this Log Types can be by NetStream V9 transmission.The value of current canApplyNsv9 can be adjusted flexibly, is not limited to 0.
Need to prove, in the method provided by the invention, Network Management Equipment is behind the Log Types of resolving the managed device issue, write down the corresponding relation of this managed device and this Log Types, for example Network Management Equipment can be set up the corresponding relation tabulation of Log Types and managed device, after the daily record that receives arbitrary managed device transmission, at first search the Log Types that whether stores this managed device correspondence in this tabulation, if have then use the stored log type to resolve daily record, otherwise handle according to the mode of above-mentioned Fig. 2 or Fig. 3.
In addition, can also set in advance the Log Types of each managed device issue, promptly set the Log Types of each the managed device issue under the network management system, and tabulate at the corresponding relation of Network Management Equipment storing daily record type and managed device.
Behind newly-increased Log Types, do not need to develop again new log processing software on the Network Management Equipment, only need in rule file, increase the log field of newly-increased Log Types.
When adopting method provided by the invention, Network Management Equipment can also be handled when resolving daily record, for example is set to ignore the information of some fields, reduces the log processing burden of Network Management Equipment, and for example some field is only resolved and do not stored in the rule file.
The invention provides a kind of treatment facility of network log, as shown in Figure 4, comprising:
Dispensing unit 11 is used for the configuration rule file, the field information of the multiple Log Types of storage in this rule file;
Described resolution unit 12 also is used for:
When equipment receives the daily record of set form, resolve daily record according to the current Log Types in the rule file, read the Log Types identification field value in the daily record;
When the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, use described current Log Types to resolve daily record;
The Log Types identification field value of Log Types identification field value in daily record and described current Log Types uses next Log Types to resolve daily record not simultaneously.
Described resolution unit 12 also is used for:
During the value of the message total length field in reading daily record, judge whether the value of this field and the message total length field value of described current Log Types mate, if coupling then uses described current Log Types to resolve daily record; Otherwise use next Log Types to resolve daily record.
Described resolution unit 12 comprises:
This equipment can also comprise:
Described resolution unit 12 also is connected with the described unit 14 of searching, and is used for describedly when searching the unit and searching successfully, uses and describedly searches the Log Types that the unit finds and resolve daily record.
Described memory cell 13 also is used for: the corresponding relation of storing daily record type and daily record transmit leg in advance; Perhaps store the Log Types that described resolution unit finds and the corresponding relation of daily record transmit leg.
Through the above description of the embodiments, those skilled in the art can be well understood to the present invention and can realize by the mode that software adds essential general hardware platform, can certainly pass through hardware, but the former is better execution mode under a lot of situation.Based on such understanding, the part that technical scheme of the present invention contributes to prior art in essence in other words can embody with the form of software product, this computer software product is stored in the storage medium, comprise that some instructions are with so that a computer equipment (can be a personal computer, server, the perhaps network equipment etc.) carry out the described method of each embodiment of the present invention.
It will be appreciated by those skilled in the art that accompanying drawing is the schematic diagram of a preferred embodiment, module in the accompanying drawing or flow process might not be that enforcement the present invention is necessary.
It will be appreciated by those skilled in the art that the module in the device among the embodiment can be distributed in the device of embodiment according to the embodiment description, also can carry out respective change and be arranged in the one or more devices that are different from present embodiment.The module of the foregoing description can be merged into a module, also can further split into a plurality of submodules.
More than disclosed only be several specific embodiment of the present invention, still, the present invention is not limited thereto, any those skilled in the art can think variation all should fall into protection scope of the present invention.
Claims (12)
1. the processing method of a network log is applied to the Network Management Equipment of network management system, it is characterized in that, comprising:
Steps A, configuration rule file on the described Network Management Equipment, the field information of the multiple Log Types of storage in this rule file;
Step B, described Network Management Equipment obtains the field information of the daily record that receives, and the Log Types of the field information coupling of searching and obtaining uses the Log Types that finds to resolve daily record.
2. the method for claim 1 is characterized in that, the Log Types of the described field information coupling of searching and obtaining comprises:
When receiving the daily record of set form, described Network Management Equipment is resolved daily record according to the current Log Types in the rule file, reads the Log Types identification field value in the daily record;
When the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, use described current Log Types to resolve daily record;
The Log Types identification field value of Log Types identification field value in daily record and described current Log Types uses next Log Types to resolve daily record not simultaneously.
3. method as claimed in claim 2 is characterized in that, when the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, uses described current Log Types to resolve daily record and comprises:
During the value of the message total length field in reading daily record, judge whether the value of this field and the message total length field value of described current Log Types mate, if coupling then uses described current Log Types to resolve daily record; Otherwise use next Log Types to resolve daily record.
4. the method for claim 1 is characterized in that, the Log Types of the described field information coupling of searching and obtaining comprises:
After described Network Management Equipment received the daily record template, described Network Management Equipment was searched the Log Types of the daily record that can transmit described daily record template correspondence in rule file;
Whether each field corresponding with described daily record template is included in the described daily record template in the Log Types that described network management equipment judges finds;
If judged result is for being, described Network Management Equipment uses the Log Types that finds to resolve the daily record of described daily record template correspondence.
5. as each described method of claim 1-4, it is characterized in that, also comprise before the step B:
Step C after described Network Management Equipment receives daily record, searches the Log Types corresponding with the transmit leg of described daily record in the corresponding relation of stored log type and daily record transmit leg, if search success then use the Log Types that finds to resolve daily record; Otherwise execution in step B.
6. method as claimed in claim 5 is characterized in that, described Network Management Equipment is the corresponding relation of storing daily record type and daily record transmit leg in advance; The Log Types that finds among the perhaps described Network Management Equipment storing step B and the corresponding relation of daily record transmit leg.
7. the treatment facility of a network log is characterized in that, comprising:
Dispensing unit is used for the configuration rule file, the field information of the multiple Log Types of storage in this rule file;
Resolution unit is connected with described dispensing unit, is used to the field information of the daily record that the equipment that obtains receives, and the Log Types of the field information of searching and obtaining coupling uses the Log Types that finds to resolve daily record.
8. equipment as claimed in claim 7 is characterized in that, described resolution unit also is used for:
When equipment receives the daily record of set form, resolve daily record according to the current Log Types in the rule file, read the Log Types identification field value in the daily record;
When the Log Types identification field value of the Log Types identification field value in the daily record and described current Log Types is identical, use described current Log Types to resolve daily record;
The Log Types identification field value of Log Types identification field value in daily record and described current Log Types uses next Log Types to resolve daily record not simultaneously.
9. equipment as claimed in claim 8 is characterized in that, described resolution unit also is used for:
During the value of the message total length field in reading daily record, judge whether the value of this field and the message total length field value of described current Log Types mate, if coupling then uses described current Log Types to resolve daily record; Otherwise use next Log Types to resolve daily record.
10. equipment as claimed in claim 7 is characterized in that, described resolution unit comprises:
Search subelement, be used for after equipment receives the daily record template, in rule file, search the Log Types of the daily record that can transmit described daily record template correspondence;
Judgment sub-unit is connected with the described subelement of searching, and is used for judging whether Log Types each field corresponding with described daily record template that finds is included in the described daily record template;
Resolve subelement, is connected with described interpretation subelement, the judged result that is used for described interpretation subelement is when being, the Log Types that use finds is resolved the daily record of described daily record template correspondence.
11. as each described equipment of claim 7-10, it is characterized in that, also comprise:
Memory cell is used for the corresponding relation of storing daily record type and daily record transmit leg;
Search the unit, be connected, be used for after equipment receives daily record, search the Log Types corresponding with the transmit leg of daily record according to described corresponding relation with described memory cell;
Described resolution unit also is connected with the described unit of searching, and is used for describedly when searching the unit and searching successfully, uses and describedly searches the Log Types that the unit finds and resolve daily record.
12. equipment as claimed in claim 11 is characterized in that, described memory cell also is used for:
The corresponding relation of storing daily record type and daily record transmit leg in advance; Perhaps store the Log Types that described resolution unit finds and the corresponding relation of daily record transmit leg.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010296462 CN101931562B (en) | 2010-09-29 | 2010-09-29 | Web log processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010296462 CN101931562B (en) | 2010-09-29 | 2010-09-29 | Web log processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101931562A true CN101931562A (en) | 2010-12-29 |
| CN101931562B CN101931562B (en) | 2013-08-28 |
Family
ID=43370496
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010296462 Active CN101931562B (en) | 2010-09-29 | 2010-09-29 | Web log processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101931562B (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102164050A (en) * | 2011-05-16 | 2011-08-24 | 北京星网锐捷网络技术有限公司 | Log parsing method and log parsing node device |
| CN103368782A (en) * | 2013-07-30 | 2013-10-23 | 浙江中烟工业有限责任公司 | Network state analysis method |
| CN103929321A (en) * | 2013-01-15 | 2014-07-16 | 腾讯科技（深圳）有限公司 | Log processing method and device |
| CN104580310A (en) * | 2013-10-21 | 2015-04-29 | 腾讯科技（深圳）有限公司 | Log processing method and server |
| CN104933077A (en) * | 2014-03-20 | 2015-09-23 | 上海宝信软件股份有限公司 | Rule-based multi-file information analysis method |
| CN105447099A (en) * | 2015-11-11 | 2016-03-30 | 中国建设银行股份有限公司 | Log structured information extraction method and apparatus |
| CN106126383A (en) * | 2016-06-01 | 2016-11-16 | 杭州华三通信技术有限公司 | A kind of log processing method and device |
| CN106201848A (en) * | 2016-06-30 | 2016-12-07 | 北京奇虎科技有限公司 | The log processing method of a kind of real-time calculating platform and device |
| CN107919981A (en) * | 2017-10-31 | 2018-04-17 | 江苏省未来网络创新研究院 | A kind of analysis method of multi-vendor log cache |
| CN108268471A (en) * | 2016-12-30 | 2018-07-10 | 北京国双科技有限公司 | The read method and device of journal file |
| CN108804106A (en) * | 2018-04-28 | 2018-11-13 | 北京机械设备研究所 | A kind of data analysis software optimization method based on allocation list management |
| CN109408479A (en) * | 2018-09-19 | 2019-03-01 | 平安科技（深圳）有限公司 | Daily record data adding method, system, computer equipment and storage medium |
| CN110826299A (en) * | 2019-10-25 | 2020-02-21 | 上海工业自动化仪表研究院有限公司 | General template log analysis method based on classification |
| CN111078657A (en) * | 2019-12-26 | 2020-04-28 | 北京思特奇信息技术股份有限公司 | Service log query method, system, medium and equipment of distributed system |
| WO2020253399A1 (en) * | 2019-06-21 | 2020-12-24 | 深圳前海微众银行股份有限公司 | Log classification rule generation method, device, apparatus, and readable storage medium |
| CN113407421A (en) * | 2021-08-19 | 2021-09-17 | 北京江融信科技有限公司 | Dynamic log record management method and system for micro-service gateway |
| CN113688006A (en) * | 2017-10-16 | 2021-11-23 | 创新先进技术有限公司 | Log data verification method and device |
| CN113810242A (en) * | 2020-06-16 | 2021-12-17 | 中盈优创资讯科技有限公司 | System log analysis method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1917445A (en) * | 2006-09-07 | 2007-02-21 | 上海交通大学 | Method for auditing log event of fire wall, and teaching experimental system |
| WO2007059057A2 (en) * | 2005-11-12 | 2007-05-24 | Logrhythm, Inc | Log collection, structuring and processing |
| CN101453378A (en) * | 2008-12-30 | 2009-06-10 | 杭州华三通信技术有限公司 | Method and system for log damp and audit |
-
2010
- 2010-09-29 CN CN 201010296462 patent/CN101931562B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007059057A2 (en) * | 2005-11-12 | 2007-05-24 | Logrhythm, Inc | Log collection, structuring and processing |
| CN1917445A (en) * | 2006-09-07 | 2007-02-21 | 上海交通大学 | Method for auditing log event of fire wall, and teaching experimental system |
| CN101453378A (en) * | 2008-12-30 | 2009-06-10 | 杭州华三通信技术有限公司 | Method and system for log damp and audit |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102164050B (en) * | 2011-05-16 | 2014-01-22 | 北京星网锐捷网络技术有限公司 | Log parsing method and log parsing node device |
| CN102164050A (en) * | 2011-05-16 | 2011-08-24 | 北京星网锐捷网络技术有限公司 | Log parsing method and log parsing node device |
| CN103929321A (en) * | 2013-01-15 | 2014-07-16 | 腾讯科技（深圳）有限公司 | Log processing method and device |
| CN103368782A (en) * | 2013-07-30 | 2013-10-23 | 浙江中烟工业有限责任公司 | Network state analysis method |
| CN103368782B (en) * | 2013-07-30 | 2016-08-10 | 浙江中烟工业有限责任公司 | A kind of network status analysis method |
| CN104580310A (en) * | 2013-10-21 | 2015-04-29 | 腾讯科技（深圳）有限公司 | Log processing method and server |
| CN104933077B (en) * | 2014-03-20 | 2018-08-14 | 上海宝信软件股份有限公司 | Rule-based multifile information analysis method |
| CN104933077A (en) * | 2014-03-20 | 2015-09-23 | 上海宝信软件股份有限公司 | Rule-based multi-file information analysis method |
| CN105447099B (en) * | 2015-11-11 | 2018-12-14 | 中国建设银行股份有限公司 | Log-structuredization information extracting method and device |
| CN105447099A (en) * | 2015-11-11 | 2016-03-30 | 中国建设银行股份有限公司 | Log structured information extraction method and apparatus |
| CN106126383A (en) * | 2016-06-01 | 2016-11-16 | 杭州华三通信技术有限公司 | A kind of log processing method and device |
| CN106126383B (en) * | 2016-06-01 | 2019-03-19 | 新华三技术有限公司 | A kind of log processing method and device |
| CN106201848A (en) * | 2016-06-30 | 2016-12-07 | 北京奇虎科技有限公司 | The log processing method of a kind of real-time calculating platform and device |
| CN108268471A (en) * | 2016-12-30 | 2018-07-10 | 北京国双科技有限公司 | The read method and device of journal file |
| CN113688006A (en) * | 2017-10-16 | 2021-11-23 | 创新先进技术有限公司 | Log data verification method and device |
| CN113688006B (en) * | 2017-10-16 | 2024-03-29 | 创新先进技术有限公司 | Log data verification method and device |
| CN107919981A (en) * | 2017-10-31 | 2018-04-17 | 江苏省未来网络创新研究院 | A kind of analysis method of multi-vendor log cache |
| CN108804106A (en) * | 2018-04-28 | 2018-11-13 | 北京机械设备研究所 | A kind of data analysis software optimization method based on allocation list management |
| CN108804106B (en) * | 2018-04-28 | 2022-04-19 | 北京机械设备研究所 | Data analysis software optimization method based on configuration table management |
| CN109408479B (en) * | 2018-09-19 | 2023-05-30 | 平安科技（深圳）有限公司 | Log data adding method, system, computer device and storage medium |
| CN109408479A (en) * | 2018-09-19 | 2019-03-01 | 平安科技（深圳）有限公司 | Daily record data adding method, system, computer equipment and storage medium |
| WO2020253399A1 (en) * | 2019-06-21 | 2020-12-24 | 深圳前海微众银行股份有限公司 | Log classification rule generation method, device, apparatus, and readable storage medium |
| CN110826299A (en) * | 2019-10-25 | 2020-02-21 | 上海工业自动化仪表研究院有限公司 | General template log analysis method based on classification |
| CN110826299B (en) * | 2019-10-25 | 2023-05-23 | 上海工业自动化仪表研究院有限公司 | General template log analysis method based on classification |
| CN111078657A (en) * | 2019-12-26 | 2020-04-28 | 北京思特奇信息技术股份有限公司 | Service log query method, system, medium and equipment of distributed system |
| CN113810242A (en) * | 2020-06-16 | 2021-12-17 | 中盈优创资讯科技有限公司 | System log analysis method and device |
| CN113407421A (en) * | 2021-08-19 | 2021-09-17 | 北京江融信科技有限公司 | Dynamic log record management method and system for micro-service gateway |
| CN113407421B (en) * | 2021-08-19 | 2021-11-30 | 北京江融信科技有限公司 | Dynamic log record management method and system for micro-service gateway |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101931562B (en) | 2013-08-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101931562B (en) | Web log processing method and device | |
| US9681204B2 (en) | Methods and apparatus to validate a tag for media | |
| CN100489879C (en) | Method, system and server for checking page data | |
| KR102099544B1 (en) | Method and device for processing distribution of streaming data | |
| CN100586109C (en) | Communication method and system for universal service data based on self-defined template | |
| US20130191890A1 (en) | Method and system for user identity recognition based on specific information | |
| US20100333172A1 (en) | Method, apparatus and system for monitoring database security | |
| CN101614781B (en) | Intelligent diagnosis method of radio and television equipment based on spatial rule index | |
| US20120099478A1 (en) | Method and Device for Auto-Generating Goose Signal Connection Topology from Substation Level | |
| CN107329888A (en) | Intelligent contract command code coverage rate computational methods and system | |
| HK1126052A1 (en) | Aggregated resource reservation for data flows | |
| US8700632B2 (en) | Managing heterogeneous data | |
| Bezerra et al. | Measures for quality evaluation of feature models | |
| KR100817562B1 (en) | Method for indexing a large scaled logfile, computer readable medium for storing program therein, and system for the preforming the same | |
| CN108563718A (en) | A kind of method and system preventing log flood | |
| CN104317939A (en) | Log statistics method and system on basis of digital film playing server | |
| CN104320704A (en) | Method and device for detecting network television installation states | |
| CN101140581A (en) | SQL statement construct method and apparatus of preprocess special-character | |
| CN100555957C (en) | Method and device thereof that a kind of incident is synthetic | |
| CN109522528A (en) | A kind of word document is converted to the method that can calculate automatically html document | |
| US20140337069A1 (en) | Deriving business transactions from web logs | |
| CN102609482A (en) | Packaging method of JSON (JavaScript Object Notiation) data format | |
| CN108270599B (en) | Data analysis processing method and system based on SNMP (simple network management protocol) | |
| CN115952146A (en) | File management system applied to key information supervision of direct-current control protection device | |
| CN115174201A (en) | Security rule management method and device based on screening label |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 310052 Binjiang District Changhe Road, Zhejiang, China, No. 466, No. Patentee after: Xinhua three Technology Co., Ltd. Address before: 310053 Hangzhou hi tech Industrial Development Zone, Zhejiang province science and Technology Industrial Park, No. 310 and No. six road, HUAWEI, Hangzhou production base Patentee before: Huasan Communication Technology Co., Ltd. |
|
| CP03 | Change of name, title or address |