CN106055452B - The method and apparatus for creating interchanger log template - Google Patents

The method and apparatus for creating interchanger log template Download PDF

Info

Publication number
CN106055452B
CN106055452B CN201610355129.3A CN201610355129A CN106055452B CN 106055452 B CN106055452 B CN 106055452B CN 201610355129 A CN201610355129 A CN 201610355129A CN 106055452 B CN106055452 B CN 106055452B
Authority
CN
China
Prior art keywords
message
log
type
keyword
interchanger
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610355129.3A
Other languages
Chinese (zh)
Other versions
CN106055452A (en
Inventor
董辉
宋磊
侯翔宇
孟伟彬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN201610355129.3A priority Critical patent/CN106055452B/en
Publication of CN106055452A publication Critical patent/CN106055452A/en
Application granted granted Critical
Publication of CN106055452B publication Critical patent/CN106055452B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3065Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G06F11/3072Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
    • G06F11/3082Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting the data filtering being achieved by aggregating or compressing the monitored data
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3006Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3065Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G06F11/3086Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves the use of self describing data formats, i.e. metadata, markup languages, human readable formats
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06Management of faults, events, alarms or notifications
    • H04L41/069Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications

Abstract

This application discloses the method and apparatus of creation interchanger log template.One specific embodiment of the method includes: to obtain the original log of the interchanger of a model;The type of message and detail message in the original log are obtained, if obtained less than type of message, creates cluster labels as type of message;The detail message is segmented, keyword is obtained;It is resequenced according to the word frequency of the keyword to the keyword, the high keyword of word frequency is come into front;Multiway tree, root node of each type of message as tree, node of the keyword of the rearrangement as tree, the corresponding multiway tree of each type of message are created according to the keyword of the rearrangement;Multiway tree described in depth-first traversal, log template corresponding to each type of message for creating the model interchanger according to the path in the multiway tree.The embodiment creates interchanger log template, is compressed using the template to log.

Description

The method and apparatus for creating interchanger log template
Technical field
This application involves field of computer technology, and in particular to Internet technical field, more particularly to creation interchanger day The method and apparatus of will template.
Background technique
Interchanger log is the significant data that switch device generates in time to live, can be obtained by interchanger log The state for taking interchanger, shake, board faults, power failure of shake, agreement including port etc., and traditional monitoring of tools System is all based on interchanger log, by specific rule match log, to find fault point and alarm.
With the explosive increase of internet data and increasing for business, many Internet enterprises are to the throwing from establishing network Enter also increasing, the network equipment being related in IDC (Internet Data Center, Internet data center) is also more next It is more, especially switch device.
One typical case is in the self-built IDC of enterprise, and there are large-scale switch device, these equipment are come respectively From different manufacturers, there are Multiple Types, while being distributed in multiple IDC.Excessive equipment can generate large-scale interchanger Log all brings difficulty for the excavation and troubleshooting of alarm rule, a kind of automation means is needed to compress log.
Summary of the invention
The purpose of the application is to propose a kind of method and apparatus for creating interchanger log template, to solve background above The technical issues of technology segment is mentioned.
In a first aspect, this application provides the methods of creation interchanger log template, which comprises obtain a type Number interchanger original log;The type of message and detail message in the original log are obtained, if obtained less than message Type then creates cluster labels as type of message;The detail message is segmented, keyword is obtained;According to the pass The word frequency of keyword resequences to the keyword, and the high keyword of word frequency is come front;According to the pass of the rearrangement Keyword creates multiway tree, and root node of each type of message as tree, the keyword of the rearrangement is as the node set, often The corresponding multiway tree of a type of message;Multiway tree described in depth-first traversal creates institute according to the path in the multiway tree State log template corresponding to each type of message of model interchanger.
In some embodiments, the method also includes: obtain a model interchanger newly-increased log;Described in acquisition Type of message and detail message in newly-increased log create cluster labels as message class if obtained less than type of message Type;The newly-increased log is filtered using the log template;It can not be detailed in matched newly-increased log by the log template Message is segmented, and keyword is obtained;It is resequenced according to the word frequency of the keyword to the keyword, by the high pass of word frequency Keyword comes front;Multiway tree, root node of each type of message as tree, institute are created according to the keyword of the rearrangement State node of the keyword of rearrangement as tree, the corresponding multiway tree of each type of message;It is more described in depth-first traversal Fork is set, newly-increased log mould corresponding to each type of message for creating the model interchanger according to the path in the multiway tree Plate.
In some embodiments, it if the child node of a node is more than Node B threshold number in the multiway tree, deletes Except all child nodes of the node, the node is as the last one child node.
In some embodiments, the creation cluster labels are as type of message, comprising: each type of message is unknown Log be divided into five kinds of classifications according to semanteme and assign weighted value respectively, five kinds of classifications include: only number or number With symbol, number, letter and symbol, symbol and letter are only alphabetical, only symbol;Extract five classes semanteme in the log The log is converted the word frequency vector for fixing five length by frequency;Calculate the word frequency vector and known message The similarity of type set obtains one group of similarity result, if maximum similarity is more than or equal to preset similarity threshold, The log is grouped into corresponding type of message;If maximum similarity is less than preset similarity threshold, by the word frequency The vector type of message new as one.
In some embodiments, when a log template is the subset of another log template, by beating node The form of label come identify a node whether be a paths end.
In some embodiments, by the log template in each type of message according to the depth ordering of tree, when a log When template is the subset of another log template, the preferential biggish log template of depth using tree is matched.
Second aspect, this application provides a kind of devices for creating interchanger log template, which is characterized in that described device Include: acquiring unit, is configured to obtain the original log of the interchanger of a model;Resolution unit is configured to obtain institute The type of message and detail message in original log are stated, if obtained less than type of message, creates cluster labels as message Type;Processing unit is configured to segment the detail message, obtains keyword;According to the word frequency of the keyword It resequences to the keyword, the high keyword of word frequency is come into front;Creating unit is configured to be arranged again according to described The keyword of sequence creates multiway tree, and root node of each type of message as tree, the keyword of the rearrangement is as tree Node, the corresponding multiway tree of each type of message;Multiway tree described in depth-first traversal, according to the path in the multiway tree Create log template corresponding to each type of message of the model interchanger.
In some embodiments, described device is also configured to: obtaining the newly-increased log of the interchanger of a model;It obtains Type of message and detail message in the newly-increased log disappear if acquisition less than type of message, creates cluster labels and is used as Cease type;The newly-increased log is filtered using the log template;It can not be in matched newly-increased log by the log template Detail message is segmented, and keyword is obtained;It is resequenced according to the word frequency of the keyword to the keyword, by word frequency height Keyword come front;Multiway tree, root section of each type of message as tree are created according to the keyword of the rearrangement Point, node of the keyword of the rearrangement as tree, the corresponding multiway tree of each type of message;Depth-first traversal institute Multiway tree is stated, newly-increased day corresponding to each type of message for creating the model interchanger according to the path in the multiway tree Will template.
In some embodiments, the creating unit is also configured to: if the son of a node saves in the multiway tree Point is more than Node B threshold number, then deletes all child nodes of the node, the node is as the last one child node.
In some embodiments, the creation cluster labels are as type of message, comprising: each type of message is unknown Log be divided into five kinds of classifications according to semanteme and assign weighted value respectively, five kinds of classifications include: only number or number With symbol, number, letter and symbol, symbol and letter are only alphabetical, only symbol;Extract five classes semanteme in the log The log is converted the word frequency vector for fixing five length by frequency;Calculate the word frequency vector and known message The similarity of type set obtains one group of similarity result, if maximum similarity is more than or equal to preset similarity threshold, The log is grouped into corresponding type of message;If maximum similarity is less than preset similarity threshold, by the word frequency The vector type of message new as one.
In some embodiments, the creating unit is also configured to: when a log template is another log mould When the subset of plate, identified by way of labelling to node a node whether be a paths end.
In some embodiments, the creating unit is also configured to: by the template in each type of message according to tree Depth ordering, when a log template is the subset of another log template, the preferential biggish template of depth using tree It is matched.
The method and apparatus of creation interchanger log template provided by the present application, by obtaining disappearing in the original log Type and detail message are ceased, multiway tree is created according to the keyword in detail message, exchange is created according to the path in multiway tree Log template corresponding to each type of message of machine, to be compressed using increment iterative training method to interchanger log.
Detailed description of the invention
By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, the application's is other Feature, objects and advantages will become more apparent upon:
Fig. 1 is that this application can be applied to exemplary system architecture figures therein;
Fig. 2 is the flow chart according to one embodiment of the method for the creation interchanger log template of the application;
Fig. 3 is the schematic diagram according to the word frequency vector of the method for the creation interchanger log template of the application;
Fig. 4 a, 4b and 4c are the signals according to an application scenarios of the method for the creation interchanger log template of the application Figure;
Fig. 5 is the flow chart according to another embodiment of the method for the creation interchanger log template of the application;
Fig. 6 is the structural schematic diagram according to one embodiment of the device of the creation interchanger log template of the application;
Fig. 7 is adapted for the structural schematic diagram for the computer system for realizing the server of the embodiment of the present application.
Specific embodiment
The application is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched The specific embodiment stated is used only for explaining related invention, rather than the restriction to the invention.It also should be noted that in order to Convenient for description, part relevant to related invention is illustrated only in attached drawing.
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase Mutually combination.The application is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Fig. 1 is shown can be using the method or creation interchanger log template of the creation interchanger log template of the application Device embodiment exemplary system architecture 100.
As shown in Figure 1, system architecture 100 may include interchanger 101,102,103, network 104 and server 105.Net Network 104 between interchanger 101,102,103 and server 105 to provide the medium of communication link.Network 104 may include Various connection types, such as wired, wireless communication link or fiber optic cables etc..
The log of interchanger 101,102,103 is transferred to server 105 by network 104.On interchanger 101,102,103 The client of various log collection tools can be installed, such as be equipped with and can be transferred to its internal log information remotely Log server rsyslog etc..
Interchanger 101,102,103, which can be, provides the electric signal exclusively enjoyed for any two network node of access switch The network equipment of access, including but not limited to Ethernet switch, fast ethernet switch, gigabit ethernet switch, FDDI interchanger, ATM switch and token ring switch etc..
Server 105 can be to provide the server of various services, such as the log of acquisition interchanger 101,102,103, Structuring processing is carried out to collected log and creates the server of log template.
It should be noted that the method for creation interchanger log template is generally by server provided by the embodiment of the present application 105 execute, and correspondingly, the device of creation interchanger log template is generally positioned in server 105.
It should be understood that the number of interchanger, network and server in Fig. 1 is only schematical.According to realize needs, It can have any number of interchanger, network and server.
With continued reference to Fig. 2, one embodiment of the method for the creation interchanger log template according to the application is shown Process 200.The method of the creation interchanger log template, comprising the following steps:
Step 201, the original log of the interchanger of a model is obtained.
In the present embodiment, the electronic equipment of the method operation of interchanger log template thereon is created (such as shown in Fig. 1 Server) original log can be obtained from interchanger by wired connection mode or radio connection.
Step 202, the type of message and detail message obtained in original log is created if obtained less than type of message Cluster labels are built as type of message.
In the present embodiment, the type of message and detail message in original log are obtained, if obtained less than type of message, Cluster labels are then created as type of message.Wherein, the detail message needs to remove variable.
In some optional implementations of the present embodiment, the creation cluster labels are as type of message, comprising: will The unknown log of each type of message is divided into five kinds of classifications according to semanteme and assigns weighted value respectively, five kinds of classification packets Include: only number or number and symbol, number, letter and symbol, symbol and letter are only alphabetical, only symbol, in table 1 It is shown;The log is converted the word frequency vector for fixing five length by the frequency for extracting five classes semanteme in the log, As shown in Figure 3;The similarity for calculating the word frequency vector Yu known type of message set obtains one group of similarity result, such as Fruit maximum similarity is more than or equal to preset similarity threshold, then the log is grouped into corresponding type of message;If maximum Similarity is less than preset similarity threshold, then the type of message that the word frequency vector is new as one.
Table 1
Step 203, detail message is segmented, obtains keyword.
In the present embodiment, it is segmented using Luene, obtains keyword.
Step 204, it is resequenced according to the word frequency of keyword to keyword, the high keyword of word frequency is come into front.
In the present embodiment, it is resequenced according to the word frequency of keyword to keyword, before the high keyword of word frequency is come Face.
Step 205, multiway tree is created according to the keyword of rearrangement.
In the present embodiment, multiway tree is created according to the keyword resequenced in step 204.
In some optional implementations of the present embodiment, if the child node of a node is more than in the multiway tree Node B threshold number then deletes all child nodes of the node, and the node is as the last one child node.For example, if The child node of one node is more than 10 (empirical values), then cuts down all child nodes of the node, the node is as the last one Child node.The purpose for the arrangement is that preventing template from expanding, because being directed to each type of message, 10 are not exceeded generally effectively State.For example, processing user name other parts keyword is consistent, if without child node quantity for logging in log Limitation, will lead to excessive template indicates same meaning.
Step 206, depth-first traversal multiway tree creates each of the model interchanger according to the path in multiway tree and disappears Cease log template corresponding to type.
In the present embodiment, the multiway tree of depth-first traversal step 205 creation, should according to the path creation in multiway tree Log template corresponding to each type of message of model interchanger.
In some optional implementations of the present embodiment, when a log template is the son of another log template When collection, identify whether a node is that (node may not be for the end of a paths by way of labelling to node The end of longest path).
In some optional implementations of the present embodiment, when a log template is the son of another log template When collection, the preferential biggish log template of depth using tree is matched.In the case where node is not the end of longest path, With the longest matching in path.
With continued reference to Fig. 4 a-4c, Fig. 4 a-4c is the application according to the method for the creation interchanger log template of the present embodiment One schematic diagram of scene.In the application scenarios of Fig. 4 a-4c, in fig.4, figure left side is after sorting according to word frequency Log, figure right side are the multiway tree constructed.In fig. 4b, the number of child nodes of node down is more than 10, therefore deletes The child node of down, and using down as the last one child node.In Fig. 4 c, node up is the end of a paths, to it It is tagged, but it is not longest path, the path length of the path of the first template than the second template.Log is carried out using template Timing preferentially uses the first template to be matched.
The method provided by the above embodiment of the application carries out word segmentation processing by the log to type of message and obtains Multiway tree is created after keyword, creates interchanger log template, can be used for the compression log of increment iterative training method, log pressure Contracting ratio can reach 2000:1.
With further reference to Fig. 5, it illustrates the processes of another embodiment of the method for creation interchanger log template 500.The process 500 of the method for the creation interchanger log template, comprising the following steps:
Step 501, the newly-increased log of the interchanger of a model is obtained.
In the present embodiment, the step and step 201 are essentially identical, and what difference was step acquisition is newly-increased day Will.
Step 502, the type of message and detail message obtained in newly-increased log is created if obtained less than type of message Cluster labels are built as type of message.
Step 502 is essentially identical with step 202, and details are not described herein.
Step 503, usage log template filters newly-increased log.
In the present embodiment, newly-increased log is filtered using the log template created in step 206, obtains former log template It can not matched log progress incremental training.
Step 504, by log template can not the detail message in matched newly-increased log segment, obtain keyword.
Step 504 is essentially identical with step 203, and details are not described herein.
Step 505, it is resequenced according to the word frequency of keyword to keyword, the high keyword of word frequency is come into front.
Step 506, multiway tree is created according to the keyword of rearrangement.
Step 507, depth-first traversal multiway tree creates each of the model interchanger according to the path in multiway tree and disappears Cease newly-increased log template corresponding to type.
Step 505-507 and step 204-206 are essentially identical, and details are not described herein.
From figure 5 it can be seen that the creation interchanger log template compared with the corresponding embodiment of Fig. 2, in the present embodiment The process 500 of method highlight the step of being filtered to newly-increased log.The scheme of the present embodiment description can use as a result, The mode of increment iterative training compresses log.
With further reference to Fig. 6, as the realization to method shown in above-mentioned each figure, this application provides a kind of creation interchangers One embodiment of the device of log template, the Installation practice is corresponding with embodiment of the method shown in Fig. 2, and the device is specific It can be applied in various electronic equipments.
As shown in fig. 6, described in the present embodiment create interchanger log template device 600 include: acquiring unit 601, Resolution unit 602, processing unit 603 and creating unit 604.Wherein, acquiring unit 601 is configured to obtain the friendship of a model The original log changed planes;Resolution unit 602 is configured to obtain type of message and detail message in the original log, if It obtains less than type of message, then creates cluster labels as type of message;Processing unit 603 is configured to the detail message It is segmented, obtains keyword;It is resequenced according to the word frequency of the keyword to the keyword, by the high keyword of word frequency Come front;Creating unit 604 is configured to create multiway tree according to the keyword of the rearrangement, and each type of message is made For the root node of tree, node of the keyword of the rearrangement as tree, the corresponding multiway tree of each type of message;Depth Multiway tree described in first traversal, each type of message institute for creating the model interchanger according to the path in the multiway tree are right The log template answered.
In some optional implementations of the present embodiment, the device 600 of the creation interchanger log template is also configured For: obtain the newly-increased log of the interchanger of a model;The type of message and detail message in the newly-increased log are obtained, such as Fruit obtains less than type of message, then creates cluster labels as type of message;The newly-increased day is filtered using the log template Will;By the log template can not the detail message in matched newly-increased log segment, obtain keyword;According to the pass The word frequency of keyword resequences to the keyword, and the high keyword of word frequency is come front;According to the pass of the rearrangement Keyword creates multiway tree, and root node of each type of message as tree, the keyword of the rearrangement is as the node set, often The corresponding multiway tree of a type of message;Multiway tree described in depth-first traversal creates institute according to the path in the multiway tree State newly-increased log template corresponding to each type of message of model interchanger.
In some optional implementations of the present embodiment, creating unit 604 is also configured to: if the multiway tree In the child node of a node be more than Node B threshold number, then delete all child nodes of the node, the node is as most The latter child node.
In some optional implementations of the present embodiment, cluster labels are created as type of message, comprising: will be each The unknown log of type of message is divided into five kinds of classifications according to semanteme and assigns weighted value respectively, and five kinds of classifications include: Only number or number and symbol, number, letter and symbol, symbol and letter are only alphabetical, only symbol;Extract the day The log is converted the word frequency vector for fixing five length by the frequency of five classes semanteme in will;Calculate the word frequency to The similarity of amount and known type of message set, obtains one group of similarity result, presets if maximum similarity is more than or equal to Similarity threshold, then the log is grouped into corresponding type of message;If maximum similarity is less than preset similarity threshold It is worth, then the type of message that the word frequency vector is new as one.
In some optional implementations of the present embodiment, creating unit 604 is also configured to: when a log template When being the subset of another log template, identify whether a node is a paths by way of labelling to node End.
In some optional implementations of the present embodiment, creating unit 604 is also configured to: by each type of message In template according to tree depth ordering, it is preferential using tree when a log template is the subset of another log template The biggish template of depth matched.
Below with reference to Fig. 7, it illustrates the computer systems 700 for the server for being suitable for being used to realize the embodiment of the present application Structural schematic diagram.
As shown in fig. 7, computer system 700 include central processing unit 603 (CPU) 701, can according to be stored in only It reads the program in memory (ROM) 702 or is loaded into the program in random access storage device (RAM) 703 from storage section 708 And execute various movements appropriate and processing.In RAM 703, also it is stored with system 700 and operates required various program sum numbers According to.CPU 701, ROM 702 and RAM 703 are connected with each other by bus 704.Input/output (I/O) interface 705 also connects To bus 704.
I/O interface 705 is connected to lower component: the importation 706 including keyboard, mouse etc.;It is penetrated including such as cathode The output par, c 707 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.;Storage section 708 including hard disk etc.; And the communications portion 709 of the network interface card including LAN card, modem etc..Communications portion 709 via such as because The network of spy's net executes communication process.Driver 710 is also connected to I/O interface 705 as needed.Detachable media 711, such as Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 710, in order to read from thereon Computer program be mounted into storage section 708 as needed.
Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be tangibly embodied in machine readable Computer program on medium, the computer program include the program code for method shown in execution flow chart.At this In the embodiment of sample, which can be downloaded and installed from network by communications portion 709, and/or from removable Medium 711 is unloaded to be mounted.When the computer program is executed by central processing unit 603 (CPU) 701, the side of the application is executed The above-mentioned function of being limited in method.
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the application, method and computer journey The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation A part of one module, program segment or code of table, a part of the module, program segment or code include one or more Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants It is noted that the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart, Ke Yiyong The dedicated hardware based system of defined functions or operations is executed to realize, or can be referred to specialized hardware and computer The combination of order is realized.
Being described in unit involved in the embodiment of the present application can be realized by way of software, can also be by hard The mode of part is realized.Described unit also can be set in the processor, for example, can be described as: a kind of processor packet Include acquiring unit, resolution unit, processing unit and creating unit.Wherein, the title of these units not structure under certain conditions The restriction of the pairs of unit itself, for example, acquiring unit is also described as " obtaining the original day of the interchanger of a model The unit of will ".
As on the other hand, present invention also provides a kind of nonvolatile computer storage media, the non-volatile calculating Machine storage medium can be nonvolatile computer storage media included in device described in above-described embodiment;It is also possible to Individualism, without the nonvolatile computer storage media in supplying terminal.Above-mentioned nonvolatile computer storage media is deposited One or more program is contained, when one or more of programs are executed by an equipment, so that the equipment: obtaining The original log of the interchanger of one model;The type of message and detail message in the original log are obtained, if obtained not To type of message, then cluster labels are created as type of message;The detail message is segmented, keyword is obtained;According to The word frequency of the keyword resequences to the keyword, and the high keyword of word frequency is come front;It is arranged again according to described The keyword of sequence creates multiway tree, and root node of each type of message as tree, the keyword of the rearrangement is as tree Node, the corresponding multiway tree of each type of message;Multiway tree described in depth-first traversal, according to the path in the multiway tree Create log template corresponding to each type of message of the model interchanger.
Above description is only the preferred embodiment of the application and the explanation to institute's application technology principle.Those skilled in the art Member is it should be appreciated that invention scope involved in the application, however it is not limited to technology made of the specific combination of above-mentioned technical characteristic Scheme, while should also cover in the case where not departing from the inventive concept, it is carried out by above-mentioned technical characteristic or its equivalent feature Any combination and the other technical solutions formed.Such as features described above has similar function with (but being not limited to) disclosed herein Can technical characteristic replaced mutually and the technical solution that is formed.

Claims (12)

1. a kind of method for creating interchanger log template, which is characterized in that the described method includes:
Obtain the original log of the interchanger of a model;
The type of message and detail message in the original log are obtained, if obtained less than type of message, creates cluster mark Label are used as type of message;
The detail message is segmented, keyword is obtained;
It is resequenced according to the word frequency of the keyword to the keyword, the high keyword of word frequency is come into front;
Multiway tree is created according to the keyword of the rearrangement, root node of each type of message as tree is described to arrange again Node of the keyword of sequence as tree, the corresponding multiway tree of each type of message;
Multiway tree described in depth-first traversal creates each message of the model interchanger according to the path in the multiway tree Log template corresponding to type.
2. the method for creation interchanger log template according to claim 1, which is characterized in that the method also includes:
Obtain the newly-increased log of the interchanger of a model;
The type of message and detail message in the newly-increased log are obtained, if obtained less than type of message, creates cluster mark Label are used as type of message;
The newly-increased log is filtered using the log template;
By the log template can not the detail message in matched newly-increased log segment, obtain keyword;
It is resequenced according to the word frequency of the keyword to the keyword, the high keyword of word frequency is come into front;
Multiway tree is created according to the keyword of the rearrangement, root node of each type of message as tree is described to arrange again Node of the keyword of sequence as tree, the corresponding multiway tree of each type of message;
Multiway tree described in depth-first traversal creates the described newly-increased of the model interchanger according to the path in the multiway tree Newly-increased log template corresponding to each type of message in log.
3. the method for creation interchanger log template according to claim 1 or 2, which is characterized in that if the multi-fork The child node of a node is more than Node B threshold number in tree, then deletes all child nodes of the node, the node conduct The last one child node.
4. the method for creation interchanger log template according to claim 1 or 2, which is characterized in that the creation cluster Label is as type of message, comprising:
The unknown log of each type of message is divided into five kinds of classifications according to semanteme and assigns weighted value respectively, described five kinds Classification includes: only digital or number and symbol, and number, letter and symbol, symbol and letter are only alphabetical, only symbol;
The log is converted the word frequency vector for fixing five length by the frequency for extracting five classes semanteme in the log;
The similarity for calculating the word frequency vector Yu known type of message set obtains one group of similarity result, if maximum Similarity is more than or equal to preset similarity threshold, then the log is grouped into corresponding type of message;If maximum similarity Less than preset similarity threshold, then the type of message that the word frequency vector is new as one.
5. the method for creation interchanger log template according to claim 1 or 2, which is characterized in that when a log mould When plate is the subset of another log template, identify whether a node is Yi Tiaolu by way of labelling to node The end of diameter.
6. the method for creation interchanger log template according to claim 5, which is characterized in that will be in each type of message Log template according to tree depth ordering, it is preferential to use when a log template is the subset of another log template The biggish log template of the depth of tree is matched.
7. a kind of device for creating interchanger log template, which is characterized in that described device includes:
Acquiring unit is configured to obtain the original log of the interchanger of a model;
Resolution unit is configured to obtain type of message and detail message in the original log, if obtained less than message Type then creates cluster labels as type of message;
Processing unit is configured to segment the detail message, obtains keyword;According to the word frequency pair of the keyword The keyword rearrangement, comes front for the high keyword of word frequency;
Creating unit is configured to create multiway tree according to the keyword of the rearrangement, and each type of message is as tree Root node, node of the keyword of the rearrangement as tree, the corresponding multiway tree of each type of message;Depth-first time The multiway tree is gone through, day corresponding to each type of message for creating the model interchanger according to the path in the multiway tree Will template.
8. the device of creation interchanger log template according to claim 7, which is characterized in that described device also configures use In:
Obtain the newly-increased log of the interchanger of a model;
The type of message and detail message in the newly-increased log are obtained, if obtained less than type of message, creates cluster mark Label are used as type of message;
The newly-increased log is filtered using the log template;
By the log template can not the detail message in matched newly-increased log segment, obtain keyword;
It is resequenced according to the word frequency of the keyword to the keyword, the high keyword of word frequency is come into front;
Multiway tree is created according to the keyword of the rearrangement, root node of each type of message as tree is described to arrange again Node of the keyword of sequence as tree, the corresponding multiway tree of each type of message;
Multiway tree described in depth-first traversal creates the described newly-increased of the model interchanger according to the path in the multiway tree Newly-increased log template corresponding to each type of message in log.
9. the device of creation interchanger log template according to claim 7 or 8, which is characterized in that the creating unit It is also configured to:
If the child node of a node is more than Node B threshold number in the multiway tree, all sub- sections of the node are deleted Point, the node is as the last one child node.
10. the device of creation interchanger log template according to claim 7 or 8, which is characterized in that the creation cluster Label is as type of message, comprising:
The unknown log of each type of message is divided into five kinds of classifications according to semanteme and assigns weighted value respectively, described five kinds Classification includes: only digital or number and symbol, and number, letter and symbol, symbol and letter are only alphabetical, only symbol;
The log is converted the word frequency vector for fixing five length by the frequency for extracting five classes semanteme in the log;
The similarity for calculating the word frequency vector Yu known type of message set obtains one group of similarity result, if maximum Similarity is more than or equal to preset similarity threshold, then the log is grouped into corresponding type of message;If maximum similarity Less than preset similarity threshold, then the type of message that the word frequency vector is new as one.
11. the device of creation interchanger log template according to claim 7 or 8, which is characterized in that the creating unit It is also configured to:
When a log template is the subset of another log template, one is identified by way of labelling to node Node whether be a paths end.
12. the device of creation interchanger log template according to claim 11, which is characterized in that the creating unit is also It is configured to:
By the template in each type of message according to the depth ordering of tree, when a log template is another log template When subset, the preferential biggish template of depth using tree is matched.
CN201610355129.3A 2016-05-25 2016-05-25 The method and apparatus for creating interchanger log template Active CN106055452B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610355129.3A CN106055452B (en) 2016-05-25 2016-05-25 The method and apparatus for creating interchanger log template

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610355129.3A CN106055452B (en) 2016-05-25 2016-05-25 The method and apparatus for creating interchanger log template

Publications (2)

Publication Number Publication Date
CN106055452A CN106055452A (en) 2016-10-26
CN106055452B true CN106055452B (en) 2019-06-14

Family

ID=57175843

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610355129.3A Active CN106055452B (en) 2016-05-25 2016-05-25 The method and apparatus for creating interchanger log template

Country Status (1)

Country Link
CN (1) CN106055452B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108696899B (en) * 2017-04-07 2021-09-03 北京京东尚科信息技术有限公司 SIP message transmitting and receiving method and transmitting and receiving device
CN110096411A (en) * 2019-03-22 2019-08-06 西安电子科技大学 Log template rapid extracting method and system based on association analysis and time window
CN110134615B (en) * 2019-04-10 2022-03-01 百度在线网络技术(北京)有限公司 Method and device for acquiring log data by application program
CN114116410A (en) * 2022-01-28 2022-03-01 北京安帝科技有限公司 Log analysis method and system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101320348A (en) * 2008-06-25 2008-12-10 中兴通讯股份有限公司 Log function implementing method of embedded system
CN103379136A (en) * 2012-04-17 2013-10-30 中国移动通信集团公司 Compression method and decompression method of log acquisition data, compression apparatus and decompression apparatus of log acquisition data
CN104408100A (en) * 2014-11-19 2015-03-11 北京融海恒信咨询有限公司 Compression method for structured web log

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050203934A1 (en) * 2004-03-09 2005-09-15 Microsoft Corporation Compression of logs of language data
US7676282B2 (en) * 2005-12-02 2010-03-09 International Business Machines Corporation Compression of servo control logging entries

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101320348A (en) * 2008-06-25 2008-12-10 中兴通讯股份有限公司 Log function implementing method of embedded system
CN103379136A (en) * 2012-04-17 2013-10-30 中国移动通信集团公司 Compression method and decompression method of log acquisition data, compression apparatus and decompression apparatus of log acquisition data
CN104408100A (en) * 2014-11-19 2015-03-11 北京融海恒信咨询有限公司 Compression method for structured web log

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
基于差分压缩的大规模日志压缩系统;唐球等;《通信学报》;20151130;第1-6页
基于网格的TCP网络日志二次聚类算法;张珠玉等;《暨南大学学报(自然科学版)》;20110430;第1-6页

Also Published As

Publication number Publication date
CN106055452A (en) 2016-10-26

Similar Documents

Publication Publication Date Title
CN106055452B (en) The method and apparatus for creating interchanger log template
CN106055608B (en) The method and apparatus of automatic collection and analysis interchanger log
CN105049247B (en) A kind of network security log template abstracting method and device
CN105072196B (en) The storage of distributed data packet, retrogressive method and system
CN109284334A (en) Real-time data base synchronous method, device, electronic equipment and storage medium
US8909562B2 (en) Markov modeling of service usage patterns
CN104778178B (en) A kind of application class method, apparatus and service server
US8326982B2 (en) Method and apparatus for extracting and visualizing execution patterns from web services
CN110362544A (en) Log processing system, log processing method, terminal and storage medium
CN105677567A (en) Automation testing method and system
WO2019237532A1 (en) Service data monitoring method, storage medium, terminal device and apparatus
CN110855473A (en) Monitoring method, device, server and storage medium
US20200021511A1 (en) Performance analysis for transport networks using frequent log sequence discovery
US20080109540A1 (en) Method and System for Distributing Software Components
CN105550179A (en) Webpage collection method and browser plug-in
CN103475532A (en) Hardware detection method and system thereof
JP6078485B2 (en) Operation history analysis apparatus, method, and program
CN111222547B (en) Traffic feature extraction method and system for mobile application
US8327292B2 (en) Distinct groupings of related objects for display in a user interface
CN108363654A (en) Association's processing method, system and the electronic equipment of system log
CN110061931B (en) Industrial control protocol clustering method, device and system and computer storage medium
WO2022048668A1 (en) Knowledge graph construction method and apparatus, check method and storage medium
CN110096411A (en) Log template rapid extracting method and system based on association analysis and time window
CN106612207B (en) The management-control method and device of ethernet device RSTP information
CN113760730A (en) Automatic testing method and device

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant