CN110262942A - A kind of log analysis method and device - Google Patents
A kind of log analysis method and device Download PDFInfo
- Publication number
- CN110262942A CN110262942A CN201910509655.4A CN201910509655A CN110262942A CN 110262942 A CN110262942 A CN 110262942A CN 201910509655 A CN201910509655 A CN 201910509655A CN 110262942 A CN110262942 A CN 110262942A
- Authority
- CN
- China
- Prior art keywords
- log
- segment
- exploitation
- application program
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 137
- 238000000034 method Methods 0.000 claims abstract description 48
- 238000011161 development Methods 0.000 claims abstract description 37
- 238000012545 processing Methods 0.000 claims abstract description 37
- 238000003860 storage Methods 0.000 claims description 25
- 238000012549 training Methods 0.000 claims description 19
- 238000012217 deletion Methods 0.000 claims description 6
- 230000037430 deletion Effects 0.000 claims description 6
- 230000010365 information processing Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 9
- 230000007547 defect Effects 0.000 description 5
- 238000000605 extraction Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000000977 initiatory effect Effects 0.000 description 3
- 239000000284 extract Substances 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 241000208340 Araliaceae Species 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000004064 recycling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/362—Debugging of software
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention discloses a kind of log analysis method and devices, are applied to technical field of information processing.In the method for the present embodiment, log analysis device can be according to time interval, the exploitation log that will acquire is divided into multiple log segments, then further according to preset log analysis strategy, classify to the problem of each log segment corresponding application program, finally further according to obtained multiple Question Classifications as a result, determining the processing information within the corresponding period of application development based on application program.It is analyzed in this way by the multiple log segments for including to exploitation log, and the corresponding time interval of each log segment, so as to be accurately located application program in the process of development, which development process will appear problem, and then perform corresponding processing, improve the development efficiency of application program.
Description
Technical field
The present invention relates to technical field of information processing, in particular to a kind of log analysis method and device.
Background technique
Existing log analysis method be mainly by terminal device run application program during running log into
Row analysis, to obtain the failure of which terminal device operation application program, either, application program is in actual application
Loophole etc..
Specifically, existing a kind of software defect fault recognition method based on internet daily record data, primarily directed to
The Internet sources syslog data and custom system Source log data, using the Internet sources syslog data as training set and from
Middle extraction feature generates software defect fault log identification prediction model by machine learning or similarity mode；For user
System source daily record data, analysis identification obtain the log segment of wherein characterization software defect failure, to obtain for user system
The software defect fault type of system log.But if when custom system Source log data are more, using existing based on interconnection
When the software defect fault recognition method of net daily record data carries out fault location, the time of cost is larger.
Summary of the invention
The embodiment of the present invention provides a kind of log analysis method and device, realizes through the day master chip to exploitation log
Section determines the processing information within the corresponding period of application program based on application program.
First aspect of the embodiment of the present invention provides a kind of log analysis method, comprising:
Obtain the exploitation log of application program；
According to time interval, the exploitation log is divided into multiple log segments；
According to preset log analysis strategy, the problem of each log segment corresponding application program, is divided respectively
Class obtains multiple Question Classification results；
The application is based within the corresponding period of application development as a result, determining according to the multiple Question Classification
The processing information of program.
First aspect of the embodiment of the present invention provide a kind of system analysis method one kind in the specific implementation, it is described according to when
Between be spaced, the exploitation log is divided into multiple log segments, is specifically included:
According to time interval, exploitation log is divided into multiple groups log segment, includes multiple logs in every group of log segment
Segment.
Second aspect of the embodiment of the present invention provides a kind of log analysis method, comprising:
Log acquisition unit, for obtaining the exploitation log of application program；
Division unit, for according to time interval, the exploitation log to be divided into multiple log segments；
Taxon, for according to preset log analysis strategy, respectively to the corresponding application program of each log segment
The problem of classify, obtain multiple Question Classification results；
Handle determination unit, for according to the multiple Question Classification as a result, determine application development it is corresponding when
Between the processing information based on the application program in section.
The third aspect of the embodiment of the present invention provides a kind of storage medium, and the storage medium stores a plurality of instruction, the finger
It enables and being suitable for as processor loads and executes the log analysis method as described in first aspect of the embodiment of the present invention.
Fourth aspect of the embodiment of the present invention provides a kind of server, comprising: including pocessor and storage media, the processing
Device, for realizing each instruction；The storage medium is for storing a plurality of instruction, and described instruction is for being loaded and being held by processor
Log analysis method of the row as described in first aspect of the embodiment of the present invention.
As it can be seen that log analysis device can be according to time interval, and the exploitation log that will acquire is drawn in the method for the present embodiment
It is divided into multiple log segments, then further according to preset log analysis strategy, to the corresponding application program of each log segment
Problem is classified, finally further according to obtained multiple Question Classifications as a result, determining the corresponding period in application development
The interior processing information based on application program.It is analyzed in this way by the multiple log segments for including to exploitation log, and it is each
Log segment corresponds to a time interval, so as to be accurately located application program in the process of development, which development process
It will appear problem, and then perform corresponding processing, improve the development efficiency of application program.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention without any creative labor, may be used also for those of ordinary skill in the art
To obtain other drawings based on these drawings.
Fig. 1 is the structural schematic diagram for the system that a kind of log analysis method provided in an embodiment of the present invention is applied to；
Fig. 2 is a kind of flow chart of log analysis method provided by one embodiment of the present invention；
Fig. 3 a is a kind of schematic diagram for dividing exploitation log in one embodiment of the invention；
Fig. 3 b is another schematic diagram for dividing exploitation log in one embodiment of the invention；
Fig. 4 is the method flow diagram of training log analysis model in one embodiment of the invention；
Fig. 5 is a kind of schematic diagram for log analysis method that Application Example of the present invention provides；
Fig. 6 is to carry out pretreated schematic diagram to exploitation log in Application Example of the present invention；
Fig. 7 is a kind of structural schematic diagram of log analysis device provided in an embodiment of the present invention；
Fig. 8 is a kind of structural schematic diagram of server provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Description and claims of this specification and term " first ", " second ", " third " " in above-mentioned attached drawing
The (if present)s such as four " are to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should manage
The data that solution uses in this way are interchangeable under appropriate circumstances, so that the embodiment of the present invention described herein for example can be to remove
Sequence other than those of illustrating or describe herein is implemented.In addition, term " includes " and " having " and theirs is any
Deformation, it is intended that cover not exclusively include, for example, containing the process, method of a series of steps or units, system, production
Product or equipment those of are not necessarily limited to be clearly listed step or unit, but may include be not clearly listed or for this
A little process, methods, the other step or units of product or equipment inherently.
The embodiment of the present invention provides a kind of log analysis method, mainly can be applied in system as shown in Figure 1, at this
Include: log analysis device in system, is mainly used for obtaining the exploitation log of application program, and carry out the exploitation log of acquisition
Analysis, so that it is determined that the processing information based on application program.
It specifically, within the system can also include log terminal, wherein log terminal, which refers to, generates opening for application program
The terminal that the terminal of hair log, the mainly development of user of application program use during development and application program, for example, opening
Edit the terminal of the code of application program, terminal and the terminal of type information of debugging code etc. in hair family.These log terminals
In the development process of application program, operation of the log terminal performed by various time points, the day of various time points will record
Will information.
Further, it can also include: processing information receiving terminal in system, be mainly used for receiving log analysis device point
The processing information based on application program of hair specifically can be any terminal for receiving information, such as mail applications terminal, or
The terminal (such as above-mentioned log terminal etc.) of the development of user of application program.
In the present embodiment, log analysis device can carry out log analysis as follows:
Obtain the exploitation log of application program；According to time interval, the exploitation log is divided into multiple log segments
(in figure by taking n log segment as an example)；It is corresponding to each log segment respectively to apply journey according to preset log analysis strategy
The problem of sequence, classifies, and obtains multiple Question Classification results；According to the multiple Question Classification as a result, determining in application program
Processing information based on the application program in the corresponding period of exploitation.
In this way by analyzing exploitation log multiple log segments for including, and when each log segment is one corresponding
Between be spaced, so as to be accurately located application program in the process of development, which development process will appear problem, and then carry out
Corresponding processing, improves the development efficiency of application program.
One embodiment of the invention provides a kind of log analysis method, performed by mainly above-mentioned log analysis device
Method, flow chart are as shown in Figure 2, comprising:
Step 101, the exploitation log of application program is obtained.
It is appreciated that log analysis device can directly initiate the process of the present embodiment according to certain period, in this way,
In one case, log analysis device can get a upper stream when initiating the process of the present embodiment from log terminal
The Cheng Faqi period to the exploitation log of the application program between current period, specifically includes: various time points in this period
Log information.
In another case, log terminal produce exploitation log after, can active reporting to log analysis device into
Row storage can directly extract upper one when process of the log analytical equipment in initiation the present embodiment from being locally stored
Process initiates the period to the exploitation log between current period.
Exploitation log in the present embodiment refers to that the development of user of application program passes through log terminal development application program
In the process, log information caused by log terminal, for example, the log of the code of log terminal editor's application program, debugs generation
Log and the log of type information of code etc..
Step 102, according to time interval, exploitation log is divided into multiple log segments.
Specifically, log analysis device can according to but be not limited to the following two kinds mode to exploitation log carry out dividing it
In:
(1) using preset maximum time interval as the time window of exploitation log；When time window is moved along exploitation log
When, using the log information in time window as a log segment, wherein time window is along the step-length that moves of exploitation log and pre-
The minimum interval set is consistent.Under normal conditions, the generation of a problem based on application program is often in some moment,
And relevant log information is then printed in the short time after the moment, and therefore, the time that exploitation log is divided
Interval can be minimum time or maximum time of problem log of problem log etc..
Such as shown in Fig. 3 a, using preset maximum time interval Tmax as the time window of exploitation log, the time window meeting
It is moved along exploitation log, mobile step-length is preset minimum interval Tmin, and arrow indicates time window along exploitation day
The mobile direction of will.When time window is when developing some position of log, the log information in the time window is log segment 1；
After time window moves minimum interval Tmin along exploitation log, the log information in the time window is log segment 2, with
This analogizes, until time window is moved to the last segment of exploitation log, using last segment as another log segment.
(2) log analysis device can carry out more wheels to exploitation log according to time interval and divide, and the division of every wheel can be with
One group of log segment is obtained, in this way, exploitation log can be divided into multiple groups log point according to time interval by log analysis device
Section, it include multiple log segments in every group of log segment.
In a kind of specific embodiment, log analysis device can will be developed first according to preset maximum time interval
Log is divided into one group of log segment；Then the log information in minimum interval preset in exploitation log is deleted, and is pressed
According to preset maximum time interval, the exploitation log after deletion is divided into another group of log segment.Further, log analysis
Device can also delete the log information in minimum interval preset in the exploitation log after deletion, and according to preset
Exploitation log after deletion is divided into another group of log segment by maximum time interval；Then for the exploitation log after deleting
Circulation executes the step of deleting and dividing.Wherein, preset maximum time interval can be the maximum time of problem log, and pre-
The minimum interval set can be the minimum time of problem log.Wherein, in every group of log segment each log segment when
Between interval be equal to or less than preset maximum time interval
Such as exploitation log shown in Fig. 3 b includes T moment corresponding log information, preset minimum interval can be with
It is Tmin, maximum time interval can be Tmax.
When dividing for the first time, the log information at T moment is respectively divided into the day master chip that 3 time intervals are Tmax
The log segment that section and time interval are T1, wherein T1 is less than Tmax；When dividing again, since developing log, delete
Time interval is the log information of Tmin, and the log information at T-Tmin moment is respectively divided into 3 time intervals and is
The log segment and time interval of Tmax is the log segment of T2, wherein T2 is less than Tmax；When dividing again, from T-Tmin
The beginning of the log information at moment, is divided into the log information of Tmin between erasing time, and by the log information at T-2Tmin moment
It is respectively divided into the log segment that 2 time intervals are Tmax and the log segment that time interval is T3, wherein T3 is less than
Tmax.In this way, 3 groups of log segments can be formed, log segment group 1 and log segment group 2 respectively include 4 log segments, and
Log segment group 3 includes 3 log segments.
Step 103, according to preset log analysis strategy, respectively to each log segment corresponding application program the problem of
Classify, obtains multiple Question Classification results.
Specifically, log analysis device can first obtain the crucial label of each log segment respectively；Then further according to each
The crucial label of a log segment and preset log analysis strategy, respectively to the corresponding application program of each log segment
Problem is classified, and obtains multiple Question Classifications as a result, each Question Classification result corresponds to a log segment.
Wherein, due to the data that the log segment of exploitation log is a kind of blended data format, there is structural data and non-
Structural data, therefore, if including structural data in some log segment, the crucial label of the log segment includes
The label for including in structural data, for example, system label, log rank (mistake, warning, debugging etc.) or definition label etc.；
If in a certain log segment including unstructured data, the crucial label of the log segment mainly passes through keyword extraction
It obtains, specifically, after can first segmenting to unstructured data, obtains multiple participles, then determined in multiple participles again
Certain participles as crucial label.
The corresponding Question Classification result of any of the above-described log segment may include: asking for the corresponding application program of log segment
Type is inscribed, fault type or loophole (BUG) type etc. is can be, is also possible to without failure or loophole other types.
Above-mentioned preset log analysis strategy can be the operation logic of log analysis model, in this way, log analysis model
Question Classification result directly can be exported by the crucial label of log segment；Above-mentioned preset log analysis strategy may be
The strategy of similarity calculation, in this way, log analysis device can calculate log segment respectively with preset various problem types
Similarity between log information, the similarity between same day master chip section and the log information of a certain problem types are greater than a certain
Threshold value, it is determined that include the problem types in classification results the problem of the log segment.
Step 104, according to multiple Question Classifications as a result, determining within the corresponding period that application program is opened based on using journey
The processing information of sequence.
Due to the corresponding log segment of a Question Classification result, and a log segment corresponds to a time interval,
In this way, a Question Classification result can indicate failure or the loophole etc. that application program occurs in the certain time period of exploitation,
Then when executing step 104, log analysis device can be for each Question Classification as a result, being given at the phase of application development
Answer the solution in the period, i.e. the processing information based on application program.
It specifically, can preset problem types pass corresponding with the processing information based on application program in log analysis device
System, in this way, determine handle information when, the problem of above-mentioned steps 103 being obtained classification results and preset corresponding relationship into
Row matching, so as to obtain the final processing information based on application program.Wherein, for fault type or loophole type
Question Classification is as a result, corresponding processing information can be fault solution or loophole solution；For non-faulting or loophole
The problem of type classification results, corresponding processing information can be with are as follows: sends notification to the terminal etc. of the development of user of application program.
It should be noted that obtaining multiple groups log segment if divided in above-mentioned steps 102, include in every group of log segment
Multiple log segments, in this way, log analysis device when executing step 103, can get each log in each group log segment
The corresponding Question Classification result of segment.In turn, log analysis device, can be in conjunction with multiple groups log point when executing this step 104
The corresponding Question Classification of log segment in section is as a result, determine the processing information within the corresponding period of application development.
For example, can respectively obtain one for each log segment in 3 groups of log segments shown in above-mentioned Fig. 3 b and ask
It is corresponding with second day master chip section Tmax2 to inscribe first log segment Tmax1 in classification results, such as log segment group 1
Question Classification result are as follows: fault-free and problem types 1, but first log segment Tmax1 is corresponding in log segment group 2 asks
Inscribe classification results are as follows: problem types 1, then log analysis device can determine first log segment in log segment group 2
In the Tmax1 corresponding period, there is the problem of problem types 1 in the exploitation of application program, needs to provide corresponding solution,
Handle information.
Further, the processing information based on application program can also be sent to respective handling information by log analysis device
Receive terminal.
As it can be seen that log analysis device can be according to time interval, and the exploitation log that will acquire is drawn in the method for the present embodiment
It is divided into multiple log segments, then further according to preset log analysis strategy, to the corresponding application program of each log segment
Problem is classified, finally further according to obtained multiple Question Classifications as a result, determining the corresponding period in application development
The interior processing information based on application program.It is analyzed in this way by the multiple log segments for including to exploitation log, and it is each
Log segment corresponds to a time interval, so as to be accurately located application program in the process of development, which development process
It will appear problem, and then perform corresponding processing, improve the development efficiency of application program.
It should be noted that if above-mentioned log analysis strategy includes the operation logic of log analysis model.At one
In specific embodiment, log analysis device can train log analysis model, flow chart such as Fig. 4 institute in accordance with the following steps
Show, comprising:
Step 201, log analysis initial model is determined.
It is appreciated that log analysis device when determining log analysis initial model, can determine whether log analysis initial model
The initial value of preset parameter in included multilayered structure and each layer mechanism specifically includes features described above extraction module and log point
Generic module, wherein the characteristic information that characteristic extracting module is used to extract any log sample specifically can first obtain log
Then the crucial label of sample obtains the feature vector of log sample further according to the crucial label of log sample；Log classification mould
The problem of block is used for the characteristic information that extracts according to characteristic extracting module, obtains log sample corresponding application program type.Tool
Body, the multilayered structure in log analysis initial model can be following any algorithm structure: convolutional neural networks
(Convolutional Neural Network, CNN), support vector machines (Support Vector Machines, SVM) etc.
Deng.
Wherein, preset parameter refers to the fixation that each layer structure is used in calculating process in log analysis initial model
, do not need the parameter of assignment at any time, such as weight, the parameters such as angle.
Step 202, it determines training sample, includes multiple log samples in training sample and each log sample is corresponding asks
Inscribe type.
Step 203, the problem of corresponding application program of each log sample being determined by log analysis initial model respectively
Classify, obtains Question Classification initial results.
Specifically, the crucial label of log sample is obtained by the characteristic extracting module in log analysis initial model, so
The feature vector of log sample is obtained further according to the crucial label of log sample afterwards；Log categorization module is used for according to feature extraction
The characteristic information that module is extracted, the problem of obtaining log sample corresponding application program type.
Step 204, the problem classification initial results determined according to log analysis initial model in above-mentioned steps 203, and instruction
Practice the problems in sample type, the preset parameter value in log analysis initial model is adjusted, to obtain final log analysis mould
Type.
Specifically, log analysis, which is set, first to classify according to the problem that log analysis initial model in above-mentioned steps 203 determines
The problems in initial results and training sample type calculate loss function relevant to log analysis initial model, the loss letter
Number is used to indicate the error that log analysis initial model classifies to the problem of each log sample corresponding application program.
Here, loss function includes: that each log sample for indicating to be determined according to log analysis initial model is corresponding
Difference between Question Classification initial results, with the actual problem types of log sample each in training sample.These errors
The mathematics form of expression establishes loss function usually using cross entropy loss function, and the training process of log analysis initial model
Exactly need to reduce to the greatest extent the value of above-mentioned error, which is a series of by backpropagation derivation and gradient decline etc.
Mathematical optimization means constantly optimize the parameter value of preset parameter in the log analysis initial model determined in above-mentioned steps 201,
And the calculated value of above-mentioned loss function is minimized.
Therefore, after loss function is calculated, log analysis device needs to adjust log according to the loss function of calculating
The preset parameter value in initial model is analyzed, to obtain final log analysis model.Specifically, when the loss function of calculating
When functional value is larger, for example it is greater than preset value, then needs to change preset parameter value, for example the weighted value of some weight is reduced
Deng so that the functional value of the loss function calculated according to preset parameter value adjusted reduces.
It should be noted that above-mentioned steps 203 to 204 are at the beginning of Question Classification is calculated by log analysis initial model
Begin as a result, primary adjustment according to Question Classification initial results to the preset parameter value in log analysis initial model, and in reality
It in the application of border, needs to execute above-mentioned steps 203 to 204 by constantly recycling, until the adjustment to preset parameter value meets one
Until fixed stop condition.
Therefore, log analysis device is after performing above-described embodiment step 201 to 204, it is also necessary to which judgement is current right
Whether the adjustment of preset parameter value meets preset stop condition, when meeting, then terminates process；When being unsatisfactory for, then it is directed to
Log analysis initial model after adjusting preset parameter value returns and executes above-mentioned steps 203 to 204.
Wherein, preset stop condition includes but is not limited to any one of following condition: the fixed ginseng currently adjusted
For the difference of numerical value and the preset parameter value of last adjustment less than a threshold value, that is, the preset parameter value adjusted reaches convergence；And it is right
The adjustment number of preset parameter value is equal to preset number etc..
The log classification method in the present invention is illustrated with specific application example below, the method for the present embodiment can
To be applied in system shown in FIG. 1 as above, and log analysis strategy preset in the present embodiment is specially log analysis model
Operation logic, log analysis model can through the foregoing embodiment in method training obtain.
As shown in figure 5, the method for the present embodiment includes the following steps:
Step 301, during the development of user of application program passes through log terminal development application program, for example editor answers
With the code of program, during debugging code and type information, log terminal will record the behaviour of the log terminal of each moment
Make information, i.e. exploitation log.
Step 302, log analysis device can initiate log analysis process, such log analysis device according to certain period
The last initiation period can be obtained to log terminal to the log information between current period, i.e. exploitation log.
Step 303, log analysis device first pre-processes the exploitation log of acquisition, specifically, log analysis device
The abnormal data in exploitation log can be removed, for example, the exploitation log etc. of format error.
Specifically, as shown in fig. 6, log analysis device can pass through during carrying out pretreated to exploitation log
Source data module and pretreated model, modules can be respectively created a data flow block, and pass through input (Put) module and defeated
(Take) module pre-processes out to realize, in which:
Input module in source data module is for the number constantly into the data flow block of creation in addition exploitation log
According to waiting if data in data flow block are full；Output module in source data module, for export specified size or
The data of designated length, when data flow block be sky, then wait.
Input module in preprocessing module into the data flow block of creation for constantly adding the output of source data module
Data waited if data in data flow block are full；Output module in preprocessing module, for in data flow block
Data pre-processed, and export pretreated exploitation log, if data flow block is sky, wait.
Step 304, log analysis device divides pretreated exploitation log, specifically, pretreated to open
Hair log is existed in the form of character string, and the characteristic with continuity and time series can be by pretreated exploitation day
Will is divided into multiple groups log segment according to time interval, includes multiple log segments in every group of log segment.
Step 305, log analysis device is according to preset log analysis model to each day master chip in every group of log segment
The problem of section corresponding application program, classifies, and obtains multiple groups Question Classification as a result, including each in every group of Question Classification result
The corresponding problem types of a log segment.
Specifically, the crucial label of each log segment can be first got by log analysis model, then further according to
Crucial label directly exports the problem of each log segment type.Above-mentioned reality wherein is shown in the acquisition of the crucial label of log segment
It applies described in example, herein without repeating.
Step 306, log analysis device according to the corresponding Question Classification of log segment each in every group of log segment as a result,
Determine the processing information within the corresponding period of application development based on application program.
Step 307, log analysis device notifies the processing information in each period to the end of corresponding development of user
End, i.e. processing information receiving terminal.
As it can be seen that using the exploitation log of application program as analysis object, and passing through log analysis mould in the embodiment of the present invention
Type realizes quick Question Classification and positioning, and then provides the i.e. above-mentioned processing information of solution, improves application program
Development efficiency.The exploitation log wherein used has richer information, and availability is stronger, and then the classification of Upgrade Problem and fixed
Position is notified that the terminal of development of user in addition, realizing determining processing information in the present embodiment, reduces the people of problem circulation
Work interference, further promotes the development efficiency of application program.
The embodiment of the present invention also provides a kind of log analysis device, and structural schematic diagram is as shown in fig. 7, specifically can wrap
It includes:
Log acquisition unit 10, for obtaining the exploitation log of application program；
Division unit 11, for according to time interval, the exploitation log that the log acquisition unit 10 obtains to be divided into
Multiple log segments.
Specifically, in a kind of situation, division unit 11, specifically for using preset maximum time interval as exploitation log
Time window；When the time window is moved along exploitation log, using the log information in the time window as a log
Segment, wherein the time window is consistent with preset minimum interval along the step-length that exploitation log is moved.
In another case, division unit 11, is specifically used for for according to time interval, exploitation log to be divided into multiple groups
Log segment includes multiple log segments in every group of log segment.Wherein, division unit 11 is divided into multiple groups will develop log
When log segment, according to preset maximum time interval, the exploitation log is divided into one group of log segment；It is opened described in deletion
The log information in minimum interval preset in log is sent out, according to preset maximum time interval, after the deletion
Exploitation log is divided into another group of log segment, and the time interval of each log segment is equal to or small in every group of log segment
In preset maximum time interval.
Taxon 12, for being obtained respectively to the division unit 11 each according to preset log analysis strategy
The problem of log segment corresponding application program, classifies, and obtains multiple Question Classification results.
Specifically, taxon 12, for obtaining the crucial label of each log segment respectively；According to described each
The crucial label of a log segment and the preset log analysis strategy, it is corresponding to each log segment respectively to answer
Classified with the problem of program, obtains multiple Question Classification results.
Wherein, when obtaining the crucial label of some log segment, which is specifically used for when described a certain
Include structural data in log segment, determines the label for including in the structural data as a certain log segment
Crucial label；When including unstructured data in a certain log segment, at least one in the unstructured data is determined
Crucial label of a participle as a certain log segment.
Determination unit 13 is handled, multiple Question Classifications for obtaining according to the taxon 12 are as a result, determination is being answered
With the processing information based on the application program in the corresponding period of program development.
Further, if log analysis strategy includes the operation logic of log analysis model, the log of the present embodiment
Analytical equipment can also include: training unit 14, for determining log analysis initial model；Determine training sample, the training
It include multiple log samples and the corresponding problem types of each log sample in sample；Pass through the log analysis initial model point
The problem of other corresponding application program to each log sample, classifies, and obtains Question Classification initial results；According to the day
Will analyzes problem classification the problems in initial results and the training sample type that initial model determines, adjusts the log
The preset parameter value in initial model is analyzed, to obtain final log analysis model.In this way, above-mentioned taxon 12 can basis
The log analysis model that the training of training unit 14 obtains, classifies to the problem of each log segment corresponding application program.
The training unit 14 is also used to when the adjustment number to the preset parameter value is equal to preset number, or works as
When the difference of the preset parameter value of the preset parameter value and last adjustment that currently adjust is less than a threshold value, then stop to described solid
Determine the adjustment of parameter value.
As it can be seen that division unit 11 understands the exploitation that will acquire according to time interval in the log analysis device of the present embodiment
Log is divided into multiple log segments, and then taxon 12 is further according to preset log analysis strategy, to each log segment
The problem of corresponding application program, classifies, finally handle determination unit 13 further according to obtained multiple Question Classifications as a result,
Determine the processing information within the corresponding period of application development based on application program.In this way by including to exploitation log
Multiple log segments analyzed, and the corresponding time interval of each log segment, so as to be accurately located application
In the process of development, which development process will appear problem to program, and then perform corresponding processing, and improve opening for application program
Send out efficiency.
The embodiment of the present invention also provides a kind of server, structural schematic diagram as shown in figure 8, the server can because configuration or
Performance is different and generates bigger difference, may include one or more central processing units (central
Processing units, CPU) 20 (for example, one or more processors) and memory 21, one or more are deposited
Store up the storage medium 22 (such as one or more mass memory units) of application program 221 or data 222.Wherein, it stores
Device 21 and storage medium 22 can be of short duration storage or persistent storage.Be stored in storage medium 22 program may include one or
More than one module (diagram does not mark), each module may include to the series of instructions operation in server.Further
Ground, central processing unit 20 can be set to communicate with storage medium 22, execute on the server a series of in storage medium 22
Instruction operation.
Specifically, application program of the application program 221 stored in storage medium 22 including log analysis, and the program
It may include the log acquisition unit 10 in above-mentioned log analysis device, division unit 11, taxon 12, processing determination unit
13 and training unit 14, herein without repeating.Further, central processing unit 20 can be set to logical with storage medium 22
Letter executes the corresponding sequence of operations of application program of the log analysis stored in storage medium 22 on the server.
Server can also include one or more power supplys 23, one or more wired or wireless network interfaces
24, and/or, one or more operating systems 223, such as Windows ServerTM, Mac OS XTM, UnixTM,
LinuxTM, FreeBSDTM etc..
The step as performed by log analysis device described in above method embodiment can be based on the clothes shown in Fig. 8
The structure of business device.
The embodiment of the present invention also provides a kind of storage medium, and the storage medium stores a plurality of instruction, and described instruction is suitable for
It is loaded as processor and executes the log analysis method as performed by above-mentioned log analysis device.
The embodiment of the present invention also provides another server, including pocessor and storage media, the processor, for real
Existing each instruction；The storage medium is for storing a plurality of instruction, and described instruction is for being loaded by processor and being executed as above-mentioned
Log analysis method performed by log analysis device.
Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of above-described embodiment is can
It is completed with instructing relevant hardware by program, which can be stored in a computer readable storage medium, storage
Medium may include: read-only memory (ROM), random access memory ram), disk or CD etc..
It is provided for the embodiments of the invention log analysis method above and device is described in detail, it is used herein
A specific example illustrates the principle and implementation of the invention, and the above embodiments are only used to help understand
Method and its core concept of the invention；At the same time, for those skilled in the art is having according to the thought of the present invention
There will be changes in body embodiment and application range, in conclusion the content of the present specification should not be construed as to the present invention
Limitation.
Claims (10)
1. a kind of log analysis method characterized by comprising
Obtain the exploitation log of application program；
According to time interval, the exploitation log is divided into multiple log segments；
According to preset log analysis strategy, classifies respectively to the problem of each log segment corresponding application program, obtain
To multiple Question Classification results；
The application program is based within the corresponding period of application development as a result, determining according to the multiple Question Classification
Processing information.
2. the method as described in claim 1, which is characterized in that it is described according to time interval, the exploitation log is divided into
Multiple log segments, specifically include:
Using preset maximum time interval as the time window of exploitation log；
When the time window is moved along exploitation log, using the log information in the time window as a log segment,
Wherein, the step-length that the time window is moved along exploitation log is consistent with preset minimum interval.
3. the method as described in claim 1, which is characterized in that it is described according to time interval, the exploitation log is divided into
Multiple log segments, specifically include:
According to preset maximum time interval, the exploitation log is divided into one group of log segment；
The log information in the exploitation log in preset minimum interval is deleted, according to preset maximum time interval,
Exploitation log after the deletion is divided into another group of log segment；
In every group of log segment include multiple log segments, and the time interval of each log segment be equal to or less than it is preset most
Big time interval.
4. method as described in any one of claims 1 to 3, which is characterized in that it is described according to preset log analysis strategy, divide
The problem of other corresponding application program to each log segment, classifies, and obtains multiple Question Classifications as a result, specifically including:
The crucial label of each log segment is obtained respectively；
According to the crucial label of each log segment and the preset log analysis strategy, respectively to each day
The problem of master chip section corresponding application program, classifies, and obtains multiple Question Classification results.
5. method as claimed in claim 4, which is characterized in that the crucial label for obtaining a certain log segment specifically includes:
When including structural data in a certain log segment, determine described in the label conduct for including in the structural data
The crucial label of a certain log segment；
When including unstructured data in a certain log segment, determine that at least one of described unstructured data segments
Crucial label as a certain log segment.
6. method as described in any one of claims 1 to 3, which is characterized in that the preset log analysis strategy includes day
The operation logic of will analysis model, the method also includes:
Determine log analysis initial model；
It determines training sample, includes multiple log samples and the corresponding problem types of each log sample in the training sample；
Classified respectively to the problem of each log sample corresponding application program by the log analysis initial model, is obtained
To Question Classification initial results；
Problem classification the problems in initial results and the training sample class determined according to the log analysis initial model
Type adjusts the preset parameter value in the log analysis initial model, to obtain final log analysis model.
7. method as claimed in claim 6, which is characterized in that when the adjustment number to the preset parameter value is equal to preset
When number, or when the difference of the preset parameter value currently adjusted and the preset parameter value of last adjustment is less than a threshold value, then
Stop the adjustment to the preset parameter value.
8. a kind of log analysis device characterized by comprising
Log acquisition unit, for obtaining the exploitation log of application program；
Division unit, for according to time interval, the exploitation log to be divided into multiple log segments；
Taxon, for being asked the corresponding application program of each log segment respectively according to preset log analysis strategy
Topic is classified, and multiple Question Classification results are obtained；
Handle determination unit, for according to the multiple Question Classification as a result, determine application development the corresponding period
The interior processing information based on the application program.
9. a kind of storage medium, which is characterized in that the storage medium stores a plurality of instruction, and described instruction is suitable for being added by processor
It carries and executes log analysis method as described in any one of claim 1 to 7.
10. a kind of server, which is characterized in that including pocessor and storage media, the processor, for realizing each finger
It enables；
The storage medium is for storing a plurality of instruction, and described instruction by processor for being loaded and executing such as claim 1 to 7
Described in any item log analysis methods.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910509655.4A CN110262942A (en) | 2019-06-13 | 2019-06-13 | A kind of log analysis method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910509655.4A CN110262942A (en) | 2019-06-13 | 2019-06-13 | A kind of log analysis method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110262942A true CN110262942A (en) | 2019-09-20 |
Family
ID=67917965
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910509655.4A Pending CN110262942A (en) | 2019-06-13 | 2019-06-13 | A kind of log analysis method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110262942A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111198817A (en) * | 2019-12-30 | 2020-05-26 | 武汉大学 | A SaaS software fault diagnosis method and device based on convolutional neural network |
| CN111414343A (en) * | 2020-02-24 | 2020-07-14 | 北京云途腾科技有限责任公司 | Log writing method and device, electronic equipment and medium |
| CN111680016A (en) * | 2020-05-28 | 2020-09-18 | 中国人民银行清算总中心 | Distributed server cluster log data processing method, device and system |
| CN111782621A (en) * | 2020-06-30 | 2020-10-16 | 中国民航信息网络股份有限公司 | Service application log processing method and device |
| CN112364163A (en) * | 2020-11-10 | 2021-02-12 | 平安普惠企业管理有限公司 | Log caching method and device and computer equipment |
| CN112749079A (en) * | 2019-10-31 | 2021-05-04 | 中国移动通信集团浙江有限公司 | Defect classification method and device for software test and computing equipment |
| CN113515625A (en) * | 2021-05-18 | 2021-10-19 | 中国工商银行股份有限公司 | Test result classification model training method, classification method and device |
| CN118227189A (en) * | 2024-05-21 | 2024-06-21 | 阿里云计算有限公司 | Data processing method and abnormality prompting method |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109474483A (en) * | 2019-01-08 | 2019-03-15 | Oppo广东移动通信有限公司 | A kind of detection method, detection device and the terminal device of unit exception situation |
-
2019
- 2019-06-13 CN CN201910509655.4A patent/CN110262942A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109474483A (en) * | 2019-01-08 | 2019-03-15 | Oppo广东移动通信有限公司 | A kind of detection method, detection device and the terminal device of unit exception situation |
Non-Patent Citations (1)
| Title |
|---|
| 张光宇: "基于挖掘稀有类型日志的系统故障诊断方法", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 12, 15 December 2018 (2018-12-15), pages 140 - 949 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112749079B (en) * | 2019-10-31 | 2023-12-26 | 中国移动通信集团浙江有限公司 | Defect classification method and device for software test and computing equipment |
| CN112749079A (en) * | 2019-10-31 | 2021-05-04 | 中国移动通信集团浙江有限公司 | Defect classification method and device for software test and computing equipment |
| CN111198817A (en) * | 2019-12-30 | 2020-05-26 | 武汉大学 | A SaaS software fault diagnosis method and device based on convolutional neural network |
| CN111198817B (en) * | 2019-12-30 | 2021-06-04 | 武汉大学 | A SaaS software fault diagnosis method and device based on convolutional neural network |
| CN111414343B (en) * | 2020-02-24 | 2023-08-04 | 北京百度云途腾科技有限责任公司 | Log writing method, device, electronic equipment and medium |
| CN111414343A (en) * | 2020-02-24 | 2020-07-14 | 北京云途腾科技有限责任公司 | Log writing method and device, electronic equipment and medium |
| CN111680016A (en) * | 2020-05-28 | 2020-09-18 | 中国人民银行清算总中心 | Distributed server cluster log data processing method, device and system |
| CN111782621A (en) * | 2020-06-30 | 2020-10-16 | 中国民航信息网络股份有限公司 | Service application log processing method and device |
| CN111782621B (en) * | 2020-06-30 | 2023-12-22 | 中国民航信息网络股份有限公司 | Business application log processing method and device |
| CN112364163A (en) * | 2020-11-10 | 2021-02-12 | 平安普惠企业管理有限公司 | Log caching method and device and computer equipment |
| CN112364163B (en) * | 2020-11-10 | 2024-06-14 | 山东浪潮创新创业科技有限公司 | Log caching method and device and computer equipment |
| CN113515625A (en) * | 2021-05-18 | 2021-10-19 | 中国工商银行股份有限公司 | Test result classification model training method, classification method and device |
| CN113515625B (en) * | 2021-05-18 | 2024-12-24 | 中国工商银行股份有限公司 | Test result classification model training method, classification method and device |
| CN118227189A (en) * | 2024-05-21 | 2024-06-21 | 阿里云计算有限公司 | Data processing method and abnormality prompting method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110262942A (en) | A kind of log analysis method and device | |
| CN110688288B (en) | Automatic test method, device, equipment and storage medium based on artificial intelligence | |
| CN110609524A (en) | An industrial equipment residual life prediction model and its construction method and application | |
| CN107045503A (en) | The method and device that a kind of feature set is determined | |
| CN110162628A (en) | A kind of content identification method and device | |
| CN111931809A (en) | Data processing method and device, storage medium and electronic equipment | |
| CN113706151A (en) | Data processing method and device, computer equipment and storage medium | |
| CN116127899A (en) | Chip design system, method, electronic device, and storage medium | |
| RU2730449C2 (en) | Method of creating model for analysing dialogues based on artificial intelligence for processing user requests and system using such model | |
| CN107392311A (en) | The method and apparatus of sequence cutting | |
| CN110070131A (en) | A kind of Active Learning Method of data-oriented driving modeling | |
| CN112685374A (en) | Log classification method and device and electronic equipment | |
| CN112363465B (en) | Expert rule set training method, trainer and industrial equipment early warning system | |
| CN116911288B (en) | Discrete text recognition method based on natural language processing technology | |
| CN113824580A (en) | Network index early warning method and system | |
| CN111352820A (en) | Method, equipment and device for predicting and monitoring running state of high-performance application | |
| CN113779864B (en) | Method and device for constructing running design area for automatic driving automobile | |
| CN117456242A (en) | Deep learning optimization control method for coprocessor | |
| Dabbous et al. | Bringing Intelligence to the Edge for Structural Health Monitoring. The Case Study of the Z24 Bridge | |
| CN113688989B (en) | Deep learning network acceleration method, device, equipment and storage medium | |
| CN113807462A (en) | AI-based network equipment fault reason positioning method and system | |
| CN116527411B (en) | Data security intelligent protection model construction method and device and collaboration platform | |
| CN113807898A (en) | Prediction method, device, equipment and storage medium for agent issuing probability | |
| Bulín et al. | Optimization of neural network | |
| CN119048121B (en) | Power grid violation supervision method, terminal equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |