CN114327983A - Log-based fault determination method, device, equipment and medium - Google Patents
Log-based fault determination method, device, equipment and medium Download PDFInfo
- Publication number
- CN114327983A CN114327983A CN202111629664.0A CN202111629664A CN114327983A CN 114327983 A CN114327983 A CN 114327983A CN 202111629664 A CN202111629664 A CN 202111629664A CN 114327983 A CN114327983 A CN 114327983A
- Authority
- CN
- China
- Prior art keywords
- fault
- log
- target
- reason
- keywords
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000012549 training Methods 0.000 claims abstract description 64
- 238000004458 analytical method Methods 0.000 claims abstract description 63
- 230000002776 aggregation Effects 0.000 claims abstract description 19
- 238000004220 aggregation Methods 0.000 claims abstract description 19
- 238000012216 screening Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 13
- 230000004931 aggregating effect Effects 0.000 claims 1
- 238000012545 processing Methods 0.000 description 8
- 230000009471 action Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011158 quantitative evaluation Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application discloses a fault determination method, a fault determination device, equipment and a computer readable storage medium based on logs, and historical logs are obtained; and according to the set corresponding relation between the fault problems and the fault reasons, performing aggregation grouping on the historical logs to obtain a training data set. When the training data set is used for model training to obtain the fault analysis model, the weight corresponding to each fault reason can be added, so that the fault analysis model can comprise fault keywords corresponding to each fault problem and the weight corresponding to each fault keyword. And analyzing the newly acquired log according to the fault analysis model, wherein the fault reason meeting the weight requirement can be used as the fault reason corresponding to the log. By performing aggregation grouping on the historical logs, information with relevance in the historical logs can be fully mined, so that a training data set is obtained. And the accurate analysis of the fault can be realized by using the trained fault analysis model.
Description
Technical Field
The present application relates to the field of server technologies, and in particular, to a log-based fault determination method, apparatus, device, and computer-readable storage medium.
Background
The application of the server is more and more extensive in the information society nowadays, and as the number of the used servers is increased, the management of the servers is increasingly complex. The role of BMC (Baseboard Management Controller) as an important tool for out-of-band Management of servers is also increasingly prominent. The BMC controller can acquire various monitoring data on the server in cooperation with BMC firmware, and gives alarm information when necessary. The BMC monitors the running state of the server, records the change of the state of the server in real time, and generates various logs to record the running state and the health state of the server.
During the operation of the server, a large amount of logs are generated. Through simple rule matching, simple problems can be positioned for operation and maintenance personnel to handle relevant problems. However, since the amount of information matched by the rule is small, only relatively rough problem positioning can be realized, and in many cases, a feasible operation cannot be formed, and a log needs to be further analyzed manually to form a feasible operation, so as to perform next positioning and processing on the fault.
It can be seen that how to implement accurate analysis of the fault is a problem that needs to be solved by those skilled in the art.
Disclosure of Invention
An object of the embodiments of the present application is to provide a log-based fault determination method, apparatus, device, and computer-readable storage medium, which can implement accurate analysis of a fault.
In order to solve the foregoing technical problem, an embodiment of the present application provides a log-based fault determining method, including:
acquiring a history log;
performing aggregation grouping on the historical logs according to the set corresponding relationship between the fault problems and the fault reasons to obtain a training data set;
performing model training by using the training data set to obtain a fault analysis model; the fault analysis model comprises fault keywords corresponding to each fault problem and weights corresponding to the fault keywords;
and analyzing the newly acquired log according to the fault analysis model so as to take the fault reason meeting the weight requirement as the fault reason corresponding to the log.
Optionally, the performing aggregation grouping on the historical logs according to the set correspondence between the failure problem and the failure cause to obtain a training data set includes:
screening a target historical log matched with the target fault reason from the historical logs according to the target fault reason corresponding to the target fault problem; wherein the target failure problem is any one of all the failure problems;
acquiring target fault keywords from the target historical log according to the keyword types corresponding to the target fault reasons;
setting weight for the target fault keywords corresponding to each target fault reason based on the importance levels of the target fault keywords;
and summarizing the fault keywords with weights corresponding to fault reasons under the fault problems to serve as a training data set.
Optionally, the setting of the weight for the target fault keyword corresponding to each target fault cause based on the importance level of the target fault keyword includes:
and setting weight for the target fault keywords corresponding to each target fault reason based on the frequency of the target fault keywords appearing in the target historical log.
Optionally, the setting of the weight for the target fault keyword corresponding to each target fault cause based on the importance level of the target fault keyword includes:
setting weight for the target fault keywords corresponding to the target fault reasons according to the initial weight corresponding to each keyword type;
and adjusting the weight of the target fault keyword corresponding to each target fault reason according to the frequency of the target fault keyword appearing in the target historical log.
Optionally, the screening, according to a target fault cause corresponding to a target fault problem, a target history log matched with the target fault cause from the history logs includes:
determining log information corresponding to the target fault reason from the historical log;
and taking the log information and the context log information adjacent to the log information as a target history log matched with the target fault reason.
Optionally, the analyzing the newly acquired log according to the fault analysis model to use the fault cause meeting the weight requirement as the fault cause corresponding to the log includes:
inputting the newly acquired log into the fault analysis model to obtain a first fault keyword matched with the newly acquired log and a weight corresponding to the first fault keyword;
determining weights and values corresponding to all the first fault keywords contained in the same fault reason according to the weights corresponding to the first fault keywords;
and taking the fault reason with the highest weight and value as the fault reason corresponding to the log.
Optionally, after determining weights and values corresponding to all the first fault keywords included in the same fault cause according to the weights corresponding to the first fault keywords, the method further includes:
taking the fault reason corresponding to each first fault keyword as the fault reason to be displayed;
according to the weight and the value corresponding to each fault reason to be displayed, performing descending order arrangement on each fault reason to be displayed;
and outputting each fault reason to be displayed in descending order.
The embodiment of the application also provides a log-based fault determination device, which comprises an acquisition unit, a grouping unit, a training unit and an analysis unit;
the acquisition unit is used for acquiring a history log;
the grouping unit is used for performing aggregation grouping on the historical logs according to the set corresponding relation between the fault problems and the fault reasons so as to obtain a training data set;
the training unit is used for carrying out model training by utilizing the training data set to obtain a fault analysis model; the fault analysis model comprises fault keywords corresponding to each fault problem and weights corresponding to the fault keywords;
and the analysis unit is used for analyzing the newly acquired log according to the fault analysis model so as to take the fault reason meeting the weight requirement as the fault reason corresponding to the log.
Optionally, the grouping unit includes a screening subunit, an obtaining subunit, a setting subunit, and a serving subunit;
the screening subunit is configured to screen, according to a target fault cause corresponding to a target fault problem, a target history log matched with the target fault cause from the history logs; wherein the target failure problem is any one of all the failure problems;
the obtaining subunit is configured to obtain a target fault keyword from the target history log according to a keyword type corresponding to each target fault cause;
the setting subunit is configured to set a weight for the target fault keyword corresponding to each target fault cause based on the importance level of the target fault keyword;
and the sub-unit is used for summarizing the fault keywords with the weights corresponding to the fault reasons under the fault problems to be used as a training data set.
Optionally, the setting subunit is configured to set a weight to the target failure keyword corresponding to each target failure cause based on the number of times that the target failure keyword appears in the target history log.
Optionally, the setting subunit is configured to set a weight for a target fault keyword corresponding to each target fault cause according to an initial weight corresponding to each keyword type; and adjusting the weight of the target fault keyword corresponding to each target fault reason according to the frequency of the target fault keyword appearing in the target historical log.
Optionally, the screening subunit is configured to determine log information corresponding to the target fault cause from the historical log; and taking the log information and the context log information adjacent to the log information as a target history log matched with the target fault reason.
Optionally, the acting unit is configured to input a newly acquired log into the fault analysis model to obtain a first fault keyword matched with the newly acquired log and a weight corresponding to the first fault keyword; determining weights and values corresponding to all the first fault keywords contained in the same fault reason according to the weights corresponding to the first fault keywords; and taking the fault reason with the highest weight and value as the fault reason corresponding to the log.
Optionally, the system further comprises a sorting unit and an output unit;
the sorting unit is used for taking the fault reason corresponding to each first fault keyword as the fault reason to be displayed; according to the weight and the value corresponding to each fault reason to be displayed, performing descending order arrangement on each fault reason to be displayed;
and the output unit is used for outputting the fault reasons to be displayed in descending order.
An embodiment of the present application further provides a log-based fault determination device, including:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the log-based failure determination method as described above.
Embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the log-based fault determination method as described above.
According to the technical scheme, the historical log is obtained; and according to the set corresponding relation between the fault problems and the fault reasons, performing aggregation grouping on the historical logs to obtain a training data set. All fault reasons related to the same fault problem are contained in the training data set, and aggregation of logs with relevance in the historical logs is achieved. The fault causes causing the fault problems are various, and the influence degree of each fault cause on the fault problems is different, so that when a fault analysis model is obtained by utilizing a training data set to perform model training, the weight corresponding to each fault cause can be added, and the fault analysis model can comprise fault keywords corresponding to each fault problem and the weights corresponding to the fault keywords. And analyzing the newly acquired log according to the fault analysis model, wherein the fault reason meeting the weight requirement can be used as the fault reason corresponding to the log. In the technical scheme, through carrying out aggregation grouping on the historical logs, information with relevance in the historical logs can be fully mined, so that a training data set is obtained. The influence degree of different fault reasons on the fault problem can be fully considered by setting the weight corresponding to each fault keyword in the fault analysis model, so that the fault can be accurately analyzed by using the trained fault analysis model.
Drawings
In order to more clearly illustrate the embodiments of the present application, the drawings needed for the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.
Fig. 1 is a flowchart of a log-based fault determination method according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a method for acquiring a training data set according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a log-based fault determination apparatus according to an embodiment of the present application;
fig. 4 is a block diagram of a log-based failure determination device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without any creative effort belong to the protection scope of the present application.
The terms "including" and "having," and any variations thereof, in the description and claims of this application and the drawings described above, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements but may include other steps or elements not expressly listed.
In order that those skilled in the art will better understand the disclosure, the following detailed description will be given with reference to the accompanying drawings.
Next, a method for determining a fault based on a log according to an embodiment of the present application will be described in detail. Fig. 1 is a flowchart of a log-based fault determination method provided in an embodiment of the present application, where the method includes:
s101: and acquiring a history log.
In the embodiment of the application, in order to realize accurate analysis of the fault, the information contained in the historical log can be analyzed in a training model mode, so that a fault analysis model is obtained, the newly obtained log is analyzed by using the fault analysis model, and the fault can be analyzed more quickly and accurately.
In order to better fit the actual operation condition of the server, the acquired history log may be a log generated by the server within a recent period of time, for example, a log generated in a recent month.
S102: and according to the set corresponding relation between the fault problems and the fault reasons, performing aggregation grouping on the historical logs to obtain a training data set.
In consideration of practical applications, the logs of the server system are often recorded according to the classification of each module included in the server system, for example, the logs corresponding to the fan modules are recorded separately, and the logs corresponding to the fan modules are recorded separately for the CPU.
There may be various failure causes causing a failure problem, and logs corresponding to the failure causes may be recorded in different logs in a scattered manner, so as to fully mine the correlation between different logs, thereby determining the cause causing the failure problem more comprehensively. In the embodiment of the present application, the correspondence between the failure problem and the failure cause may be set in advance based on experience of a manager.
All relevant fault reasons of a fault problem can be recorded in the corresponding relation between the fault problem and the fault reason, for example, the fault problem is that the system temperature is too high, and the fault problem is mainly caused by two reasons, one reason is that the rotating speed of the fan does not meet the requirement, and the other reason is that the software for executing speed regulation in the CPU has problems.
By setting the corresponding relationship between the fault problems and the fault causes, all the log numbers of the fault causes causing the fault problems can be searched from the historical logs aiming at each fault problem, and the log numbers are aggregated to obtain a group of log information corresponding to the fault problems. By analogy, log information corresponding to all fault problems can be obtained, and the aggregation grouping of the historical logs is realized. In order to facilitate subsequent model training, each fault problem and the corresponding log information thereof can be used as a training data set.
S103: and carrying out model training by using the training data set to obtain a fault analysis model.
Considering that there are a plurality of fault causes causing the fault problem, and the influence degree of each fault cause on the fault problem is different, therefore, when the fault analysis model is obtained by performing model training using the training data set, the weight corresponding to each fault cause can be added, so that the fault analysis model can include the fault keyword corresponding to each fault problem and the weight corresponding to each fault keyword.
S104: and analyzing the newly acquired log according to the fault analysis model so as to take the fault reason meeting the weight requirement as the fault reason corresponding to the log.
And inputting the newly acquired log into a fault analysis model to obtain fault problems existing in the log and fault keywords related to the fault problems, wherein each fault keyword has a corresponding weight. Therefore, the weights of all the fault keywords under each fault cause can be added to obtain a weight sum value corresponding to each fault cause, which may be referred to as a weight value for short.
Satisfying the weight requirement may be selecting the fault cause with the highest weight. According to the weight corresponding to each fault reason, the fault reason with the highest weight can be used as the most main factor causing the fault problem, namely the fault reason with the highest weight is used as the fault reason corresponding to the log.
In a specific implementation, the newly acquired log can be input into a fault analysis model to obtain a first fault keyword matched with the newly acquired log and a weight corresponding to the first fault keyword; and determining the weights and values corresponding to all the first fault keywords contained in the same fault reason according to the weights corresponding to all the first fault keywords.
There may be a plurality of fault keywords matched with the log, and for convenience of distinguishing, the fault keyword matched with the newly acquired log may be referred to as a first fault keyword. Each first failure keyword has a corresponding weight, the weights corresponding to all the first failure keywords included in the same failure cause are added to obtain a weight sum value corresponding to each failure cause, and the failure cause with the highest weight sum value can be used as the failure cause corresponding to the log.
In the embodiment of the application, in order to facilitate managers to more intuitively and comprehensively know the reason causing the fault problem, the fault reason corresponding to each first fault keyword can be used as the fault reason to be displayed; according to the weight and the value corresponding to each fault reason to be displayed, performing descending order arrangement on each fault reason to be displayed; and outputting the fault reasons to be displayed in descending order.
According to the technical scheme, the historical log is obtained; and according to the set corresponding relation between the fault problems and the fault reasons, performing aggregation grouping on the historical logs to obtain a training data set. All fault reasons related to the same fault problem are contained in the training data set, and aggregation of logs with relevance in the historical logs is achieved. The fault causes causing the fault problems are various, and the influence degree of each fault cause on the fault problems is different, so that when a fault analysis model is obtained by utilizing a training data set to perform model training, the weight corresponding to each fault cause can be added, and the fault analysis model can comprise fault keywords corresponding to each fault problem and the weights corresponding to the fault keywords. And analyzing the newly acquired log according to the fault analysis model, wherein the fault reason meeting the weight requirement can be used as the fault reason corresponding to the log. In the technical scheme, through carrying out aggregation grouping on the historical logs, information with relevance in the historical logs can be fully mined, so that a training data set is obtained. The influence degree of different fault reasons on the fault problem can be fully considered by setting the weight corresponding to each fault keyword in the fault analysis model, so that the fault can be accurately analyzed by using the trained fault analysis model.
In the embodiment of the application, the trained fault analysis model is the basis for executing the fault problem corresponding to the subsequent determination log, and the training data set is a key factor influencing the analysis accuracy of the fault analysis model. The acquisition of the training data set will be described next. Fig. 2 is a flowchart of a method for acquiring a training data set according to an embodiment of the present application, where the method includes:
s201: and screening a target historical log matched with the target fault reason from the historical logs according to the target fault reason corresponding to the target fault problem.
For each fault problem, the process of screening log information related to the fault problem from the history log is the same, and for convenience of introduction, a single fault problem is taken as an example for explanation. For ease of description, the failure problem may be referred to as a target failure problem. The target failure problem may be any one of all failure problems. The failure cause corresponding to the target failure issue may be referred to as a target failure cause, and the log information related to the target failure issue may be referred to as a target history log.
In consideration of the fact that in practical application, context log information related to log information where fault causes are located often has a high reference value, when historical logs corresponding to the fault causes are screened, log information corresponding to a target fault cause can be determined from the historical logs; and then taking the log information and the context log information adjacent to the log information as a target history log matched with the target fault reason.
S202: and acquiring target fault keywords from the target historical log according to the keyword types corresponding to the target fault reasons.
Considering that the target history log often contains some information without analysis value, the accuracy of model training is improved in order to reduce the data volume of model training. After the target history log corresponding to the target fault problem is obtained, further screening may be performed on information included in the target history log.
The target fault problem often has a plurality of corresponding target fault reasons, a corresponding keyword type can be set for each target fault reason, and target keywords can be screened from the target historical log based on the keyword type.
The target keyword may include an instruction executed by the module, a result generated after the instruction is executed, and the like.
Taking the server system temperature as an example, the fan speed is an important factor influencing the server system temperature, and the log corresponding to the fan includes the speed regulation action of the fan, the theoretical speed before the speed regulation action is executed, and the actual speed reached after the speed regulation action is executed.
S203: and setting weight for the target fault keywords corresponding to the target fault reasons based on the importance levels of the target fault keywords.
There may be a plurality of fault causes corresponding to one fault problem, there may be a plurality of keywords included in each fault cause, and the influence degree of the actions corresponding to different keywords on the system is different, so that different importance levels may be set for different keywords.
In order to achieve quantitative evaluation of the keywords, different importance levels may correspond to different weights, for example, the importance levels may be divided into one level, two levels, three levels, and four levels from high to low, and the corresponding weights may be 80%, 60%, 40%, and 20% in sequence. The above is only an example, and in practical applications, the division of the importance level and the corresponding weight can be flexibly set.
In the embodiment of the application, the importance level corresponding to each type of keyword can be preset, and the importance level of the keyword can be evaluated according to the occurrence frequency of the keyword in a history log, so that the weight is set.
In a specific implementation, weights may be set for target fault keywords corresponding to respective target fault causes based on the number of times that the target fault keywords appear in the target history log.
For example, assuming that the keyword a appears 20 times in the target history log, the keyword b appears 10 times in the target history log, and the keyword c appears 1 time in the target history log, the importance level of the keyword a may be divided into one level, the importance level of the keyword b may be divided into two levels, and the importance level of the keyword c may be divided into four levels. The weights corresponding to the keywords a, b, and c may be 80%, 60%, and 20% in this order.
In order to ensure that the weight corresponding to the keyword is set within a reasonable range, in the embodiment of the application, the weight can be set for the target fault keyword corresponding to each target fault reason according to the initial weight corresponding to each keyword type; and adjusting the weight of the target fault keyword corresponding to each target fault reason according to the frequency of the target fault keyword appearing in the target historical log.
S204: and summarizing the fault keywords with weights corresponding to fault reasons under fault problems to serve as a training data set.
According to the operations of S201 to S203, the target failure keyword corresponding to the target failure problem may be obtained, and the weight corresponding to each target failure keyword may be determined. By analogy, the fault keywords corresponding to all fault problems and the weight corresponding to each fault keyword can be obtained. In the embodiment of the application, the fault keywords corresponding to the fault reasons under the fault problems can be collected to be used as the training data set.
In the embodiment of the application, the fault keyword can be obtained from the historical log by setting the keyword type corresponding to the fault reason, so that log information without analysis value in the historical log is effectively screened out, and a high-quality training data set is obtained. And corresponding weights can be set for different keywords, so that quantitative evaluation can be performed for different fault reasons when the model is used for analysis in the follow-up process.
Fig. 3 is a schematic structural diagram of a log-based fault determination apparatus according to an embodiment of the present application, including an obtaining unit 31, a grouping unit 32, a training unit 33, and an analyzing unit 34;
an acquisition unit 31 for acquiring a history log;
the grouping unit 32 is configured to perform aggregation grouping on the historical logs according to the set correspondence between the failure problem and the failure cause to obtain a training data set;
a training unit 33, configured to perform model training using the training data set to obtain a fault analysis model; the fault analysis model comprises fault keywords corresponding to each fault problem and weights corresponding to the fault keywords;
and the analysis unit 34 is used for analyzing the newly acquired log according to the fault analysis model, so that the fault reason meeting the weight requirement is taken as the fault reason corresponding to the log.
Optionally, the grouping unit includes a screening subunit, an obtaining subunit, a setting subunit, and a serving subunit;
the screening subunit is used for screening a target historical log matched with the target fault reason from the historical logs according to the target fault reason corresponding to the target fault problem; wherein, the target failure problem is any one of all failure problems;
the acquisition subunit is used for acquiring target fault keywords from the target historical logs according to the keyword types corresponding to the target fault reasons;
the setting subunit is used for setting weight for the target fault keywords corresponding to each target fault reason based on the importance levels of the target fault keywords;
and the sub-unit is used for summarizing the fault keywords with the weights corresponding to the fault reasons under the fault problems to be used as a training data set.
Optionally, the setting subunit is configured to set a weight for the target fault keyword corresponding to each target fault reason based on the number of times that the target fault keyword appears in the target history log.
Optionally, the setting subunit is configured to set a weight for the target failure keyword corresponding to each target failure cause according to the initial weight corresponding to each keyword type; and adjusting the weight of the target fault keyword corresponding to each target fault reason according to the frequency of the target fault keyword appearing in the target historical log.
Optionally, the screening subunit is configured to determine log information corresponding to the target failure cause from the historical log; and the target history log takes the log information and the context log information adjacent to the log information as the target failure reason.
Optionally, the unit is configured to input the newly acquired log into the fault analysis model to obtain a first fault keyword matched with the newly acquired log and a weight corresponding to the first fault keyword; determining weights and values corresponding to all first fault keywords contained in the same fault reason according to the weights corresponding to all the first fault keywords; and taking the fault reason with the highest weight and value as the fault reason corresponding to the log.
Optionally, the system further comprises a sorting unit and an output unit;
the sorting unit is used for taking the fault reason corresponding to each first fault keyword as the fault reason to be displayed; according to the weight and the value corresponding to each fault reason to be displayed, performing descending order arrangement on each fault reason to be displayed;
and the output unit is used for outputting the fault reasons to be displayed which are arranged in a descending order.
For the description of the features in the embodiment corresponding to fig. 3, reference may be made to the related description of the embodiments corresponding to fig. 1 and fig. 2, which is not repeated here.
According to the technical scheme, the historical log is obtained; and according to the set corresponding relation between the fault problems and the fault reasons, performing aggregation grouping on the historical logs to obtain a training data set. All fault reasons related to the same fault problem are contained in the training data set, and aggregation of logs with relevance in the historical logs is achieved. The fault causes causing the fault problems are various, and the influence degree of each fault cause on the fault problems is different, so that when a fault analysis model is obtained by utilizing a training data set to perform model training, the weight corresponding to each fault cause can be added, and the fault analysis model can comprise fault keywords corresponding to each fault problem and the weights corresponding to the fault keywords. And analyzing the newly acquired log according to the fault analysis model, wherein the fault reason meeting the weight requirement can be used as the fault reason corresponding to the log. In the technical scheme, through carrying out aggregation grouping on the historical logs, information with relevance in the historical logs can be fully mined, so that a training data set is obtained. The influence degree of different fault reasons on the fault problem can be fully considered by setting the weight corresponding to each fault keyword in the fault analysis model, so that the fault can be accurately analyzed by using the trained fault analysis model.
Fig. 4 is a structural diagram of a log-based failure determination device according to an embodiment of the present application, and as shown in fig. 4, the log-based failure determination device includes: a memory 20 for storing a computer program;
a processor 21 for implementing the steps of the log-based failure determination method as described in the above embodiments when executing the computer program.
The log-based fault determination device provided by the present embodiment may include, but is not limited to, a smart phone, a tablet computer, a notebook computer, or a desktop computer.
The processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 21 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 21 may further include an AI (Artificial Intelligence) processor for processing a calculation operation related to machine learning.
The memory 20 may include one or more computer-readable storage media, which may be non-transitory. Memory 20 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used for storing a computer program 201, wherein after being loaded and executed by the processor 21, the computer program can implement the relevant steps of the log-based fault determination method disclosed in any one of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202, data 203, and the like, and the storage manner may be a transient storage manner or a permanent storage manner. Operating system 202 may include, among others, Windows, Unix, Linux, and the like. The data 203 may include, but is not limited to, historical logs, correspondence between failure problems and failure causes, training data sets, failure keywords corresponding to each failure problem, weights corresponding to the failure keywords, and the like.
In some embodiments, the log-based fault determination device may also include a display screen 22, an input-output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.
Those skilled in the art will appreciate that the configuration shown in FIG. 4 does not constitute a limitation of log-based fault determination devices and may include more or fewer components than those shown.
It is to be understood that, if the log-based failure determination method in the above-described embodiment is implemented in the form of a software functional unit and sold or used as a separate product, it may be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application may be substantially or partially implemented in the form of a software product, which is stored in a storage medium and executes all or part of the steps of the methods of the embodiments of the present application, or all or part of the technical solutions. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), an electrically erasable programmable ROM, a register, a hard disk, a removable magnetic disk, a CD-ROM, a magnetic or optical disk, and other various media capable of storing program codes.
Based on this, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the log-based fault determination method as described above.
The functions of the functional modules of the computer-readable storage medium according to the embodiment of the present invention may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the related description of the foregoing method embodiment, which is not described herein again.
A method, an apparatus, a device, and a computer-readable storage medium for log-based failure determination provided by the embodiments of the present application are described in detail above. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
A method, an apparatus, a device and a computer readable storage medium for log-based fault determination provided by the present application are described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present application.
Claims (10)
1. A log-based failure determination method, comprising:
acquiring a history log;
performing aggregation grouping on the historical logs according to the set corresponding relationship between the fault problems and the fault reasons to obtain a training data set;
performing model training by using the training data set to obtain a fault analysis model; the fault analysis model comprises fault keywords corresponding to each fault problem and weights corresponding to the fault keywords;
and analyzing the newly acquired log according to the fault analysis model so as to take the fault reason meeting the weight requirement as the fault reason corresponding to the log.
2. The log-based fault determination method of claim 1, wherein the aggregating and grouping the historical logs according to the set correspondence between fault problems and fault causes to obtain a training data set comprises:
screening a target historical log matched with the target fault reason from the historical logs according to the target fault reason corresponding to the target fault problem; wherein the target failure problem is any one of all the failure problems;
acquiring target fault keywords from the target historical log according to the keyword types corresponding to the target fault reasons;
setting weight for the target fault keywords corresponding to each target fault reason based on the importance levels of the target fault keywords;
and summarizing the fault keywords with weights corresponding to fault reasons under the fault problems to serve as a training data set.
3. The log-based fault determination method of claim 2, wherein the setting of the weight for the target fault keyword corresponding to each target fault cause based on the importance level of the target fault keyword comprises:
and setting weight for the target fault keywords corresponding to each target fault reason based on the frequency of the target fault keywords appearing in the target historical log.
4. The log-based fault determination method of claim 2, wherein the setting of the weight for the target fault keyword corresponding to each target fault cause based on the importance level of the target fault keyword comprises:
setting weight for the target fault keywords corresponding to the target fault reasons according to the initial weight corresponding to each keyword type;
and adjusting the weight of the target fault keyword corresponding to each target fault reason according to the frequency of the target fault keyword appearing in the target historical log.
5. The log-based fault determination method of claim 2, wherein the screening out the target history log matched with the target fault cause from the history logs according to the target fault cause corresponding to the target fault problem comprises:
determining log information corresponding to the target fault reason from the historical log;
and taking the log information and the context log information adjacent to the log information as a target history log matched with the target fault reason.
6. The log-based fault determination method according to any one of claims 1 to 5, wherein the analyzing the newly acquired log according to the fault analysis model to use the fault cause meeting the weight requirement as the fault cause corresponding to the log comprises:
inputting the newly acquired log into the fault analysis model to obtain a first fault keyword matched with the newly acquired log and a weight corresponding to the first fault keyword;
determining weights and values corresponding to all the first fault keywords contained in the same fault reason according to the weights corresponding to the first fault keywords;
and taking the fault reason with the highest weight and value as the fault reason corresponding to the log.
7. The log-based fault determination method according to claim 6, wherein after determining the weights and values corresponding to all the first fault keywords included in the same fault cause according to the weights corresponding to the first fault keywords, the method further comprises:
taking the fault reason corresponding to each first fault keyword as the fault reason to be displayed;
according to the weight and the value corresponding to each fault reason to be displayed, performing descending order arrangement on each fault reason to be displayed;
and outputting each fault reason to be displayed in descending order.
8. A log-based fault determination device is characterized by comprising an acquisition unit, a grouping unit, a training unit and an analysis unit;
the acquisition unit is used for acquiring a history log;
the grouping unit is used for performing aggregation grouping on the historical logs according to the set corresponding relation between the fault problems and the fault reasons so as to obtain a training data set;
the training unit is used for carrying out model training by utilizing the training data set to obtain a fault analysis model; the fault analysis model comprises fault keywords corresponding to each fault problem and weights corresponding to the fault keywords;
and the analysis unit is used for analyzing the newly acquired log according to the fault analysis model so as to take the fault reason meeting the weight requirement as the fault reason corresponding to the log.
9. A log-based failure determination device, comprising:
a memory for storing a computer program;
a processor for executing the computer program for carrying out the steps of the log-based failure determination method according to any of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the log-based failure determination method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111629664.0A CN114327983A (en) | 2021-12-28 | 2021-12-28 | Log-based fault determination method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111629664.0A CN114327983A (en) | 2021-12-28 | 2021-12-28 | Log-based fault determination method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114327983A true CN114327983A (en) | 2022-04-12 |
Family
ID=81015275
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111629664.0A Pending CN114327983A (en) | 2021-12-28 | 2021-12-28 | Log-based fault determination method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114327983A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115225460A (en) * | 2022-07-15 | 2022-10-21 | 北京天融信网络安全技术有限公司 | Failure determination method, electronic device, and storage medium |
| CN115426276A (en) * | 2022-08-22 | 2022-12-02 | 神华准格尔能源有限责任公司 | Monitoring method for strip mine 5G major equipment and cloud server |
| CN115766404A (en) * | 2022-10-24 | 2023-03-07 | 浪潮通信信息系统有限公司 | Communication operator network fault management method and system based on intelligent analysis |
| CN115981240A (en) * | 2023-03-22 | 2023-04-18 | 中科航迈数控软件(深圳)有限公司 | Method, device, equipment and medium for determining fault reason of numerical control machine tool |
| CN117057676A (en) * | 2023-10-11 | 2023-11-14 | 深圳润世华软件和信息技术服务有限公司 | Multi-data fusion fault analysis method, equipment and storage medium |
-
2021
- 2021-12-28 CN CN202111629664.0A patent/CN114327983A/en active Pending
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115225460A (en) * | 2022-07-15 | 2022-10-21 | 北京天融信网络安全技术有限公司 | Failure determination method, electronic device, and storage medium |
| CN115225460B (en) * | 2022-07-15 | 2023-11-28 | 北京天融信网络安全技术有限公司 | Fault determination method, electronic device, and storage medium |
| CN115426276A (en) * | 2022-08-22 | 2022-12-02 | 神华准格尔能源有限责任公司 | Monitoring method for strip mine 5G major equipment and cloud server |
| CN115426276B (en) * | 2022-08-22 | 2024-03-12 | 神华准格尔能源有限责任公司 | Method for monitoring 5G major equipment of strip mine and cloud server |
| CN115766404A (en) * | 2022-10-24 | 2023-03-07 | 浪潮通信信息系统有限公司 | Communication operator network fault management method and system based on intelligent analysis |
| CN115981240A (en) * | 2023-03-22 | 2023-04-18 | 中科航迈数控软件(深圳)有限公司 | Method, device, equipment and medium for determining fault reason of numerical control machine tool |
| CN117057676A (en) * | 2023-10-11 | 2023-11-14 | 深圳润世华软件和信息技术服务有限公司 | Multi-data fusion fault analysis method, equipment and storage medium |
| CN117057676B (en) * | 2023-10-11 | 2024-02-23 | 深圳润世华软件和信息技术服务有限公司 | Multi-data fusion fault analysis method, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114327983A (en) | Log-based fault determination method, device, equipment and medium | |
| CN105183625B (en) | A kind of daily record data treating method and apparatus | |
| CN112181758B (en) | Fault root cause positioning method based on network topology and real-time alarm | |
| CN107944721B (en) | Universal machine learning method, device and system based on data mining | |
| CN113010389A (en) | Training method, fault prediction method, related device and equipment | |
| CN110716966A (en) | Data visualization processing method and system, electronic device and storage medium | |
| CN111931809A (en) | Data processing method and device, storage medium and electronic equipment | |
| CN108696486B (en) | Abnormal operation behavior detection processing method and device | |
| WO2018122890A1 (en) | Log analysis method, system, and program | |
| CN113268403A (en) | Time series analysis and prediction method, device, equipment and storage medium | |
| CN115686910A (en) | Fault analysis method and device, electronic equipment and medium | |
| CN115392501A (en) | Data acquisition method and device, electronic equipment and storage medium | |
| CN108009582B (en) | Method for setting standard working index of electronic equipment | |
| CN109240916B (en) | Information output control method, information output control device and computer readable storage medium | |
| CN114047710A (en) | Intelligent household data analysis method and device, terminal equipment and storage medium | |
| CN112598353A (en) | Material substitution method, device, storage medium and equipment | |
| CN116860555A (en) | Software operation monitoring method, device, equipment and computer storage medium | |
| CN113535458B (en) | Abnormal false alarm processing method and device, storage medium and terminal | |
| KR102464688B1 (en) | Method and apparatus for detrmining event level of monitoring result | |
| CN113032209B (en) | Operation monitoring method, device, server and medium | |
| CN115114070A (en) | Fault diagnosis method, device, equipment and medium | |
| CN109284354B (en) | Script searching method and device, computer equipment and storage medium | |
| CN113064807A (en) | Log diagnosis method and device | |
| CN112131090A (en) | Business system performance monitoring method and device, equipment and medium | |
| CN113568769A (en) | Exception handling method, exception handling device, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |