CN114896094A - Fault positioning method and device, electronic equipment and storage medium - Google Patents
Fault positioning method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114896094A CN114896094A CN202210506106.3A CN202210506106A CN114896094A CN 114896094 A CN114896094 A CN 114896094A CN 202210506106 A CN202210506106 A CN 202210506106A CN 114896094 A CN114896094 A CN 114896094A
- Authority
- CN
- China
- Prior art keywords
- target
- fault
- operation log
- link
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/079—Root cause analysis, i.e. error or fault diagnosis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0766—Error or fault reporting or storing
- G06F11/0769—Readable error formats, e.g. cross-platform generic formats, human understandable formats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0766—Error or fault reporting or storing
- G06F11/0787—Storage of error reports, e.g. persistent data storage, storage using memory protection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention provides a fault positioning method, a fault positioning device, electronic equipment and a storage medium, wherein under the condition that a target link fails in the continuous integration process of a target program, an operation log of the target link is obtained and used as a target operation log; processing the target operation log based on a pre-trained fault recognition model to obtain a fault corresponding to the target operation log as a target fault; the identification model is obtained by training sample operation logs based on a sample link with a fault and sample faults corresponding to the sample operation logs; and determining a solution corresponding to the target fault in the preset corresponding relation between the fault and the solution as a target solution. Based on the processing, the target fault and the target solution of the target link can be automatically determined, the operation log does not need to be analyzed manually, the cost for fault positioning can be reduced, and the cost for fault processing is further reduced.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a fault location method and apparatus, an electronic device, and a storage medium.
Background
The continuous integration process refers to the process from development to online of the program. The continuous integration process comprises the steps of constructing a developed source code to obtain an intermediate program, packaging the intermediate program to obtain a target program, releasing the target program to a server to be online and the like.
Various failures may occur in various links involved in the above-described continuous integration process. In the related art, when a fault occurs in the continuous integration process, a user can only locate the cause of the fault and determine a solution in a manual mode through a running log of the continuous integration process. Specifically, the user analyzes the operation log of the failed link to determine the failure cause of the failure. Then, the user determines a corresponding solution based on the analyzed cause of the failure. Furthermore, the fault can be solved according to the determined solution.
In the process, the user needs to have professional software development knowledge, the running log is analyzed manually, and if the accuracy of self judgment of the user is not high, the user can do a lot of useless work. Therefore, the related art needs to consume a large labor cost and a large time cost for fault location, and further the related art needs to have a high cost for fault processing.
Disclosure of Invention
Embodiments of the present invention provide a fault location method and apparatus, an electronic device, and a storage medium, so as to reduce the cost of performing fault location, and further reduce the cost of performing fault processing. The specific technical scheme is as follows:
in a first aspect, to achieve the above object, an embodiment of the present invention discloses a fault location method, where the method includes:
under the condition that a target link in the continuous integration process of a target program fails, acquiring an operation log of the target link as a target operation log;
processing the target operation log based on a pre-trained fault recognition model to obtain a fault corresponding to the target operation log as a target fault; the identification model is obtained by training a sample operation log of a sample link with a fault and a sample fault corresponding to the sample operation log;
and determining a solution corresponding to the target fault in a preset corresponding relation between the fault and the solution as a target solution.
Optionally, the obtaining, as the target operation log, an operation log of the target link when the target link fails in the continuous integration process of the target program includes:
when detecting that a target link in the continuous integration process aiming at a target program fails, displaying an error report message aiming at the target link;
and when a processing instruction aiming at the error report message input by a user is received, determining an operation log corresponding to the link identifier of the target link, and acquiring the operation log corresponding to the program identifier of the target program from the determined operation log to be used as a target operation log.
Optionally, the processing the target operation log based on the pre-trained fault recognition model to obtain a fault corresponding to the target operation log as a target fault includes:
performing word segmentation processing on the log content of the target operation log to obtain an initial word segmentation result;
deleting preset stop words from the initial respective results to obtain target word segmentation results;
and inputting the target word segmentation result into a pre-trained fault recognition model, and obtaining a fault corresponding to the target operation log output by the fault recognition model as a target fault.
Optionally, after determining a solution corresponding to the target fault in the preset correspondence between the fault and the solution, as a target solution, the method further includes:
and displaying the target fault and the target solution to a user so that the user processes the target fault based on the target solution.
Optionally, the training process of the fault identification model includes the following steps:
obtaining the sample operation log and a sample fault corresponding to the sample operation log;
processing the sample operation log based on a fault identification model of an initial structure to obtain a fault corresponding to the sample operation log as a predicted fault;
calculating a loss function value representing a difference between the predicted fault and the sample fault;
and adjusting model parameters of the fault recognition model of the initial structure based on the calculated loss function value until a preset convergence condition is reached, so as to obtain a trained fault recognition model.
In a second aspect, in order to achieve the above object, an embodiment of the present invention discloses a fault location apparatus, including:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring an operation log of a target link as a target operation log under the condition that the target link fails in the continuous integration process of a target program;
the first identification module is used for processing the target operation log based on a pre-trained fault identification model to obtain a fault corresponding to the target operation log as a target fault; the identification model is obtained by training a sample operation log of a sample link with a fault and a sample fault corresponding to the sample operation log;
and the first determining module is used for determining a solution corresponding to the target fault in a preset corresponding relation between the fault and the solution as a target solution.
Optionally, the first obtaining module is specifically configured to display an error report message for a target link when detecting that the target link in a continuous integration process for the target program fails;
and when a processing instruction aiming at the error report message input by a user is received, determining an operation log corresponding to the link identifier of the target link, and acquiring the operation log corresponding to the program identifier of the target program from the determined operation log to be used as a target operation log.
Optionally, the first identification module is specifically configured to perform word segmentation on the log content of the target operation log to obtain an initial word segmentation result;
deleting preset stop words from the initial result to obtain a target word segmentation result;
and inputting the target word segmentation result into a pre-trained fault recognition model, and obtaining a fault corresponding to the target operation log output by the fault recognition model as a target fault.
Optionally, the apparatus further comprises:
and the display module is used for displaying the target fault and the target solution to a user so that the user processes the target fault based on the target solution after the first determination module determines the solution corresponding to the target fault in the preset corresponding relation between the fault and the solution as the target solution.
Optionally, the apparatus further comprises:
the second acquisition module is used for acquiring the sample operation log and a sample fault corresponding to the sample operation log;
the second identification module is used for processing the sample operation log based on the fault identification model of the initial structure to obtain a fault corresponding to the sample operation log as a predicted fault;
a second determining module for calculating a loss function value representing a difference between the predicted fault and the sample fault;
and the training module is used for adjusting the model parameters of the fault recognition model of the initial structure based on the calculated loss function value until a preset convergence condition is reached, so as to obtain the trained fault recognition model.
The embodiment of the invention also provides electronic equipment which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing any one of the steps of the fault positioning method when executing the program stored in the memory.
An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method for locating a fault is implemented.
Embodiments of the present invention further provide a computer program product containing instructions, which when run on a computer, cause the computer to perform any one of the above-mentioned fault location methods.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by referring to these drawings.
Fig. 1 is a flowchart of a fault location method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a persistent integration method according to an embodiment of the present invention;
fig. 3 is a flowchart of another fault location method according to an embodiment of the present invention;
fig. 4 is a flowchart of another fault location method according to an embodiment of the present invention;
FIG. 5 is a flowchart of a method for training a network model according to an embodiment of the present invention;
FIG. 6 is a flow chart of a persistent integration method provided by the related art;
fig. 7 is a flowchart of another fault location method according to an embodiment of the present invention;
fig. 8 is a structural diagram of a fault location device according to an embodiment of the present invention;
fig. 9 is a structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived from the embodiments given herein by one of ordinary skill in the art, are within the scope of the invention.
In the related art, when a fault occurs in a continuous integration process, an operation log of a failed link is manually analyzed to determine a fault reason causing the fault, and a corresponding solution is determined based on the analyzed fault reason. In the process, the user needs to have professional software development knowledge, the running log is analyzed manually, and if the accuracy of self judgment of the user is not high, the user can do a lot of useless work. Therefore, the related art needs to consume a large labor cost and a large time cost for fault location, and further the related art needs to have a high cost for fault processing.
In order to solve the above problem, an embodiment of the present invention provides a fault location method, which is applied to an electronic device, where the electronic device may be a server. Under the condition that a target link fails in the continuous integration process of the target program, the electronic device can process the target operation log of the target link according to the method provided by the embodiment of the invention to obtain a target failure and a target solution corresponding to the target link. The target fault and the target solution may then be provided to the user so that the user processes the target fault according to the target solution.
Based on the processing, the target failure corresponding to the target link can be automatically determined under the condition that the target link in the continuous integration process of the target program fails, and then the target solution corresponding to the target failure can be automatically determined.
Referring to fig. 1, fig. 1 is a flowchart of a fault location method according to an embodiment of the present invention, where the method may include the following steps:
s101: and under the condition that a target link in the continuous integration process of the target program fails, acquiring an operation log of the target link as a target operation log.
S102: and processing the target operation log based on a pre-trained fault recognition model to obtain a fault corresponding to the target operation log as a target fault.
The identification model is obtained by training a sample operation log of a sample link with a fault and a sample fault corresponding to the sample operation log.
S103: and determining a solution corresponding to the target fault in the preset corresponding relation between the fault and the solution as a target solution.
Based on the fault positioning method provided by the embodiment of the invention, the target fault corresponding to the target link can be automatically determined under the condition that the target link fails in the continuous integration process of the target program, and then the target solution corresponding to the target fault can be automatically determined, even if a user does not have professional software development knowledge, the target fault and the target solution of the target link can be determined, and the operation log does not need to be analyzed manually, so that the cost for fault positioning can be reduced, and the cost for fault processing can be further reduced.
For step S101, the target program may be any program in a continuous integration process, and the continuous integration process for the target program refers to a process from development to online of the target program. The continuous integration process for the target program may include a step of constructing a source code corresponding to the target program to obtain an intermediate program (which may be referred to as a construction step), a step of packaging the intermediate program to obtain the target program (which may be referred to as a packaging step), and a step of publishing the target program to a server for online (which may be referred to as a publishing step).
In some embodiments, referring to fig. 2, fig. 2 is a flowchart of a persistent integration method provided by an embodiment of the present invention, where the method may include the following steps:
s201: and acquiring the source code corresponding to the target program from the code database.
S202: and constructing the acquired source code to obtain a program in a target format as an intermediate program, and storing the intermediate program into an Artifact database.
S203: and acquiring an intermediate program corresponding to the target program from the Artifact database.
S204: and packaging the acquired intermediate program to obtain a corresponding Docker mirror image as a target program.
S205: and releasing the target program to a corresponding server, and storing the target program to a Harbor database.
The source code corresponding to the target program is the source code used for generating the target program.
For each program, after a user develops and obtains a source code corresponding to the program, the source code can be uploaded to a code database. The electronic equipment can sequentially acquire the source codes from the code database according to the sequence of the time for uploading the source codes to the code database. The electronic device may construct the acquired source code to obtain a program in a target format (i.e., an intermediate program), and store the intermediate program in an Artifact database.
The Artifact database refers to a database for storing Artifact files, and the Artifact files refer to program packages (for example, a compression package in a war format, or a compression package in a jar format, etc.) obtained by constructing the source code. The program in the target format may be a compression packet in a war format or a compression packet in a jar format, which is not specifically limited in this embodiment.
Then, the electronic device may further sequentially obtain the intermediate programs from the Artifact database according to the sequence of the time when each intermediate program is stored in the Artifact database. The electronic device may package the acquired intermediate program to obtain a Docker image (i.e., an object program).
The Docker is an open-source application container engine, and based on the fact that a Docker developer can package programs and dependency packages into a portable image, the image is published to a corresponding device. The mirror image is a lightweight executable independent package, and includes all source code, libraries, environment variables, configuration files, and the like required for running a certain program.
Accordingly, the target program is an executable package, and the target program can be deployed to the server to provide corresponding services for the user. Further, the electronic device may publish the target program to the corresponding server. The electronic device may also store the object program in a Harbor database (Harbor is a warehouse for constructing enterprise-level private Docker images) for backup.
Each link included in the continuous integration process may have a fault, and when an error occurs in a certain link (i.e., a target link in the embodiment of the present invention), the electronic device may obtain an operation log of the target link (i.e., a target operation log).
For example, if a syntax error exists in the source code of the target program, a fault occurs in the process of constructing the source code, and the construction link is the target link. The electronic device can obtain a target operation log of the construction link.
In some embodiments, on the basis of fig. 1, referring to fig. 3, step S101 may include the steps of:
s1011: and when detecting that a target link in the continuous integration process aiming at the target program fails, displaying an error report message aiming at the target link.
S1012: when a processing instruction aiming at an error report message input by a user is received, determining an operation log corresponding to a link identifier of a target link, and acquiring the operation log corresponding to a program identifier of a target program from the determined operation log as a target operation log.
In the continuous integration process, if a target link for a target program fails, the electronic device can display an error message for the target link for a user to browse. When browsing the error report message displayed by the electronic equipment, the user can determine that the target link fails.
Accordingly, the user can input a processing instruction for the error message. For example, the electronic device is a server, and the electronic device may push an error message for the target link to a terminal used by the user. The terminal can display an error message for the target link. After the user browses the error message, a processing instruction for the error message may be input at the terminal. The terminal may then send processing instructions for the error message to the server.
The processing instruction carries a link identifier of the target link and a program identifier of the target program. When the electronic device receives the processing instruction, the electronic device may determine, at a preset storage location, a log corresponding to the link identifier carried in the processing instruction, and obtain, from the determined operation log, an operation log corresponding to the program identifier carried in the processing instruction, to obtain a target operation log. The running logs of all links in the continuous integration process are stored in the preset storage position.
Based on the processing, when the target link in the continuous integration process of the target program fails, an error can be automatically reported to the user, the user can know the target fault and the target solution corresponding to the target link only by inputting a processing instruction aiming at an error report message, even if the user does not have professional software development knowledge, the target fault and the target solution of the target link can be determined, and the running log does not need to be analyzed manually, so that the cost for fault positioning can be further reduced, and further, the cost for fault processing is reduced.
For step S102, the fault identification model may be a model for classification, for example, the fault identification model may be a CNN (Convolutional Neural Networks) model, or the fault identification model may also be an LSTM (Long Short-Term Memory network) model, which is not specifically limited in this embodiment.
The target failure represents the cause of failure of the target link. For example, the target link is a building link, and the target failure may be: there is a syntax error in the source code of the target program. Or, the target link is a packaging link, and the target fault may be: tool configuration such as container arrangement required by a packaging link has errors.
In some embodiments, on the basis of fig. 1, referring to fig. 4, step S102 may include the steps of:
s1021: and performing word segmentation processing on the log content of the target running log to obtain an initial word segmentation result.
S1022: and deleting preset stop words from the initial word segmentation result to obtain a target word segmentation result.
S1023: and inputting the target word segmentation result into a pre-trained fault recognition model to obtain a fault corresponding to a target operation log output by the fault recognition model, and taking the fault as a target fault.
The preset stop words are functional words without actual meanings, and the preset stop words can include: delimiters (e.g., "a," "the," etc.), prepositions (e.g., "ones," "on," etc.), and conjunctions (e.g., "however," "next," etc.).
After the target operation log is obtained, the electronic device performs word segmentation processing on the log content of the target operation log based on a preset word segmentation algorithm to obtain a corresponding word segmentation result (namely an initial word segmentation result). The preset word segmentation algorithm may be a two-way maximum matching method, or a jieba (jieba) word segmentation algorithm, or an HMM (Hidden Markov Model), and the like, which is not specifically limited in this embodiment.
The preset stop words are functional words without actual meanings, so that the preset stop words cannot represent the operation condition of the target link, and the electronic equipment can delete the preset stop words in the initial word segmentation result to obtain the target word segmentation result. It can be understood that the target word segmentation result obtained after the preset stop word is deleted usually includes words of parts of speech such as nouns and verbs, and such words may represent the operation condition of the target link.
Furthermore, the electronic device may input the target word segmentation result to a pre-trained fault recognition model, so as to obtain a fault (i.e., a target fault) corresponding to a target operation log output by the fault recognition model, where the target fault is a fault occurring in a target link.
Based on the processing, the target word segmentation result obtained by deleting the preset stop word in the initial word segmentation result is input to the fault identification model to obtain the target fault corresponding to the target operation log, so that the calculated amount of the fault identification model can be reduced, and the fault positioning efficiency can be improved.
In some embodiments, before processing the target log based on the trained fault recognition model, the electronic device may further train the fault recognition model of the initial structure based on a preset training sample to obtain the trained fault recognition model.
Referring to fig. 5, fig. 5 is a flowchart of a method for training a network model according to an embodiment of the present invention, where the method may include the following steps:
s501: and acquiring a sample operation log and a sample fault corresponding to the sample operation log.
S502: and processing the sample operation log based on the fault identification model of the initial structure to obtain a fault corresponding to the sample operation log as a predicted fault.
S503: a loss function value is calculated that represents the difference between the predicted fault and the sample fault.
S504: and adjusting model parameters of the fault recognition model of the initial structure based on the calculated loss function value until a preset convergence condition is reached, so as to obtain the trained fault recognition model.
The sample run log is: and continuously integrating the running logs of the sample links with faults in the process in the historical time period. Sample failure represents the cause of failure in the sample link. The sample fault is obtained by analyzing the sample operation log by a user with professional software development knowledge.
The electronic device may obtain a sample log and a sample failure. The electronic device may then process the sample log based on the initially structured fault recognition model to obtain a predicted fault for the sample log, and calculate a loss function value representing a difference between the predicted fault and the sample fault. Furthermore, the electronic device may adjust model parameters of the fault recognition model of the initial structure based on the calculated loss function value until a preset convergence condition is reached, so as to obtain a trained fault recognition model.
The preset convergence condition may be that the number of times of training of the fault identification model of the initial structure reaches a preset number of times, or the preset convergence condition may be that the loss function values obtained by calculation for a continuous preset number of times are all smaller than a preset value.
For step S103, the electronic device may locally store a preset correspondence between the failure and the solution. For example, the correspondence relationship may include:
failure 1: external dedicated dependent files (e.g., jar packages) required for the build link are not uploaded, solution 1: and uploading an external special dependent file required by the construction link.
And (3) failure 2: the source code of the external special dependent file required by the construction link is wrongly written, and the solution 2 is as follows: file dependent modification for writing errors.
Failure 3: there is a syntax error in the source code, solution 3: the source code modification mode aiming at syntax errors.
And 4, fault: configuration files in source code have errors, solution 4: the modification mode of the configuration file aiming at the error.
And (5) failure: the configuration of tools such as container arrangement and the like required by the packaging link has errors, and the solution is as follows, namely, 5: the correct way of configuring tools such as container scheduling.
And 6, fault: the information packed by the Dockerfile has errors, and the solution 6 is as follows: the correct way to configure the Dockerfile-packed information. Dockerfile is a source code file used to make an image, which is an instruction in the process of building a container.
After determining the target failure occurring in the target link, the electronic device may search the correspondence recorded locally to obtain a solution (i.e., a target solution) corresponding to the target failure. The target solution is a processing mode aiming at the target fault.
In some embodiments, after step S103, the method may further comprise the steps of: and displaying the target fault and the target solution to a user so that the user processes the target fault based on the target solution.
After determining the target solution, the electronic device may display the target fault and the target solution for the user to browse. For example, the electronic device may be a server, and the electronic device may push the target failure and the target solution to a terminal used by the user. The terminal may then display the target failure and the target solution to the user. Subsequently, the user can process according to the target solution to solve the target failure of the target link.
Based on the processing, the target fault and the target solution can be displayed to the user, so that the user can process the target fault according to the target solution, and the fault processing efficiency can be improved.
In some embodiments, the fault location method provided by the embodiment of the present invention may also be applied to a Development Operations (develop and operation) scenario, and is used to detect a target link that fails in a continuous integration process in the Development Operations scenario, so as to obtain a target fault and a target solution of the target link. DevOps represents a system that integrates development, operation, maintenance and quality.
Referring to fig. 6, fig. 6 is a flowchart of a persistent integration method provided in the related art, the method being applied to a persistent integration system, the persistent integration system including: a code library (the code database in the previous embodiment), a building module, a packaging module, a publishing module, a log storage module and a display module. The building module, the packaging module and the publishing module are matched with one another to form a production line for generating the target program.
The code base is used for storing source codes developed by users.
The construction module is used for acquiring a source code from the code library, constructing the acquired source code to obtain an intermediate program, and storing the intermediate program to the Artifact database.
The packaging module is used for acquiring the intermediate program from the Artifact database, and packaging the acquired intermediate program to obtain a deployment ready product (namely the target program).
And the issuing module is used for issuing the packaged target program to the server and storing the target program to the Harbor database for backup.
The log storage module is used for storing running logs generated in a construction link, a packaging link and a release link in the continuous integration process.
The display module is used for displaying a target operation log of a target link with a fault.
In the related art, when a fault occurs in a continuous integration process, an operation log of a failed link is manually analyzed to determine a fault reason causing the fault, and a corresponding solution is determined based on the analyzed fault reason. In the process, the user needs to have professional software development knowledge, the running log is analyzed manually, and if the accuracy of self judgment of the user is not high, the user can do a lot of useless work. Therefore, the related art needs to consume a large labor cost and a large time cost for fault location, and further the related art needs to have a high cost for fault processing.
Referring to fig. 7, fig. 7 is a flowchart of a fault location method provided in an embodiment of the present invention, where the method is applied to a fault location system, and the fault location system includes: the system comprises a code library (a code storage database in the previous embodiment), a construction module, a packaging module, a release module, a log storage module, a trigger module, an extraction module, a problem identification module, a matching module and a display module. The building module, the packaging module and the publishing module are matched with one another to form a production line for generating the target program.
The code base is used for storing source codes developed by users.
The construction module is used for acquiring a source code from the code library, constructing the acquired source code to obtain an intermediate program, and storing the intermediate program to the Artifact database.
The packaging module is used for acquiring the intermediate program from the Artifact database, and packaging the acquired intermediate program to obtain a deployment ready product (namely the target program).
And the issuing module is used for issuing the packaged target program to the server and storing the target program to the Harbor database for backup.
The log storage module is used for storing running logs generated in a construction link, a packaging link and a release link in the continuous integration process.
The triggering module is used for triggering the assistance flow according to the instruction of the user, namely when any one of the construction link, the packaging link and the release link fails, the user can trigger the assistance flow through an assistance entry provided by the triggering module. Namely, the user inputs a processing instruction of an error report message aiming at the target link to the trigger module.
The extraction module is used for automatically extracting the corresponding log line from the log storage module after the user triggers the assistance process, namely extracting the running log of the target link.
And the problem identification module is used for taking the extracted log line as the input of the fault identification model and automatically identifying the problem on the current assembly line. Namely, the target fault of the target link is automatically identified based on the extracted target operation log and the fault identification model.
Specifically, the problem identification module may obtain a configuration file and an expectation of the fault identification model with the initial structure, where the configuration file records a topology structure and a weight parameter of the fault identification model with the initial structure, the expectation includes a training document and a verification document, the training document includes a sample operation log used for training the fault identification model with the initial structure and a sample fault, and the verification document includes a verification operation log used for verifying a training effect of the identification model and a verification fault corresponding to the verification operation log.
The problem identification module may then load the configuration file and the expectations through the adaptation layer. Namely, the problem identification module carries out format conversion on the expectation to obtain the expectation of the format supported by the fault identification model. The problem recognition module can also perform word segmentation on the expectation after format conversion, and delete preset stop words from the obtained word segmentation result to obtain a target word segmentation result. The problem recognition module can divide the target word segmentation result to obtain a divided data set, and the divided data set comprises training materials and verification materials.
The problem recognition module may load parameters and train the model. The problem recognition module loads the fault recognition model of the initial structure according to the topological structure and the weight parameters of the fault recognition model of the initial structure recorded in the configuration file, and trains the fault recognition model of the initial structure based on the training files to obtain the trained fault recognition model.
The problem recognition module can perform model evaluation, namely, verify the training effect of the fault recognition model based on verification documents. Then, the problem identification module can perform model persistence processing, namely, adjust the topological structure and the weight parameters of the fault identification model based on the verification result to obtain the finally trained fault identification model.
Furthermore, the problem identification module can perform model verification and output a result. Namely, the target operation log is processed based on the trained fault recognition model, and the target fault corresponding to the target operation log is obtained.
And the matching module is used for matching a corresponding target solution according to the preset fault and the item of the solution after the target fault is determined. That is, in the preset corresponding relationship between the failure and the solution, the solution corresponding to the target failure is determined as the target solution corresponding to the target link.
And the display module is used for presenting the determined target faults and the corresponding target solutions to a user for reference and assisting the user in fault treatment. Namely, the target fault and the target solution are displayed for the user to browse, so that the user can process the target fault according to the target solution.
Based on the processing, under the condition that the target link in the continuous integration process of the target program fails, the target fault corresponding to the target link can be automatically determined, and then the target solution corresponding to the target fault is automatically determined, so that the problems that the fault occurs in the continuous integration process and the solution is difficult to find or low in accuracy are solved. Even if a user does not have professional software development knowledge, the target fault and the target solution of the target link can be automatically determined, the accuracy of fault processing can be improved, the operation log does not need to be analyzed manually, the cost of fault positioning can be reduced, and the cost of fault processing is further reduced.
Corresponding to the embodiment of the method in fig. 1, referring to fig. 8, fig. 8 is a structural diagram of a fault location apparatus provided in an embodiment of the present invention, where the apparatus includes:
a first obtaining module 801, configured to obtain, as a target operation log, an operation log of a target link in a continuous integration process of a target program when the target link fails;
a first identification module 802, configured to process the target operation log based on a pre-trained fault identification model, to obtain a fault corresponding to the target operation log, where the fault is used as a target fault; the identification model is obtained by training a sample operation log of a sample link with a fault and a sample fault corresponding to the sample operation log;
a first determining module 803, configured to determine, in a preset correspondence between a fault and a solution, a solution corresponding to the target fault as a target solution.
Optionally, the first obtaining module is specifically configured to display an error report message for a target link when detecting that the target link in a continuous integration process for the target program fails;
and when a processing instruction aiming at the error report message input by a user is received, determining an operation log corresponding to the link identifier of the target link, and acquiring the operation log corresponding to the program identifier of the target program from the determined operation log to be used as a target operation log.
Optionally, the first identification module is specifically configured to perform word segmentation on the log content of the target operation log to obtain an initial word segmentation result;
deleting preset stop words from the initial respective results to obtain target word segmentation results;
and inputting the target word segmentation result to a pre-trained fault recognition model, and obtaining a fault corresponding to the target operation log output by the fault recognition model as a target fault.
Optionally, the apparatus further comprises:
and the display module is used for displaying the target fault and the target solution to a user so that the user processes the target fault based on the target solution after the first determination module determines the solution corresponding to the target fault in the preset corresponding relation between the fault and the solution as the target solution.
Optionally, the apparatus further comprises:
the second acquisition module is used for acquiring the sample operation log and a sample fault corresponding to the sample operation log;
the second identification module is used for processing the sample operation log based on the fault identification model of the initial structure to obtain a fault corresponding to the sample operation log as a predicted fault;
a second determining module for calculating a loss function value representing a difference between the predicted fault and the sample fault;
and the training module is used for adjusting the model parameters of the fault recognition model of the initial structure based on the calculated loss function value until a preset convergence condition is reached, so as to obtain the trained fault recognition model.
Based on the fault positioning device provided by the embodiment of the invention, under the condition that the target link fails in the continuous integration process of the target program, the target fault corresponding to the target link can be automatically determined, and then the target solution corresponding to the target fault can be automatically determined, even if a user does not have professional software development knowledge, the target fault and the target solution of the target link can be determined, and the operation log does not need to be analyzed manually, so that the cost for fault positioning can be reduced, and the cost for fault processing can be further reduced.
An embodiment of the present invention further provides an electronic device, as shown in fig. 9, including a processor 901, a communication interface 902, a memory 903 and a communication bus 904, where the processor 901, the communication interface 902, and the memory 903 complete mutual communication through the communication bus 904;
a memory 903 for storing computer programs;
the processor 901 is configured to implement the following steps when executing the program stored in the memory 903:
under the condition that a target link in the continuous integration process of a target program fails, acquiring an operation log of the target link as a target operation log;
processing the target operation log based on a pre-trained fault recognition model to obtain a fault corresponding to the target operation log as a target fault; the identification model is obtained by training a sample operation log of a sample link with a fault and a sample fault corresponding to the sample operation log;
and determining a solution corresponding to the target fault in a preset corresponding relation between the fault and the solution as a target solution.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
In yet another embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the above fault location processing methods.
In a further embodiment provided by the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the fault location methods of the above embodiments.
In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus, the electronic device, the computer-readable storage medium, and the computer program product embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and in relation to them, reference may be made to the partial description of the method embodiments.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (12)
1. A method of fault location, the method comprising:
under the condition that a target link in the continuous integration process of a target program fails, acquiring an operation log of the target link as a target operation log;
processing the target operation log based on a pre-trained fault recognition model to obtain a fault corresponding to the target operation log as a target fault; the identification model is obtained by training a sample operation log of a sample link with a fault and a sample fault corresponding to the sample operation log;
and determining a solution corresponding to the target fault in a preset corresponding relation between the fault and the solution as a target solution.
2. The method according to claim 1, wherein in the case of a failure of a target link in the continuous integration process for the target program, acquiring the operation log of the target link as the target operation log comprises:
when detecting that a target link in the continuous integration process aiming at a target program fails, displaying an error report message aiming at the target link;
and when a processing instruction aiming at the error report message input by a user is received, determining an operation log corresponding to the link identifier of the target link, and acquiring the operation log corresponding to the program identifier of the target program from the determined operation log to be used as a target operation log.
3. The method according to claim 1, wherein the processing the target operation log based on the pre-trained fault recognition model to obtain a fault corresponding to the target operation log as a target fault comprises:
performing word segmentation on the log content of the target operation log to obtain an initial word segmentation result;
deleting preset stop words from the initial respective results to obtain target word segmentation results;
and inputting the target word segmentation result into a pre-trained fault recognition model, and obtaining a fault corresponding to the target operation log output by the fault recognition model as a target fault.
4. The method according to claim 1, wherein after determining a solution corresponding to the target fault in the preset correspondence relationship between faults and solutions as a target solution, the method further comprises:
and displaying the target fault and the target solution to a user so that the user processes the target fault based on the target solution.
5. The method of claim 1, wherein the training process of the fault recognition model comprises the steps of:
obtaining the sample operation log and a sample fault corresponding to the sample operation log;
processing the sample operation log based on a fault identification model of an initial structure to obtain a fault corresponding to the sample operation log as a predicted fault;
calculating a loss function value representing a difference between the predicted fault and the sample fault;
and adjusting model parameters of the fault recognition model of the initial structure based on the calculated loss function value until a preset convergence condition is reached, so as to obtain a trained fault recognition model.
6. A fault locating device, characterized in that the device comprises:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring an operation log of a target link as a target operation log under the condition that the target link fails in the continuous integration process of a target program;
the first identification module is used for processing the target operation log based on a pre-trained fault identification model to obtain a fault corresponding to the target operation log as a target fault; the identification model is obtained by training a sample operation log of a sample link with a fault and a sample fault corresponding to the sample operation log;
and the first determining module is used for determining a solution corresponding to the target fault in a preset corresponding relation between the fault and the solution as a target solution.
7. The apparatus according to claim 6, wherein the first obtaining module is specifically configured to display an error message for a target link in a continuous integration process of a target program when a failure of the target link is detected;
and when a processing instruction aiming at the error report message input by a user is received, determining an operation log corresponding to the link identifier of the target link, and acquiring the operation log corresponding to the program identifier of the target program from the determined operation log to be used as a target operation log.
8. The apparatus according to claim 6, wherein the first recognition module is specifically configured to perform a word segmentation process on log content of the target running log to obtain an initial word segmentation result;
deleting preset stop words from the initial respective results to obtain target word segmentation results;
and inputting the target word segmentation result into a pre-trained fault recognition model, and obtaining a fault corresponding to the target operation log output by the fault recognition model as a target fault.
9. The apparatus of claim 6, further comprising:
and the display module is used for displaying the target fault and the target solution to a user so that the user processes the target fault based on the target solution after the first determination module determines the solution corresponding to the target fault in the preset corresponding relation between the fault and the solution as the target solution.
10. The apparatus of claim 6, further comprising:
the second acquisition module is used for acquiring the sample operation log and a sample fault corresponding to the sample operation log;
the second identification module is used for processing the sample operation log based on the fault identification model of the initial structure to obtain a fault corresponding to the sample operation log as a predicted fault;
a second determining module for calculating a loss function value representing a difference between the predicted fault and the sample fault;
and the training module is used for adjusting the model parameters of the fault recognition model of the initial structure based on the calculated loss function value until a preset convergence condition is reached, so as to obtain the trained fault recognition model.
11. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 1 to 5 when executing a program stored in the memory.
12. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of the claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210506106.3A CN114896094A (en) | 2022-05-10 | 2022-05-10 | Fault positioning method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210506106.3A CN114896094A (en) | 2022-05-10 | 2022-05-10 | Fault positioning method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114896094A true CN114896094A (en) | 2022-08-12 |
Family
ID=82722737
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210506106.3A Pending CN114896094A (en) | 2022-05-10 | 2022-05-10 | Fault positioning method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114896094A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115408112A (en) * | 2022-11-02 | 2022-11-29 | 确信信息股份有限公司 | Bug processing method, system, medium and equipment for intranet production environment |
-
2022
- 2022-05-10 CN CN202210506106.3A patent/CN114896094A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115408112A (en) * | 2022-11-02 | 2022-11-29 | 确信信息股份有限公司 | Bug processing method, system, medium and equipment for intranet production environment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10503478B2 (en) | System and method for guiding a user in a software development lifecycle using machine learning | |
| US20230251960A1 (en) | Machine learning techniques for automated software testing configuration management | |
| US20200401382A1 (en) | Autonomously delivering software features | |
| US11294755B2 (en) | Automated method of identifying troubleshooting and system repair instructions using complementary machine learning models | |
| WO2018153140A1 (en) | System and method for recommending service experience package | |
| CN111563051B (en) | Crawler-based data verification method and device, computer equipment and storage medium | |
| CN113255792B (en) | Data anomaly point detection method, device, system and storage medium | |
| CN110851471A (en) | Distributed log data processing method, device and system | |
| CN109324956B (en) | System testing method, apparatus and computer readable storage medium | |
| CN113778864A (en) | Test case generation method and device, electronic equipment and storage medium | |
| CN113419950A (en) | Method and device for generating UI automation script, computer equipment and storage medium | |
| CN114896094A (en) | Fault positioning method and device, electronic equipment and storage medium | |
| CN110908903B (en) | Test method based on editable YAML file | |
| CN113627168A (en) | Method, device, medium and equipment for checking component packaging conflict | |
| CN113947076A (en) | Policy data detection method and device, computer equipment and storage medium | |
| CN113590172B (en) | Code file publishing method, device, equipment and storage medium | |
| CN114492576A (en) | Abnormal user detection method, system, storage medium and electronic equipment | |
| WO2021183382A1 (en) | Graph-based method for inductive bug localization | |
| US9830215B1 (en) | Computing system error analysis based on system dump data | |
| CN115098362B (en) | Page test method, page test device, electronic equipment and storage medium | |
| CN113705097B (en) | Vehicle model construction method and device, computer equipment and storage medium | |
| EP3941035A1 (en) | Digital evidence management method and digital evidence management system | |
| CN118194277B (en) | Privacy behavior consistency analysis method, device and medium based on APP usage scene | |
| US20240289262A1 (en) | Automatic testing with feature tags trained by machine learning for updates in version control systems | |
| JP6085574B2 (en) | Work record content analysis apparatus, method and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |