CN117453485A - Database defect analysis method, device, equipment and storage medium - Google Patents

Database defect analysis method, device, equipment and storage medium Download PDF

Info

Publication number
CN117453485A
CN117453485A CN202311497857.4A CN202311497857A CN117453485A CN 117453485 A CN117453485 A CN 117453485A CN 202311497857 A CN202311497857 A CN 202311497857A CN 117453485 A CN117453485 A CN 117453485A
Authority
CN
China
Prior art keywords
defect
historical
defects
target
stack information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311497857.4A
Other languages
Chinese (zh)
Inventor
崔校郡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Baidu International Technology Shenzhen Co ltd
Original Assignee
Baidu International Technology Shenzhen Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Baidu International Technology Shenzhen Co ltd filed Critical Baidu International Technology Shenzhen Co ltd
Priority to CN202311497857.4A priority Critical patent/CN117453485A/en
Publication of CN117453485A publication Critical patent/CN117453485A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3034Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a storage system, e.g. DASD based or network based
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/079Root cause analysis, i.e. error or fault diagnosis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0793Remedial or corrective actions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3065Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G06F11/3072Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Abstract

The disclosure provides a database defect analysis method, a device, equipment and a storage medium, relates to the technical field of computers, in particular to the technical fields of data processing, data analysis and the like, and can be applied to the scenes of database data processing, database data analysis and the like. The specific implementation scheme comprises the following steps: stack information of target defects in a database is obtained; and inputting the stack information of the target defect into a preset defect analysis model, and predicting and outputting a solution corresponding to the target defect according to the stack information of the target defect through the defect analysis model. The method and the device can rapidly and accurately predict and output the solution corresponding to the target defect through the preset defect analysis model, improve the efficiency and accuracy of solving the target defect, and greatly reduce the labor cost.

Description

Database defect analysis method, device, equipment and storage medium
Technical Field
The disclosure relates to the technical field of computers, in particular to the technical fields of data processing, data analysis and the like, and can be applied to the scenes of database data processing, database data analysis and the like, in particular to a database defect analysis method, a device, equipment and a storage medium.
Background
The log files of the database have a critical role in the operation and management of the database system, which is used to record all the change operations performed in the database, such as insert, update, delete, etc. The log file may include a transaction log, an error log, and the like. The error log stores errors and abnormal conditions of the database system, and a user can analyze and process fault problems generated in the database according to the error log in the log file.
In the prior art, a user can search in a fault elimination website according to keywords in an error log, and inquire identification information and a solution corresponding to the fault.
However, the above-mentioned method of manually retrieving the identification information and the solution corresponding to the fault from the online website is not intelligent enough, and when the user does not know the fault log, it is difficult for the user to determine the keyword from the fault log, so that the user cannot solve the fault problem, and the working efficiency is affected.
Disclosure of Invention
The invention provides a database defect analysis method, a device, equipment and a storage medium, which can rapidly and accurately predict and output a solution corresponding to a target defect through a preset defect analysis model, improve the efficiency and accuracy of solving the target defect and greatly reduce the labor cost.
According to a first aspect of the present disclosure, there is provided a database defect analysis method, the method comprising: stack information of target defects in a database is obtained; and inputting the stack information of the target defect into a preset defect analysis model, and predicting and outputting a solution corresponding to the target defect according to the stack information of the target defect through the defect analysis model.
According to a second aspect of the present disclosure, there is provided a model training method comprising: stack information of historical defects in a database is obtained, and a solution of the historical defects is obtained; training the neural network by adopting stack information of the historical defects and solutions of the historical defects to obtain a defect analysis model, wherein the defect analysis model has the function of predicting and outputting solutions corresponding to the target defects according to the input stack information of the target defects.
According to a third aspect of the present disclosure, there is provided a database defect analysis apparatus, the apparatus comprising: an acquisition unit and an output unit.
And the acquisition unit is used for acquiring stack information of the target defect in the database.
The output unit is used for inputting the stack information of the target defect into a preset defect analysis model, and predicting and outputting a solution corresponding to the target defect according to the stack information of the target defect through the defect analysis model.
According to a fourth aspect of the present disclosure, there is provided a model training apparatus, the apparatus comprising: extraction unit, training unit.
And the extraction unit is used for acquiring stack information of the historical defects in the database and solving the historical defects.
The training unit is used for training the neural network by adopting the stack information of the historical defects and the solution of the historical defects to obtain a defect analysis model, and the defect analysis model has the function of predicting and outputting the solution corresponding to the target defects according to the input stack information of the target defects.
According to a fifth aspect of the present disclosure, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method as in the first aspect, or in the second aspect.
According to a sixth aspect of the present disclosure there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method according to the first aspect, or the second aspect.
According to a seventh aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method according to the first aspect, or the second aspect.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a flowchart of a database defect analysis method according to an embodiment of the present disclosure;
FIG. 2 is a flow chart of a model training method provided in an embodiment of the present disclosure;
FIG. 3 is another flow chart of a model training method provided by an embodiment of the present disclosure;
FIG. 4 is a schematic flow chart of a model training method according to an embodiment of the disclosure;
FIG. 5 is a schematic flow chart of a model training method according to an embodiment of the disclosure;
FIG. 6 is a schematic diagram of a system component of a database defect analysis method according to an embodiment of the present disclosure;
FIG. 7 is a schematic diagram of a database defect analysis apparatus according to an embodiment of the present disclosure;
FIG. 8 is a schematic diagram illustrating the composition of a model training apparatus according to an embodiment of the present disclosure
Fig. 9 is a schematic block diagram of an example electronic device 900 that may be used to implement embodiments of the present disclosure provided by embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be appreciated that in embodiments of the present disclosure, the character "/" generally indicates that the context associated object is an "or" relationship. The terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated.
The log files of the database have a critical role in the operation and management of the database system, which is used to record all the change operations performed in the database, such as insert, update, delete, etc. The log file may include a transaction log, an error log, and the like. The error log stores errors and abnormal conditions of the database system, and a user can analyze and process fault problems generated in the database according to the error log in the log file.
Illustratively, the log file of the database is a file that records detailed information of database operations. When the database performs a write operation, the detailed information of the operation may be written to the log file. The detailed information includes the start and commit of the transaction, the insertion, modification, deletion of the data, and the like. The log file of the database may also be used to store error information, which may be stored in an error log of the database when a transaction in operation of the database fails. The user can analyze the cause of the fault based on the error log. The user can extract the keywords in the error log to analyze the main reasons of the fault and analyze and solve the fault according to the main reasons of the fault.
In the prior art, a user can search in a fault elimination website according to keywords in an error log, and inquire identification information and a solution corresponding to the fault.
For example, after the user obtains the error log, the user may extract the keywords in the error log, and after the keywords are extracted, the user may search in the fault analysis website of the database according to the keywords, so as to search the fault number corresponding to the corresponding keywords and the solution corresponding to the fault. The user can analyze and solve the faults according to the solutions corresponding to the faults, so that the normal operation of the database is ensured.
However, the above-mentioned method of manually retrieving the identification information and the solution corresponding to the fault from the online website is not intelligent enough, and when the user does not know the fault log, it is difficult for the user to determine the keyword from the fault log, so that the user cannot solve the fault problem, and the working efficiency is affected.
For example, the fault log may include keywords corresponding to faults. When the keyword is obtained according to the error log, the user is required to know the error log, the keyword corresponding to the fault can be extracted rapidly according to the error information, and the fault related solution can be retrieved according to the keyword. When the user does not know the error log, the user can hardly extract the keywords corresponding to the faults from the complex and professional error log, so that the fault problem corresponding to the error log can not be solved in time, and the working efficiency can be seriously influenced.
Under the background technology, the disclosure provides a database defect analysis method, which can rapidly and accurately predict and output a solution corresponding to a target defect through a preset defect analysis model, so that the efficiency and accuracy of solving the target defect are improved, and the labor cost is greatly reduced.
The subject of the database defect analysis method may be a computer or a server, or may be other devices with data processing capabilities, for example. The subject of execution of the method is not limited herein.
In some embodiments, the server may be a single server, or may be a server cluster formed by a plurality of servers. In some implementations, the server cluster may also be a distributed cluster. The present disclosure is not limited to a specific implementation of the server.
Fig. 1 is a flowchart illustrating a method for analyzing database defects according to an embodiment of the present disclosure. As shown in fig. 1, the method may include steps S101-S102.
S101, acquiring stack information of target defects in a database.
For example, the stack information of the target defect in the database may refer to error information generated in the database, such as Crash, bug, crash information, GUG information, etc., where the stack information of the target defect in the database is not specifically limited. The stack information of the target defect in the database may be stored in an error log, a log file, a core file, or the like of the database, and the storage location of the stack information of the target defect in the database is not particularly limited.
For example, when the database is in error in performing insertion, modification and deletion of data, corresponding error information is generated and stored in an error log of the database. According to the error log of the database, the stack information of the target defect in the database can be obtained, which is used for indicating the fault generated in the running process of the database, for example, the stack information of the target defect in the obtained data can be: agent no longer collects tablestatus data items.
S102, inputting stack information of the target defect into a preset defect analysis model, and predicting and outputting a solution corresponding to the target defect according to the stack information of the target defect through the defect analysis model.
For example, the preset defect analysis model may take stack information of the database defect as input, take a solution corresponding to the stack information of the database defect as output, and train the neural network model to obtain a preset defect classification model. In addition, an information input interface and a solution output page can be provided for the user at the client, and the user can input stack information of the target defect in the information input interface; the user can view the solution corresponding to the target defect in the solution output page.
For example, the stack information of the target defect may be: by a preset defect analysis model, "agent no longer collects tablestatus data items", the solution for predicting and outputting the target defect can be: you can use INFORMATION —schema. Tabs to get this information ".
According to the method and the device, the stack information of the target defects in the database is acquired, the stack information of the target defects is input into the preset defect analysis model, the solutions corresponding to the target defects are predicted and output through the defect analysis model according to the stack information of the target defects, the solutions corresponding to the target defects can be rapidly and accurately predicted and output through the preset defect analysis model, the efficiency and the accuracy of solving the target defects are improved, and the labor cost is greatly reduced.
In some embodiments, the database defect analysis method further comprises: and predicting and outputting a defect identifier corresponding to the target defect according to the stack information of the target defect through the defect analysis model.
Illustratively, the defect identification corresponding to the target defect may be used to indicate the uniqueness of the target defect. The defect identifier corresponding to the target defect may be in the form of a character string, a number, etc., and may be set according to actual needs, where the form of the defect identifier corresponding to the target defect is not specifically limited.
For example, the defect identifier corresponding to the target defect may be in the form of a character string, for example: "Null Pointer Reference", "Memory leak" may also be in the form of numbers, such as: "913", "852", and the like. The stack information of the input target defect may be: the "agent no longer collects tablestatus data items" corresponding defect identification corresponding to the target defect is predicted and output according to the stack information of the target defect through the defect analysis model, and may be: "913", etc.
According to the method and the device, the defect identification corresponding to the target defect is predicted and output according to the stack information of the target defect through the defect analysis model, the defect identification corresponding to the stack information of the target defect can be rapidly and accurately output, further, the stack information of the target defect and the corresponding defect identification can be bound through the defect identification corresponding to the stack information of the target defect, so that the working efficiency is improved, and a user can rapidly and accurately search the stack information and the solution of the target defect according to the defect identification, so that the working efficiency is further improved.
In some embodiments, the solution to the target defect includes at least one of: the analysis result of stack information of the target defect, the problem description and problem reproduction method of other users related to the target defect, the historical version information of the target defect, and the historical version information of the target defect which is solved.
Illustratively, the analysis result of the stack information of the target defect may be the type and location of the target defect, the trigger source of the target defect, the call relationship and call path of the target defect, the root cause analysis result of the target defect, the related resources and states of the target defect, and the like. The problem descriptions of other users associated with the target defect may be descriptions of the target defect by all users associated with the target defect or descriptions of the target defect by some of the users associated with the target defect. The historical version information of the target defect after occurrence can be the historical version information of all corresponding databases when the target defect occurs or the historical version information of corresponding partial databases when the target defect occurs. The historical version information for which the target defect is resolved may be historical version information for all databases for which the target defect is resolved or historical version information for a portion of databases for which the target defect is resolved.
The solution to the target defect in this embodiment includes at least one of the following: the stack information analysis result of the target defect, the problem description and problem reproduction method of other users related to the target defect, the history version information of the target defect and the history version information of the target defect which is solved, the user can quickly and accurately solve the target defect according to the solution of the target defect, and can inquire all information related to the target defect and view the information for the user, so that the user can be helped to quickly know and locate the target defect, the target defect can be solved more quickly and accurately, and the working efficiency is improved.
The disclosure may be applied to a scenario of analysis of a relational (Relational Database Service, RDS) database instance Crash stack information (i.e., stack information of a target defect) of a private cloud, a scenario of an RDS custom problem analysis case library constructed by a private cloud database manager (Database Administrator, DBA), and a scenario of all system schemes with open source software analysis and experience case sets. Taking an example of an RDS database instance Crash stack information analysis scene applied to private cloud, when a user cannot rapidly analyze the root cause corresponding to stack information of a target defect, after receiving an error warning of the database instance, the user can acquire the stack information of the target defect, store the stack information as text data, input the text data into a defect analysis model, and return a solution corresponding to the stack information of the target defect and a defect identifier.
In some embodiments, before step S102, the database defect analysis method may further include: and segmenting the stack information of the target defect to obtain at least one keyword corresponding to the stack information of the target defect. Step S102 includes: and inputting at least one keyword corresponding to the stack information of the target defect into a defect analysis model.
For example, the word segmentation of the stack information of the target defect may be implemented by using a function word_token of ntk in Python language, or may be implemented by using other functions or a neural network model, where the word segmentation of the stack information of the target defect is not specifically limited, so long as the word segmentation of the stack information of the target defect can be implemented. After at least one keyword corresponding to stack information of the target defect is acquired, the keyword may be input into the defect analysis model.
For example, the stack information of the target defect may be: "list index out of bounds (0)", the keywords obtained after analysis of stack information of the target defect using a function word_token of ntk in the Python language may be "list", "index", "out of bound", "(0)".
According to the embodiment, before the stack information of the target defect is input into the preset defect analysis model, the stack information of the target defect is segmented to obtain at least one keyword corresponding to the stack information of the target defect, the stack information of the target defect can be decomposed into smaller and more specific words or phrases, so that a user can quickly locate the target defect, and further, the stack information of the target defect is segmented to better classify the keywords, so that the target defect can be classified, and the defect analysis model can more quickly search out a solution corresponding to the target defect according to the keywords, thereby improving the working efficiency.
Fig. 2 is a flow chart of a model training method according to an embodiment of the disclosure. As shown in fig. 2, the model training method may include steps S201 to S202.
S201, stack information of historical defects in a database and a solution of the historical defects are obtained.
The stack information of the historical defect in the database and the solution of the historical defect can be obtained from an online database and stored in the database, or can be input into the database by a user, and the stack information of the historical defect in the database and the solution of the historical defect are not limited.
For example, stack information of historical defects in a database, and solutions to historical defects may be obtained from an online database and stored in the database. An online database may refer to an online database storing or querying historical defects, e.g., a database corresponding to com, from which a user may crawl data using crawler tools, e.g., crawl network addresses may be as follows:
https://****.****.com/search.phpcmd=display&status[]=All&os=0&cpu_arch=0&bug_age=0&last_updated=0&phpver=5.7&limit=30&mine=0&begin=30。
where phpver=5.7 is a version key of the database, limit=30 indicates that data of each page is displayed as 30 lines, begin=30 indicates that a search is started from page 30. After crawling, the crawled data may be stored in a database.
For another example, stack information for the historical defect in the database, and solutions to the historical defect may also be user entered, and the user may enter stack information for common or unique historical defects, and solutions to the historical defect into the database.
S202, training a neural network by adopting stack information of historical defects and solutions of the historical defects to obtain a defect analysis model, wherein the defect analysis model has the function of predicting and outputting solutions corresponding to target defects according to the input stack information of the target defects.
Illustratively, the neural network model may use a convolutional neural network, may use a network employing a residual-brown similarity algorithm, or may employ a bayesian prior network, without limitation to the specific implementation of the neural network.
For example, taking a neural network as an example of a bayesian prior network, examples of codes using the bayesian prior network may be: classifer=naive bayes class identifier.
For another example, a code example of predicting and outputting a solution corresponding to the target defect may be: f=open ('class. Clip', 'rb'); classifer = pickle. Load (f); close (); f=open ('mysql. Err', 'r'); print ("Dear, acordingtexthipmql. Err, the BugsNo#is: #" + prediction (word_token ()), classifier); close ().
According to the embodiment, the stack information of the historical defects in the database and the solution of the historical defects are obtained, and the neural network is trained by adopting the stack information of the historical defects and the solution of the historical defects, so that the defect analysis model is obtained, the analysis time can be shortened, the solution corresponding to the target defects is rapidly and accurately output through the defect analysis model, the rapid classification and positioning of the target defects are realized, the target defects are timely solved, the labor cost is reduced, and the working efficiency is improved.
In some embodiments, the historical defects include online historical defects obtained from an online historical defect library and local historical defects preset by a user.
Illustratively, an online history defect may refer to data entered in a database by a user by retrieving history defects in an online database and storing the data in the database.
In this embodiment, the historical defects include online historical defects and local historical defects, the online historical defects are obtained from an online historical defect library, the local historical defects are preset by a user, so that the historical defects in the database are more complete and conform to the actual demands of the user, further, the trained defect classification model can analyze the target defects more flexibly and comprehensively, and the solutions corresponding to the target defects are output, so that the working efficiency is further improved.
Fig. 3 is another flow chart of a model training method according to an embodiment of the disclosure. As shown in fig. 3, the model training method may further include step S301 to step S302, training the neural network by using stack information of the historical defect and a solution of the historical defect to obtain a defect analysis model, including: and training the neural network by adopting stack information of the updated online historical defects and a solution of the updated online historical defects to obtain a defect analysis model.
S301, responding to the operation of updating the online historical defect, and acquiring the incremental historical defect.
S302, updating the online historical defects according to the increment historical defects to obtain updated online historical defects.
The operation of updating the online history defect may be triggered by the user clicking on an "update" button, or may be a timing operation of the system setting. The incremental historical defects can be updated historical defects in the online database, and the mode of acquiring the incremental historical defect data can be acquired from the online database in a crawling mode or can be downloaded from the online database by a user. The updating of the online history defect may be performed by adding the incremental history defect to the online history defect, or by replacing the online history defect with the incremental history defect, which is not limited herein.
For example, a user may click on an "update" button to trigger an operation to update the online history defect and use the crawler model to crawl the incremental history defect from the online database and store the incremental history defect in the database to update the online history defect.
In this embodiment, the incremental historical defects may be obtained by responding to the operation of updating the online historical defects, and the online historical defects may be updated according to the incremental historical defects, so that the data integrity of the online historical defects may be ensured, and further, the defect analysis model may analyze the target defects more flexibly and comprehensively, and output a solution corresponding to the target defects, so as to further improve the working efficiency.
Fig. 4 is a schematic flow chart of a model training method according to an embodiment of the disclosure. As shown in fig. 4, the model training method may further include step S401 to step S402, training the neural network by using stack information of the historical defect and a solution of the historical defect to obtain a defect analysis model, including: and training the neural network by adopting stack information of the updated local historical defects and a solution of the updated local historical defects to obtain a defect analysis model.
S401, receiving the local increment historical defect input by a user.
S402, updating the local historical defects according to the local increment historical defects to obtain updated local historical defects.
Illustratively, an input interface may be provided at the client for the user, in which the user may enter the local incremental history defect. According to the local increment historical defect, the local increment historical defect can be added into the local historical defect to update the local historical defect, or the local increment historical defect can be used for replacing the local historical defect, so that the purpose of updating the local historical defect is achieved.
According to the embodiment, the local increment historical defects input by the user can be received, the local increment historical defects are updated according to the local increment historical defects, the data integrity of the local historical defects can be guaranteed, further, the defect analysis model can analyze target defects more flexibly and comprehensively, a solution corresponding to the target defects is output, and the working efficiency is further improved.
In some embodiments, using stack information of the historical defect and a solution of the historical defect, training the neural network to obtain the defect analysis model, the model training method may further include: and cleaning the stack information of the historical defects and the solution of the historical defects according to a preset character string database to obtain the stack information of the cleaned historical defects and the solution of the historical defects. Step S202 may include: and training the neural network by adopting stack information of the cleaned historical defects and a solution of the historical defects to obtain a defect analysis model.
The preset character string database may be a database for storing preset character strings, and the preset character strings may be input by a user and stored in the database, or may be obtained and stored in the database by acquiring online common character strings. The data cleaning method for the stack information of the historical defect and the solution of the historical defect may be implemented by comparing the character strings in the stack information of the historical defect and the solution of the historical defect with the character strings in the preset character string database and deleting the character strings in the preset character string database.
Alternatively, data cleansing of stack information of historical defects and solutions to historical defects may be implemented by a natural language processing toolkit (Natural Language Toolkit, NLTK) module. NLTK is a Python library commonly used in the field of natural language processing research, and can be used for cleaning stack information of historical defects and solutions of the historical defects.
For example, stack information of the history defect may be: "list index out of bounds (0)", the solution of the history defect may be: "agent no longer collects tablestatus data items", the preset character string included in the character string database may be: "list". Comparing the stack information of the history defect with the character strings in the preset character string database, deleting the character strings in the preset character string database, and obtaining the stack information of the history defect and the solution of the history defect after cleaning, wherein the solutions of the history defect are as follows: "indexout of bounds (0)", "agent no longer collects tablestatus data items", and training the neural network to obtain a defect analysis model using stack information of the cleaned historical defects and solutions of the historical defects.
Before the neural network is trained to obtain the defect analysis model, the stack information of the historical defect and the solution of the historical defect are subjected to data cleaning according to the preset character string database to obtain the stack information of the historical defect and the solution of the historical defect after cleaning, corresponding character strings in the stack information of the historical defect and the solution of the historical defect and the preset character string database can be cleaned, and then the stack information of the historical defect and the solution of the historical defect after cleaning are more in line with the requirements of users, so that the defect analysis model can analyze target defects more flexibly, output the solution corresponding to the target defects, and further improve the working efficiency.
Fig. 5 is a schematic flow chart of a model training method according to an embodiment of the disclosure. As shown in fig. 5, the model training method may further include step S501 to step S502.
S501, receiving character string information input by a user.
S502, updating a preset character string database according to the character string information.
For example, a character string information input interface may be provided for a user at a client, and the user may input character string information in the character string information input interface. The updating of the preset character string database according to the character string information can be realized by adding the character string information into the preset character string database, or can be realized by replacing the character string in the preset character string database by using the character string information so as to realize the aim of updating the preset character string database.
For example, the character string information input by the user may be: "i ndex", the character strings in the preset character string database may be: the step of updating the preset character string database according to the character string information can be to add the character string "i ndex" into the preset character string database so as to achieve the purpose of updating the preset character string database.
In some embodiments, the string information includes at least one of: nonsensical character strings, special character strings, misleading character strings.
For example, the user can take some meaningless character strings, special character strings and character strings which are easy to mislead as input according to actual demands, and update the preset character string database, so that the preset character string database can better meet the actual demands of the user, and the working efficiency is improved.
According to the embodiment, the preset character string database is updated according to the character string information by receiving the character string information input by the user, so that the integrity and the instantaneity of the preset character string database can be enhanced, the accuracy of inquiry and search can be improved, the content and the application field of the database can be expanded, the user experience and satisfaction can be improved, and the iterative performance and the improvement of the preset character string database can be kept.
Fig. 6 is a schematic diagram of a system composition of a database defect analysis method according to an embodiment of the disclosure. As shown in fig. 6, in one particular embodiment, the database defect analysis system may include: the system comprises an online defect library data crawling module, a local database module, an incremental defect data crawling module, a local defect input module, a model training module, a model storage module, a defect analysis module, a maintenance module and the like.
The online defect library data crawling module can acquire stack information of historical defects from an online database and a solution of the historical defects, and store the stack information and the solution of the historical defects in the local database module. The incremental defect data crawling module may obtain the incremental historical defects from the online database in response to the operation of updating the online historical defects, and update the online historical defects according to the incremental historical defects and store the updated online historical defects in the local database module. The local defect input module may receive the local incremental history defect input by the user, update the local historical defect according to the local incremental history defect, and store the local incremental history defect in the local database module. The model training module can acquire stack information of the historical defects in the database and a solution of the historical defects, and train the neural network by adopting the stack information of the historical defects and the solution of the historical defects to obtain a defect analysis model. The model storage module may be used to store a defect analysis model. The defect analysis module can acquire stack information of the target defect, and predicts and outputs a solution corresponding to the target defect according to the stack information of the target defect through the defect analysis model. The maintenance module can collect operation maintenance information of each module, perform operation maintenance monitoring and timely find out the problem of system operation.
Optionally, the online defect library data crawling module may also be referred to as Info Spider, the incremental defect data crawling module may also be referred to as Incre Spider, the Model training module may also be referred to as Model tracker, the defect analysis module may also be referred to as Root cause Analysis, the Model storage module may also be referred to as Model data, and the maintenance module may also be referred to as Agent.
Optionally, the database defect analysis system may include four implementation layers, respectively: a storage layer, a service realization layer, an API layer and a user interface layer. The Storage layer (Storage level) may be referred to as a data Storage layer, the Service implementation layer (Service level) may be referred to as an implementation layer, the API layer (API level) may be referred to as an interface layer, and the user interface layer (Portal level) may be referred to as a user interface.
Optionally, the service implementation layer may include: the system comprises an online defect library data crawling module, an incremental defect data crawling module, a model training module, a defect analysis module and the like. The storage layer may include: a local database module, a model storage module and the like.
In an exemplary embodiment, the embodiment of the present disclosure further provides a database defect analysis apparatus, which may be used to implement the database defect analysis method as in the foregoing embodiment. Fig. 7 is a schematic diagram of a database defect analysis device according to an embodiment of the disclosure. As shown in fig. 7, the apparatus may include: an acquisition unit 701 and an output unit 702.
An acquiring unit 701, configured to acquire stack information of the target defect in the database.
And an output unit 702, configured to input the stack information of the target defect into a preset defect analysis model, and predict and output a solution corresponding to the target defect according to the stack information of the target defect through the defect analysis model.
Optionally, the output unit 702 is further configured to: and predicting and outputting a defect identifier corresponding to the target defect according to the stack information of the target defect through the defect analysis model.
Optionally, the solution of the target defect comprises at least one of the following: the analysis result of the stack information of the target defect, the problem description and problem reproduction method of other users related to the target defect, the history version information of the target defect, and the history version information of the target defect to be solved.
Optionally, before the stack information of the target defect is input into a preset defect analysis model, the output unit 702 is further configured to: segmenting the stack information of the target defect to obtain at least one keyword corresponding to the stack information of the target defect; the output unit 702 is specifically configured to: and inputting at least one keyword corresponding to the stack information of the target defect into the defect analysis model.
In an exemplary embodiment, the embodiment of the present disclosure further provides a model training apparatus, which may be used to implement the model training method as in the foregoing embodiment. Fig. 8 is a schematic diagram of the composition of a model training device according to an embodiment of the disclosure. As shown in fig. 8, the apparatus may include: extraction unit 801, training unit 802.
An extracting unit 801, configured to obtain stack information of a history defect in a database, and a solution of the history defect.
And a training unit 802, configured to train the neural network by using the stack information of the historical defect and the solution of the historical defect, so as to obtain a defect analysis model, where the defect analysis model has a function of predicting and outputting the solution corresponding to the target defect according to the input stack information of the target defect.
Optionally, the history defects include an online history defect and a local history defect, the online history defect is obtained from an online history defect library, and the local history defect is preset by a user.
Optionally, the apparatus further comprises: an updating unit 803 for acquiring an incremental history defect in response to an operation of updating the online history defect; updating the online historical defects according to the increment historical defects to obtain updated online historical defects; the training unit 802 is specifically configured to train the neural network by using the stack information of the updated online historical defect and the updated solution of the online historical defect, to obtain a defect analysis model.
Optionally, the updating unit 803 is further configured to: receiving a local increment history defect input by a user; updating the local historical defects according to the local increment historical defects to obtain updated local historical defects; the training unit 802 is specifically configured to train the neural network to obtain a defect analysis model by using the updated stack information of the local history defect and the updated solution of the local history defect.
Optionally, before training the neural network to obtain the defect analysis model by using the stack information of the historical defect and the solution of the historical defect, the training unit 802 is further configured to: according to a preset character string database, stack information of the history defect and a solution of the history defect are obtained; the training unit 802 is specifically configured to: and training the neural network by adopting the stack information of the history defect after cleaning and the solution of the history defect to obtain a defect analysis model.
Optionally, the apparatus further comprises: a response unit 804, configured to receive character string information input by a user; and updating the preset character string database according to the character string information.
Optionally, the character string information includes at least one of: nonsensical character strings, special character strings, misleading character strings.
In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, a computer program product.
In an exemplary embodiment, an electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in the above embodiments.
In an exemplary embodiment, the readable storage medium may be a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method according to the above embodiment.
In an exemplary embodiment, the computer program product comprises a computer program which, when executed by a processor, implements the method according to the above embodiments.
Fig. 9 shows a schematic block diagram of an example electronic device 900 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 9, the electronic device 900 includes a computing unit 901 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other by a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.
A number of components in the electronic device 900 are connected to the I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, or the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 908 such as a magnetic disk, an optical disk, or the like; and a communication unit 909 such as a network card, modem, wireless communication transceiver, or the like. The communication unit 909 allows the electronic device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunications networks.
The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the respective methods and processes described above, such as database defect analysis and model training methods. For example, in some embodiments, the database defect analysis and model training method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 900 via the ROM 902 and/or the communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the database defect analysis and model training method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the database defect analysis and model training methods in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims (25)

1. A database defect analysis method, the method comprising:
stack information of target defects in a database is obtained;
inputting the stack information of the target defect into a preset defect analysis model, and predicting and outputting a solution corresponding to the target defect according to the stack information of the target defect through the defect analysis model.
2. The method of claim 1, the method further comprising:
And predicting and outputting a defect identifier corresponding to the target defect according to the stack information of the target defect through the defect analysis model.
3. The method according to claim 1 or 2, the solution of the target defect comprising at least one of the following: the analysis result of the stack information of the target defect, the problem description and problem reproduction method of other users related to the target defect, the history version information of the target defect, and the history version information of the target defect to be solved.
4. A method according to any of claims 1-3, said method further comprising, prior to said entering stack information of said target defect into a pre-set defect analysis model:
segmenting the stack information of the target defect to obtain at least one keyword corresponding to the stack information of the target defect;
the step of inputting the stack information of the target defect into a preset defect analysis model comprises the following steps:
and inputting at least one keyword corresponding to the stack information of the target defect into the defect analysis model.
5. A method of model training, the method comprising:
stack information of historical defects in a database is obtained, and a solution of the historical defects is obtained;
Training a neural network by adopting the stack information of the historical defects and the solution of the historical defects to obtain a defect analysis model, wherein the defect analysis model has the function of predicting and outputting the solution corresponding to the target defects according to the input stack information of the target defects.
6. The method of claim 5, the historical defects comprising online historical defects and local historical defects, the online historical defects being obtained from an online historical defect library, the local historical defects being user-preset.
7. The method of claim 6, the method further comprising:
responding to the operation of updating the online historical defect, and acquiring an incremental historical defect;
updating the online historical defects according to the increment historical defects to obtain updated online historical defects;
training the neural network by adopting the stack information of the historical defect and the solution of the historical defect to obtain a defect analysis model, wherein the training comprises the following steps:
and training the neural network by adopting the stack information of the updated online historical defects and the updated online historical defect solution to obtain a defect analysis model.
8. The method of claim 6 or 7, the method further comprising:
receiving a local increment history defect input by a user;
updating the local historical defects according to the local increment historical defects to obtain updated local historical defects;
training the neural network by adopting the stack information of the historical defect and the solution of the historical defect to obtain a defect analysis model, wherein the training comprises the following steps:
and training the neural network by adopting the stack information of the updated local historical defects and the updated solution of the local historical defects to obtain a defect analysis model.
9. The method according to any of claims 5-8, wherein the training of the neural network using the stack information of the historical defect and the solution of the historical defect, before obtaining the defect analysis model, further comprises:
according to a preset character string database, carrying out data cleaning on the stack information of the historical defects and the solution of the historical defects to obtain stack information of the cleaned historical defects and the solution of the historical defects;
training the neural network by adopting the stack information of the historical defect and the solution of the historical defect to obtain a defect analysis model, wherein the training comprises the following steps:
And training the neural network by adopting the stack information of the history defect after cleaning and the solution of the history defect to obtain a defect analysis model.
10. The method of claim 9, the method further comprising:
receiving character string information input by a user;
and updating the preset character string database according to the character string information.
11. The method of claim 10, the string information comprising at least one of: nonsensical character strings, special character strings, misleading character strings.
12. A database defect analysis apparatus, the apparatus comprising:
the acquisition unit is used for acquiring stack information of the target defects in the database;
and the output unit is used for inputting the stack information of the target defect into a preset defect analysis model, and predicting and outputting a solution corresponding to the target defect according to the stack information of the target defect through the defect analysis model.
13. The apparatus of claim 12, the output unit further to:
and predicting and outputting a defect identifier corresponding to the target defect according to the stack information of the target defect through the defect analysis model.
14. The apparatus of claim 12 or 13, the solution of the target defect comprising at least one of: the analysis result of the stack information of the target defect, the problem description and problem reproduction method of other users related to the target defect, the history version information of the target defect, and the history version information of the target defect to be solved.
15. The apparatus according to any one of claims 12-14, wherein the output unit, before inputting the stack information of the target defect into a preset defect analysis model, is further configured to: segmenting the stack information of the target defect to obtain at least one keyword corresponding to the stack information of the target defect;
the output unit is specifically configured to:
and inputting at least one keyword corresponding to the stack information of the target defect into the defect analysis model.
16. A model training apparatus, the apparatus comprising:
the extraction unit is used for acquiring stack information of the historical defects in the database and a solution of the historical defects;
the training unit is used for training the neural network by adopting the stack information of the historical defects and the solutions of the historical defects to obtain a defect analysis model, and the defect analysis model has the function of predicting and outputting the solutions corresponding to the target defects according to the input stack information of the target defects.
17. The apparatus of claim 16, the historical defects comprising online historical defects and local historical defects, the online historical defects being obtained from an online historical defect library, the local historical defects being user-preset.
18. The apparatus of claim 17, the apparatus further comprising:
an updating unit for acquiring an incremental history defect in response to an operation of updating the online history defect; updating the online historical defects according to the increment historical defects to obtain updated online historical defects;
the training unit is specifically configured to train the neural network by using the stack information of the updated online historical defect and the updated solution of the online historical defect, so as to obtain a defect analysis model.
19. The apparatus according to claim 17 or 18, the updating unit further configured to:
receiving a local increment history defect input by a user;
updating the local historical defects according to the local increment historical defects to obtain updated local historical defects;
the training unit is specifically configured to train the neural network by using the updated stack information of the local historical defect and the updated solution of the local historical defect, so as to obtain a defect analysis model.
20. The apparatus according to any of claims 16-19, wherein the training unit is further configured to, before training the neural network to obtain the defect analysis model using the stack information of the historical defect and the solution of the historical defect:
according to a preset character string database, stack information of the history defect and a solution of the history defect are obtained;
the training unit is specifically configured to:
and training the neural network by adopting the stack information of the history defect after cleaning and the solution of the history defect to obtain a defect analysis model.
21. The apparatus of any of claims 20, the apparatus further comprising:
the response unit is used for receiving character string information input by a user;
and updating the preset character string database according to the character string information.
22. The apparatus of claim 21, the string information comprising at least one of: nonsensical character strings, special character strings, misleading character strings.
23. An electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor;
Wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4 or any one of claims 5-11.
24. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1-4 or any one of claims 5-11.
25. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-4, or any one of claims 5-11.
CN202311497857.4A 2023-11-10 2023-11-10 Database defect analysis method, device, equipment and storage medium Pending CN117453485A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311497857.4A CN117453485A (en) 2023-11-10 2023-11-10 Database defect analysis method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311497857.4A CN117453485A (en) 2023-11-10 2023-11-10 Database defect analysis method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN117453485A true CN117453485A (en) 2024-01-26

Family

ID=89588959

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311497857.4A Pending CN117453485A (en) 2023-11-10 2023-11-10 Database defect analysis method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117453485A (en)

Similar Documents

Publication Publication Date Title
US11604782B2 (en) Systems and methods for scheduling concurrent summarization of indexed data
US11194779B2 (en) Generating an index for a table in a database background
KR102485179B1 (en) Method, device, electronic device and computer storage medium for determining description information
US9940581B2 (en) Ontology-aided business rule extraction using machine learning
US11366840B2 (en) Log-aided automatic query expansion approach based on topic modeling
US11681282B2 (en) Systems and methods for determining relationships between defects
US11599539B2 (en) Column lineage and metadata propagation
US10628250B2 (en) Search for information related to an incident
EP3916584A1 (en) Information processing method and apparatus, electronic device and storage medium
CN113128209B (en) Method and device for generating word stock
CN114091426A (en) Method and device for processing field data in data warehouse
EP3118807A1 (en) Prioritizing and planning issues in automation
US10534788B2 (en) Automatically determining a recommended set of actions from operational data
US10257055B2 (en) Search for a ticket relevant to a current ticket
CN114491232B (en) Information query method and device, electronic equipment and storage medium
CN117453485A (en) Database defect analysis method, device, equipment and storage medium
GB2603594A (en) Maintenance of a data glossary
CN112860652A (en) Operation state prediction method and device and electronic equipment
US11947958B2 (en) Method, device, and program product for managing object in software development project
EP3282404A1 (en) System and method for analyzing and prioritizing issues for automation
US11971891B1 (en) Accessing siloed data across disparate locations via a unified metadata graph systems and methods
CN113032581B (en) Method and device for updating product list
US20230143297A1 (en) Production knowledge management system, production knowledge management method, and production knowledge management program
US20230132618A1 (en) Method for denoising click data, electronic device and storage medium
US9501274B1 (en) Qualitative feedback correlator

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination