WO2022013944A1

WO2022013944A1 - Failure analysis assistance device, failure analysis assistance method, and program

Info

Publication number: WO2022013944A1
Application number: PCT/JP2020/027375
Authority: WO
Inventors: 振宇徐
Original assignee: 日本電信電話株式会社
Priority date: 2020-07-14
Filing date: 2020-07-14
Publication date: 2022-01-20

Abstract

This failure analysis assistance device comprises: an analysis unit which analyzes an executable path for the calling relationship between components of a program; a selection unit which, on the basis of a prescribed rule, selects, from among the components included in the path, components to be caused to output information indicating traces of the execution thereof; an altering unit which alters the program so that the components selected by the selection unit output the information; and a generation unit which, on the basis of said path and the component sequence indicated by the information that the program altered by the altering unit outputted until a failure occurred in the program, identifies a sequence of statements for causing such failure, and generates a test case for executing the sequence. Thus, the failure analysis assistance device improves program failure analysis efficiency.

Description

Failure analysis support device, failure analysis support method and program

The present invention relates to a failure analysis support device, a failure analysis support method, and a program.

As a method for reproducing failures and failures that occur in a program, an approach that reproduces based on the user's operation (for example, Non-Patent Document 1) and an approach that reapplies input data to an object (for example, Non-Patent Document 2). There is.

Specifically, the approach of Non-Patent Document 1 automatically reproduces the application crash from the bug report.

The approach of Non-Patent Document 2 utilizes time travel debugging to efficiently and accurately reconstruct the state of an object as a unit test, and uses differential analysis of code coverage data.

In any case, the existing method is an approach that realizes the reproduction of the failure by recording the input (external input data) to the application and reapplying it.

However, since the approach of reproducing the failure by reapplying the input (external input) to the application is indirect, it is difficult to apply it to the commercial environment where the reproducibility of the failure is uncertain.

Therefore, in the existing failure analysis, maintenance personnel, etc. make multiple hypotheses leading to the failure from the information (error messages and logs) indicating the situation of the failure, and try the hypotheses one by one in the verification environment. It has been confirmed whether or not the failure is reproduced. However, such a method causes a large amount of operation.

The present invention has been made in view of the above points, and an object of the present invention is to improve the efficiency of failure analysis of a program.

Therefore, in order to solve the above problem, the failure analysis support device uses an analysis unit that analyzes an executable route and a trace of execution among the components included in the route for the call relationship of the program components. A selection unit that selects a component for outputting the indicated information based on a predetermined rule, a modification unit that modifies the program so that the component selected by the selection unit outputs the information, and the modification unit. Based on the sequence of components indicated by the information output from the program before the occurrence of the failure of the program modified by, and the path, a sequence of statements for causing the failure is specified. It has a generation unit that generates a test case for executing the sequence.

The failure analysis of the program can be streamlined.

It is a figure for demonstrating the whole picture of the method proposed in embodiment of this invention. It is a figure which shows the hardware configuration example of the fault analysis support apparatus 10 in embodiment of this invention. It is a figure which shows the functional structure example of the trouble analysis support apparatus 10 in embodiment of this invention. It is a flowchart for demonstrating an example of the processing procedure which the failure analysis support apparatus 10 executes with respect to Part 1. It is a figure which shows an example of CFG. It is a flowchart for demonstrating an example of the processing procedure which the failure analysis support apparatus 10 executes with respect to Part 2 and Part 3. It is a figure for demonstrating the output level of a log.

In this embodiment, information showing an execution trace of a component of a target application executed before an application program (hereinafter referred to as "target application") fails in a commercial environment is collected, and the collected information is collected. It provides an approach to automatically generate a test case in a verification environment, execute the test case, and finally determine whether or not the same event (conventional log or error message) as at the time of failure appears. By doing so, we will realize an approach that directly reproduces the obstacle.

However, collecting execution information up to the failure imposes a load on the commercial environment, but it is possible that the allowable load limit differs depending on the operation site (commercial environment) of the target application.

Therefore, in the present embodiment, a mechanism is prepared in which the component that outputs the execution trace can be selected from the components of the target application in the verification environment before collecting the information in the commercial environment.

The commercial environment refers to a computer or computer system in which a user who has purchased the target application uses the target application. On the other hand, the verification environment refers to a computer or computer system used by the developer or maintenance person of the target application to analyze the failure of the target application.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram for explaining an overall picture of the method proposed in the embodiment of the present invention. As shown in FIG. 1, the proposed method is composed of three parts, Part 1 to 3.

[Part 1: Recording run-time information]
In the verification environment, the structure of the entire target application is analyzed, and among the components of the target application, the components that output the trace that is the trace of execution (execution trace) (hereinafter referred to as "target")) are narrowed down. A list of targets (hereinafter referred to as "target list") is generated. In addition, as an example of the type of the component of the target application, Class (class), Method (method), Statement (statement) and the like can be mentioned. That is, the component of the target application has a hierarchical structure (grain size). Specifically, the class is the component of the highest level (maximum particle size), and the statement is the component of the lowest level (minimum particle size). Any one of these hierarchies (grain size) (class, method, or statement) is selected as the target. After that, the target application executed in the commercial environment is changed (modified) in the verification environment so that the call sequence (execution trace) between the targets can be acquired using the generated target list. As a result, when the target application is used in the commercial environment, the execution trace of the component corresponding to the target among the components of the target application is acquired (collected).

[Part2: Acquisition of failure path]
After a failure occurs in the target application in the commercial environment, a call graph between targets is generated as a candidate for the failure path for each function of the target application based on the execution trace collected in Part 1 in the verification environment. Here, the function of the target application means, for example, a unit until an output is performed with respect to an input by a user, and is a concept classified by a set of an input and an output. For example, from the viewpoint of the GUI (Graphical User Interface) of the target application, the functions may be distinguished for each menu item.

[Part3: Automatic reproduction and judgment of failure]
A test case that covers the call sequence at the statement level is created using the failure path candidates that are the call graphs between targets generated in Part2. After that, the created test case is used, the test is performed in the verification environment, and the failure is reproduced.

In order to carry out the above, the following inputs are required in the verification environment.
(1) Source code / binary code of the target application (2) Setting of target hierarchy (grain size) (Class / Method / Status) (3) Conventional log at the time of failure (including error message)
(4) Clone of commercial environment (to build verification environment)
The conventional log in (3) is a log output by the target application from the beginning, in addition to the execution trace, and includes an error message at the time of failure.

Further, as the faults that can be dealt with in this embodiment, the faults in which the call sequence (execution trace) is different from the normal time at the statement level, and the faults in which the program behaves the same as in the normal time are included in the troubles that can be dealt with. No. For example, failures when exception handling or functions that would not normally be executed are executed are targeted, and failures such as memory leaks that gradually deplete resources are excluded.

Next, a computer (fault analysis support device 10) that functions as a verification environment in the present embodiment will be described.

FIG. 2 is a diagram showing a hardware configuration example of the failure analysis support device 10 according to the embodiment of the present invention. The failure analysis support device 10 of FIG. 2 has a drive device 100, an auxiliary storage device 102, a memory device 103, a CPU 104, an interface device 105, a display device 106, an input device 107, and the like, which are connected to each other by a bus B, respectively. ..

The program that realizes the processing in the failure analysis support device 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed in the auxiliary storage device 102 from the recording medium 101 via the drive device 100. However, the program does not necessarily have to be installed from the recording medium 101, and may be downloaded from another computer via the network. The auxiliary storage device 102 stores the installed program and also stores necessary files, data, and the like.

The memory device 103 reads a program from the auxiliary storage device 102 and stores it when there is an instruction to start the program. The CPU 104 realizes the function related to the failure analysis support device 10 according to the program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network. The display device 106 displays a GUI (Graphical User Interface) or the like by a program. The input device 107 is composed of a keyboard, a mouse, and the like, and is used for inputting various operation instructions.

FIG. 3 is a diagram showing a functional configuration example of the failure analysis support device 10 according to the embodiment of the present invention. In FIG. 3, the failure analysis support device 10 includes a code analysis unit 11, a target selection unit 12, a modification unit 13, a failure path candidate identification unit 14, a test case generation unit 15, a verification unit 16, and the like. Each of these parts is realized by a process of causing the CPU 104 to execute one or more programs installed in the failure analysis support device 10.

Hereinafter, the processing procedure executed by the failure analysis support device 10 will be described. FIG. 4 is a flowchart for explaining an example of a processing procedure executed by the failure analysis support device 10 with respect to Part 1.

In step S101, the code analysis unit 11 inputs the source code or binary code of the target application (hereinafter referred to as “target application code”) in the verification environment, and calls the components of the target application based on the target application code. Regarding the relationship, the executable route is analyzed, and a graph (CFG (Control Flow Graph): call graph) showing the route is generated for each function of the target application.

FIG. 5 is a diagram showing an example of CFG. Each node in the CFG shown in FIG. 5 corresponds to a component corresponding to the target. That is, when "Class" is specified as the target hierarchy, each node corresponds to the class constituting the target application. When "Method" is specified as the target hierarchy, each node corresponds to the method of each class. When "Statement" is specified as the target hierarchy, each node corresponds to the statement that constitutes each method. In addition, L0 to 3 described on the left side of FIG. 5 indicate the hierarchy in the depth direction of CFG.

CFG can be generated using known technology. In the present embodiment, a case where "Method" is specified in advance by the user as the target hierarchy will be described. In this case, for example, the code analysis unit 11 analyzes the code line by line for each file constituting the target application code, and each time a method call is found, the node corresponding to the called method is newly added to the CFG. Add and add an edge to the CFG that has a direction from the node corresponding to the calling method to the node corresponding to the called method. If the target hierarchy is a statement, a node is created for each statement. If the target hierarchy is a class, a node will be created for each class. In this case, for example, every time the code analysis unit 11 finds a call to a method of another class from within a method of one class, it adds a node of the called class to the CFG and corresponds to the calling class. Add an edge to the CFG that has a direction from the node to the node corresponding to the called class.

Note that the code analysis unit 11 generates two files as files for storing information indicating the CFG along with the generation of the CFG.

The first file is a file for storing a list of target names (hereinafter, referred to as "node list") corresponding to each node of CFG. In the present embodiment, the file name of the file is "csv.txt". Specific examples of the contents of the file are as follows.
['Aa','bb', ...,'ff' ...]
Here, each of'aa','bb', ..., and'ff'is a target name. When the target hierarchy is a method, the target name is a method. If the target hierarchy is a class, the target name is a class. If the target hierarchy is a statement, the target name is a statement.

The second file is a file for storing a list (hereinafter referred to as "adjacency list") of adjacency relations (hereinafter referred to as "adjacency list") of each node of CFG (when the target particle size is a method, the method call relations). In the present embodiment, the file name of the file is "cfg.txt". Specific examples of the contents of the file are as follows.
[[0, [1,2,3]], [2, [3,4]], ...]
Each numerical value in the adjacency list (hereinafter referred to as "node number") corresponds to the order of method names in the node list, and the beginning of this order is 0 (that is, 0 origin). For example, [0, [1, 2, 3]] indicates that the 0th method in the node list calls the 1st, 2nd, and 3rd methods in the node list.

Note that the node list and adjacency list are generated for each CFG. In addition, CFG is generated for each function of the target application. Therefore, the node list and the adjacency list are generated for each function of the target application. The CFGs that support different functions have different root nodes. FIG. 3 shows an example in which CFGs (cfg_A, cfg_B, cfg_C, cfg_D) are generated for each of the functions A, B, C and D.

Subsequently, the target selection unit 12 accepts the input of the monitoring target function (hereinafter referred to as “target function”) and the trace pattern among the functions of the target application from the user (user in the verification environment) (S102). For example, according to the example of FIG. 3, any one or more of the functions A, B, C, and D is selected as the target function. Only some functions may be selected as target functions, or all functions may be selected as target functions. A CFG is generated for each function. Therefore, the target selection unit 12 may display the GUI with the generated CFG as an option on the display device 106, and may set the function related to the CFG corresponding to the option selected by the user as the target function.

Further, the trace pattern refers to a rule composed of a combination of a selection criterion of a node to be a target (trace target) among the nodes constituting the CFG of the target function and information output to an execution trace regarding the target. .. In the present embodiment, it is assumed that any of the following four trace patterns A to D can be selected.

The A pattern is a trace pattern in which nodes in every other layer in the depth direction of the CFG (that is, nodes that are not adjacent in the CFG) are selected as targets, and the date and time when the target is executed and the target name are included in the output information. .. In the case of the A pattern, in the CFG of FIG. 5, the node surrounded by the thick line is targeted. The A pattern has a feature that the recording load of the execution trace is small and the reproduction time can be suppressed.

The B pattern is a trace pattern in which the branch destination node of each branch in CFG is selected as a target, and the date and time when the target is executed and the target name are included in the output information. The B pattern has a feature that the recording load of the execution trace is relatively large, but the reproduction time is small.

The C pattern is a trace pattern in which the target is modified based on the execution frequency after the A pattern is applied for a certain period of time. For example, when the A pattern is applied, the target with high execution frequency may be excluded, and the nodes before and after the target with low execution frequency may be added to the target. The C pattern has a feature that the recording load of the execution trace can be further reduced and the reproduction time can be further suppressed.

The D pattern is a trace pattern in which the target argument and the return value are further added to the output information when the A pattern, the B pattern, or the C pattern is applied. The D pattern has a feature that the recording load of the execution trace is large, but the reproduction time can be reduced.

Subsequently, the target selection unit 12 selects a target node from each node of the CFG of the target function based on the trace pattern specified by the user, and a list of the target names of the selected node (that is, that is). "Target list") is generated (S103). Here, as an example of the trace pattern, an example in which the A pattern is specified will be described.

Specifically, when the A pattern is specified, the target selection unit 12 uses the cfg. Of the target function. Depth-first search processing is performed on the adjacency list stored in txt, node numbers with even-numbered depths are selected, and csv. Of the target function. In the node list stored in txt, a list of target names (for example, method names) corresponding to the selected node number is generated. Therefore, for example, if the CFG in FIG. 5 is the CFG of the target function, a list of target names surrounded by a thick line is generated. That is, non-adjacent nodes are selected as targets in CFG.

The code analysis unit 11 always generates a CFG having a target (statement in the present embodiment) of the lowest layer (minimum particle size) as a node regardless of the designation of the target layer (particle size). You may. In this case, the target selection unit 12 may change (aggregate) the CFG so as to correspond to the designated target hierarchy, and generate a target list based on the changed CFG. That is, by aggregating the nodes of the CFG of the lowest layer, the CFG of another layer can be generated based on the known technology.

Subsequently, the modification unit 13 outputs the target application so that the execution trace of the target is output when each target (for example, a method) whose target name is included in the target list generated by the target selection unit 12 is called. Generate a setting file for modification (S104).

Subsequently, the modification unit 13 applies the setting file to the source code of the target application so that when each target whose target list includes the target name is called, the execution trace of the target is output. The source code is modified to generate a modified binary code based on the modified source code (S105).

Steps S104 and S105 can be realized by using, for example, the mechanism of AspectJ that creates and uses a script that embeds a print statement. If the target hierarchy is a method, the print statement is modified to be called at the beginning of the method.

The execution trace should include the time when the target was called, the target name, and so on.

After that, the modified version of the target application is applied to the commercial environment. As a result, depending on the user's use of the target application in the commercial environment, the target application outputs the execution time information such as the conventional log and the sequence of the execution traces to a predetermined file. When a failure occurs in the target application (hereinafter referred to as "target failure") in the commercial environment, the verification environment acquires the run-time information output from the target application before the target failure occurs. The acquisition of run-time information may be performed by any method. For example, the target application may automatically upload the run-time information to the verification environment, or the run-time information may be manually sent from the commercial environment to the verification environment. Further, the run-time information acquired by the verification environment may be limited to the information for a predetermined period before the occurrence of the target failure.

FIG. 6 is a flowchart for explaining an example of the processing procedure executed by the failure analysis support device 10 with respect to Part 2 and Part 3.

In step S201, the failure analysis support device 10 receives input information from the user in the verification environment. The input information is a series of conventional logs and execution traces acquired from a commercial environment, a failure occurrence date and time specified by a user based on the conventional logs, and the like. For example, the recording time of the error message corresponding to the target failure, which is included in the conventional log, is specified as the failure occurrence date and time.

Subsequently, the failure path candidate identification unit 14 generates failure path candidates based on the sequence of execution traces included in the input information and the failure occurrence date and time (S202). The failure path is a sequence (execution order) of targets (for example, methods) executed (called) by the failure occurrence date and time, which is specified based on a series of execution traces.

Specifically, the failure path candidate identification unit 14 has a call relationship at the target level based on the sequence of execution traces up to the failure occurrence date and time and the trace pattern specified by the user for each CFG of each function of the target application. Generate failure path candidates that indicate. For example, when the trace pattern specified by the user is the A pattern and the sequence of the execution traces is "aa, dd", the sequence of the execution traces is the execution traces of every other layer in the depth direction of the CFG. It should be a series of (ie, a series of discontinuous execution traces). Therefore, for example, for the CFG (csv. Txt, cfg. Txt) in FIG. 5, the following three sequences are generated as failure path candidates.
aa → bb → dd
aa → bb → dd → ee
aa → bb → dd → ff
That is, for a certain CFG, a path that can be taken based on the execution trace is generated as a candidate for a failure path. Therefore, failure path candidates include the target names of targets that may have been executed between the targets included in the sequence of execution traces, based on the CFG.

In the present embodiment, for convenience of explanation, an example has been described in which the execution trace includes information about the call (call), but does not include the return information from the call (return) in order to suppress the information output load. .. This is because if the CFG and the call information are combined, the information to be returned is unnecessary in most cases.

For this reason, complicated sequences such as reciprocating are not included in the above example of failure path candidates. For example, in the above example, there may be a sequence such as aa → gg → bb → dd, but the sequence is omitted from the above example.

If there is a source code or binary code of the target application, it is possible to determine a complicated call that goes back and forth depending on the order between methods and the dependency between methods that are not included in CFG. Therefore, the failure path candidate identification unit 14 may also refer to the source code or binary code of the target application to generate failure path candidates.

Subsequently, the test case generation unit 15 identifies all possible sequences (routes) at the statement level for each failure path candidate, and tests for reproducing the sequence for each specified sequence. Generate a case (S203). For example, if the target hierarchy is a method, the failure path is a sequence of methods. In one method, there may be multiple sequences (routes) at the statement level due to branching or the like. For example, in a failure path showing a sequence of two methods, if there are three statement-level sequences in one method and two statement-level sequences in the other method, the failure path is 3 × 2 = 6 statement-level sequences are specified. Therefore, six test cases are generated for the failure path. The content of the test case is a group of instructions for executing the sequence of the specified statements.

If the CFG is generated with a hierarchy (method or class) larger than the statement as a unit (node), the test case generator 15 will use the failure path based on the source code or binary code of the target application. You just need to identify the statement-level sequence that can be executed in.

Subsequently, the verification unit 16 executes the loop process L1 including steps S204 to S206 for each test case included in the generated test case group. Hereinafter, the test case targeted for processing in the loop processing is referred to as a "target test case".

In step S204, the verification unit 16 executes the target test case for the clone of the commercial environment constructed in the verification environment. Subsequently, the verification unit 16 determines whether or not the same phenomenon as the target failure has occurred as a result of executing the target test case (S205). Such a judgment is made by the conventional log output from the target application by executing the target test case (hereinafter referred to as "verification log") and the conventional log included in the input information (hereinafter referred to as "commercial log"). It can be done by comparing with. For example, if the verification log and the portion of the commercial log corresponding to the target test case match, it may be determined that the same event as the target failure has occurred. Here, the match may include, for example, different states for parameters that change depending on the execution timing and the execution user (login user). Further, it may be determined that the same phenomenon as the target failure has occurred because the verification log contains the same error message as the error message corresponding to the target failure.

When the same phenomenon as the target failure occurs (Yes in S205), the verification unit 16 outputs the target test case and the like (S206). For example, the file containing the target test case may be saved in a predetermined location (folder or the like) in the auxiliary storage device 102.

A user in the verification environment (for example, a maintenance person, etc.) can reproduce the target failure by executing the target test case (that is, the statement level failure path).

As described above, according to the present embodiment, it is possible to generate a test case for reproducing the failure of the target application based on the execution trace of the target application. Therefore, it is possible to reduce the dependence of the external input on the program in reproducing the failure of the target application, and it is possible to improve the efficiency of the failure analysis of the program of the target application or the like.

Note that Non-Patent Document 3 discloses a commercially available product "log4j" that sets and reduces the trace level. "Log4j" is a log API for the Java (registered trademark) program under development in the Jakarta project. If you set the log output level in the configuration file, all logs with higher levels will be output. As for the log output level, the arrangement as shown in FIG. 7 is common.

With "log4j", you can set the log output level. In order to realize log output, the developer needs to select the code to be monitored and add the content to be output as a log and the output timing (output level of the log) to the code to be monitored.

On the other hand, in this embodiment, the trace level can be set by the hierarchy (class, method, statement) of the components of the target application. Further, it is not necessary to select the log output location on the code to be monitored, and it is not necessary to set the content and timing to be output one by one.

In this embodiment, it may be applied to a program other than the application program.

In the present embodiment, the target application is an example of a predetermined program. The code analysis unit 11 is an example of an analysis unit. The target selection unit 12 is an example of the selection unit. The failure path candidate specifying unit 14 is an example of the specific unit. The test case generation unit 15 is an example of the generation unit. The verification unit 16 is an example of a determination unit.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications are made within the scope of the gist of the present invention described in the claims.・ Can be changed.

10 Failure analysis support device 11 Code analysis unit 12 Target selection unit 13 Modification unit 14 Failure path candidate identification unit 15 Test case generation unit 16 Verification unit 100 Drive device 101 Recording medium 102 Auxiliary storage device 103 Memory device 104 CPU
105 Interface device 106 Display device 107 Input device B Bus

Claims

An analysis unit that analyzes the feasible route for the call relationship of the program components,
Among the components included in the path, a selection unit for selecting a component for outputting information indicating an executed trace based on a predetermined rule, and a selection unit.
A modification unit that modifies the program so that the component selected by the selection unit outputs the information.
Based on the sequence of the components indicated by the information output from the program before the occurrence of the failure of the program modified by the modification unit and the path, a sequence of statements for causing the failure is obtained. A generator that identifies and generates a test case to execute the sequence,
A fault analysis support device characterized by having.
The components have a hierarchical structure
The analysis unit analyzes the feasible route for the call relationship of the components of the hierarchy specified by the user.
The fault analysis support device according to claim 1, wherein the fault analysis support device is characterized by the above.
The selection unit selects, among the components included in the path, the components that are not adjacent to each other in the path.
The component that may have been executed between the components included in the sequence of the components indicated by the information output from the program before the occurrence of the failure of the program modified by the modification unit. It has a specific part to be specified based on the above route, and has a specific part.
The generator generates the test case based on the sequence including the component identified by the particular unit.
The failure analysis support device according to claim 1 or 2, wherein the fault analysis support device is characterized by the above.
A determination unit that executes the test case generated by the generation unit and determines whether or not the same event as the failure has occurred due to the execution of the test case.
The fault analysis support device according to any one of claims 1 to 3, wherein the fault analysis support device is characterized by having the above.
The determination unit outputs the test case in which the same event as the failure has occurred.
The fault analysis support device according to claim 4, wherein the fault analysis support device is characterized by the above.
An analysis procedure that analyzes the feasible route for the call relationship of the program components, and
Among the components included in the path, a selection procedure for selecting a component for outputting information indicating an executed trace based on a predetermined rule, and a selection procedure.
A modification procedure for modifying the program so that the component selected by the selection procedure outputs the information.
Based on the sequence of the components indicated by the information output from the program and the route before the failure of the program modified by the modification procedure, a sequence of statements for causing the failure is obtained. A generation procedure to identify and generate a test case to execute the sequence,
A failure analysis support method characterized by a computer executing.
A program characterized by operating a computer as the failure analysis support device according to any one of claims 1 to 5.