JP5067705B2 - Abnormal test support device, abnormal test support method, and program - Google Patents

Description

  The present invention relates to an abnormal system test support apparatus, and more particularly to an abnormal system test support apparatus that tests an error process of a program written in C language or the like.

  In software development, the ratio of test man-hours to the total man-hours for development is very large.

  In particular, it is difficult to test errors that occur depending on the execution environment, such as network errors, memory shortages, and file access errors, which are part of abnormal systems, without an actual environment. As a countermeasure, a test using a stub code that artificially generates an error is complicated to switch between an actual code and a stub code, and requires a lot of man-hours.

  A stub code is a program code that acts as an intermediary when a program calls another program. For example, when using a system call, a device driver, or the like from a user program, it is used to simplify a complicated interface.

  Japanese Patent Laid-Open No. 5-257740 (Patent Document 1) discloses a technique related to a “unit test execution method”. This prior art describes a method for dynamically replacing an object during program execution. By using this technology, it is possible to switch between a function to be called during program execution and its stub code. However, each time the stub code is switched, the user has to input something, and there is no particular description about automatic switching.

  Japanese Patent Laid-Open No. 2005-338987 (Patent Document 2) discloses a technique related to “exception test support program and exception test support device”. In this conventional technique, a source code is analyzed, a driver that satisfies the coverage of the source code is created, and a test that switches between a function that generates an exception and a stub code is performed, and an exception test is automatically executed. In this prior art, since all functions that generate exceptions are switched to stub code, exceptions can be generated only at a specific timing. For example, as can be seen by referring to the correspondence relationship between the function and the exception output in Patent Document 2, in this conventional technique, the same function can output an exception only at one location. In addition, since errors such as file access and network errors occur at arbitrary timings, pseudo errors must also be generated at arbitrary timings.

  Japanese Patent Laid-Open No. 2000-181749 (Patent Document 3) discloses a technique relating to “software error processing test system, test method, and test program recording medium”. In this prior art, after instructing the execution of the test target program, the calling state of the application interface unit of the operating system by the test target program is monitored, and the state is stored in the storage device. One of the stored calls is crafted to cause an error in the application interface unit. After execution instruction to determine the error processing contents of the test target program executed after the setting, one of the calls in the stored call status is selected in the execution order, error setting for the call, execution of the test target program The instruction and determination of the error processing contents in the execution are sequentially performed for each call in the call status, and the determination result is output. As a result, an error processing test of the program is automatically executed. This conventional technique is intended for an abnormal test that generates an exception at a predetermined timing, and cannot perform an exception test at an arbitrary timing.

  Japanese Patent Application Laid-Open No. 3-010436 (Patent Document 4) discloses a technique relating to “anomaly processing classification method in an electronic mail system”. In this conventional technique, not only one pattern of abnormal processing is mounted, but all patterns are mounted by having information as a basis for selecting a pattern and dynamically changing the information. This allows the user to set the process when an abnormality occurs.

JP-A-5-257740 JP 2005-338987 A JP 2000-181749 A JP-A-3-010436

  An object of the present invention is to comprehensively create a test pattern that generates an error that depends on the environment at the time of occurrence at a timing when the error may occur, and convert the function to a stub function at a timing according to the created test pattern. It is to provide an abnormal system test support device that automatically performs error processing tests by replacing them.

  In the following, means for solving the problem will be described using the numbers used in [Best Mode for Carrying Out the Invention] in parentheses. These numbers are added to clarify the correspondence between the description of [Claims] and [Best Mode for Carrying Out the Invention]. However, these numbers should not be used to interpret the technical scope of the invention described in [Claims].

  The abnormal system test support apparatus of the present invention reads and executes a source file (1) to be tested, and executes a test pattern template creation processing unit (5) for creating a test pattern template file from the order of function calls at the time of execution; When the stub file (3) and test pattern template file corresponding to the source file (1) are read, the full search algorithm is executed, and the function described in the test pattern template file exists in the stub file (3) The test pattern creation processing unit (7) for creating a test pattern file including all test patterns obtained by the full search from the test pattern template file, the source file (1) and the stub file (3) are read and executed, Dynamic switching between standard and stub functions, included in test pattern file Comprising test execution processor for executing the test along a test pattern and (9).

  The test pattern template creation processing unit (5) builds the source file (1), performs step execution, and writes a function name to the test pattern template file when there is a function in the steps until the execution step ends.

  The test pattern creation processing unit (7) creates a test pattern file in which a switching flag is added to the function for which the stub file (3) is prepared among the functions described in the test pattern template file. The test execution processing unit (9) builds the source file (1) and the stub file (3) and executes the steps. If there is a function in the steps up to the final step, the test execution file is read in order. When the value of the target function switching flag is 0 or empty, the function is executed as it is. When the value of the target function switching flag is 1, the stub function is executed.

  The test execution processing unit (9) deletes test patterns having the same function switching flag.

  In other words, the abnormal test support apparatus of the present invention includes a test pattern creation processing unit (7) that comprehensively creates test patterns that generate errors depending on the environment at the time of occurrence at a timing that may occur. A test execution processing unit (9) that automatically replaces a function with a stub function at a timing according to the created test pattern and automatically performs an error processing test.

  The abnormal system test support method of the present invention includes a step of executing a source file (1) to be tested and creating a test pattern template file from a function call sequence at the time of execution, and a stub file corresponding to the source file (1) (3) When the test pattern template file is read, the full search algorithm is executed, and the function described in the test pattern template file exists in the stub file (3), it is obtained from the test pattern template file by a full search. Create a test pattern file that includes all the test patterns that are created, build and execute the source file (1) and stub file (3), dynamically switch between the standard function and the stub function, and include in the test pattern file Performing a test according to a test pattern .

  In the abnormal system test support method of the present invention, the source file (1) is built and executed in an environment where step execution is possible, the step of checking whether a function exists in the execution step, and the function exists in the execution step In this case, the method further includes a step of writing the function name in the test pattern template file, a step of determining whether or not the final step has been reached, and a step of saving the test pattern template file if the final step has been reached.

  The abnormal test support method of the present invention includes a step of reading a test pattern template file in order to the end of the file, and a function in the test pattern template file described in the test pattern template file when it exists in the stub file (3). The test pattern file in which the switch flag is added to the function for which the stub file (3) is prepared, the step of determining whether the end of the file has been reached, and the end of the file are reached In this case, the method further includes a step of storing the created test pattern file.

The abnormal system test support method of the present invention builds a source file (1) and a stub file (3), performs step execution, and builds the stub file (3). Prefixing the stub functions so that they are not the same,
A step for checking whether a function exists in the execution step, a step for reading the test pattern file in order when a function exists in the step, obtaining the value of the target function switching flag, and a target function switching flag If the value of the target function switching flag is 0, or the target function switching flag is empty (no stub function), the function is left as is. And a step of changing the control to the stub function and executing the stub function when the value of the target function switching flag is 1.

  The abnormal system test support method of the present invention, when executing a test according to a test pattern, the step of storing the executed function, the step of checking whether there is a function that is not executed among the executed test pattern, And a step of deleting a test pattern having the same function switching flag when there is a function that has not been executed.

  In other words, the abnormal system test support method according to the present invention includes a step of comprehensively creating a test pattern for generating an error depending on an environment at the time of execution at a timing that may occur, and the created test pattern. And replacing the function with a stub function at the same timing, and automatically performing an error processing test.

  The program of the present invention causes a computer to execute one of the abnormal system test support methods described above.

  Abnormal test can be performed more efficiently and comprehensively. The reason is that a test pattern file can be created automatically and comprehensively, and error processing can be generated in a pseudo manner at an arbitrary timing.

Hereinafter, a first embodiment of the present invention will be described with reference to the accompanying drawings.
Referring to FIG. 1, the abnormal system test support apparatus of the present invention includes a source file 1, a source file storage unit 2, a stub file 3, a stub file storage unit 4, a test pattern template creation processing unit 5, and a test. A pattern template file storage unit 6, a test pattern creation processing unit 7, a test pattern file storage unit 8, a test execution processing unit 9, and a test result file 10 are provided.

  The source file 1 is a source file in which source code to be tested is described. Here, a source file of a program written in C language or the like is shown. The source file 1 may be input from an input device, or may be acquired from a server on the network via a communication device.

  The source file storage unit 2 is a storage device that stores information. Here, the source file 1 is stored.

  The stub file 3 is a system file indicating the metadata of the source file to be tested and the destination. Here, a stub function corresponding to the function in the source file to be tested is described. The stub file 3 may be input from an input device, or may be acquired from a server on the network via a communication device.

  The stub file storage unit 4 is a storage device for storing information. Here, the stub file 3 is stored.

  The test pattern template creation processing unit 5 is a data processing device that operates under program control. An environment that allows step execution of source files to be tested is provided, and a test pattern template file is created from the order of function calls during execution.

  The test pattern template file storage unit 6 is a storage device that stores information. Here, the test pattern template file created by the test pattern template creation processing unit 5 is stored.

  The test pattern creation processing unit 7 is a data processing device that operates under program control. An environment capable of executing the full search algorithm is prepared, and all test patterns obtained by the full search are created from the test pattern template, and a test pattern file is created.

  The test pattern file storage unit 8 is a storage device that stores information. Here, the test pattern file created by the test pattern creation processing unit 7 is stored.

  The test execution processing unit 9 is a data processing device that operates under program control. A test file according to a test pattern is executed by providing an environment in which a source file to be tested can be stepped and an environment in which a standard function and a stub function can be dynamically switched.

  The test result file 10 is a data file indicating the result of the test executed by the test execution processing unit 9.

Hereinafter, the entire processing flow will be described with reference to FIG.
(1) Step S101
The test pattern template creation processing unit 5 executes the source file 1 to be tested and creates a test pattern template file.
(2) Step S102
Next, the test pattern creation processing unit 7 creates a test pattern file from the test pattern template file. Of the functions described in the test pattern template file, a switching flag (0, 1) is added to the function for which the stub file 3 is prepared. When the switching flag is 0, a normal function is executed, and when it is 1, a stub function is executed.
(3) Step S103
Next, the test execution processing unit performs a test along the test pattern file.

Hereinafter, with reference to FIG. 3, an operation when the test pattern template creation processing unit 5 creates a test pattern template file will be described.
(1) Step S201
First, the source file 1 to be tested is acquired from the source file storage unit.
(2) Step S202
The acquired source file 1 to be tested is built with a compiler or the like and executed in an environment where step execution is possible such as a debugger. Note that building means creating a final executable file by compiling source code and linking libraries.
(3) Step S203
Steps are advanced in order until the execution step ends.
(4) Step S204
It is determined whether or not the final step has been reached.
(5) Step S205
If the final step has not been reached, the next step is executed to check whether a function exists in the execution step.
(6) Step S206
If a function exists during the execution step, the function name is written to the test pattern template file.
(7) Step S207
When the final step is reached, the created test pattern template file is stored in the test pattern template file storage unit 6.

FIG. 4 shows an example of the source file 1 to be tested.
In the source file to be tested shown in FIG. 4, the “read function” is called three times by the repetition syntax, and finally the “print function” is called once.

FIG. 5 shows the result of stepping the source file of FIG.
In the step execution of FIG. 5, according to the contents of the source file of FIG. 4, the “read function” is called three times and the “print function” is called once during the step execution. That is, “read function” is called once in the first iteration, “read function” is called once in the second iteration, “read function” is called once in the third iteration, and “print function” is finally set to 1. Called times.

FIG. 6 shows an example of a test pattern template file in which the function executed in the step execution of FIG. 5 is written.
Looking at the step execution in FIG. 5, since the “read function” is called three times and the “print function” is called once during the step execution, the test pattern template file is as shown in FIG. In the test pattern template file shown in FIG. 6, “read”, “read”, “read”, and “print” are written.

Hereinafter, with reference to FIG. 7, an operation in the case of creating a test pattern file in the test pattern creation processing unit 7 will be described.
(1) Step S301
A test pattern template file is read from the test pattern template file storage unit 6.
(2) Step S302
The stub file 3 is read from the stub file storage unit 4.
(3) Step S303
Read the test pattern template file to the end of the file in order.
(4) Step S304
Determine if the end of the file has been reached.
(5) Step S305
While the test pattern template file is being read to the end of the file, when a function in the read test pattern template file exists in the stub file 3, a test pattern file is created.
(6) Step S306
A new test pattern file with a switching flag added is created from the test pattern files created so far.
(7) Step S307
When the end of the file is reached, all the created test pattern files are stored in the test pattern file storage unit 8.

FIG. 8 shows an example of the stub file 3.
The stub file 3 includes a stub function read that fails to read the file and returns NULL.

FIG. 9 shows an example of creating a new test pattern file.
A new test pattern file is created by adding a switching flag (0, 1) to the “read function” to the test pattern files D1, D2 created so far. Two new test pattern files are created from one test pattern file. Further, the test pattern files D1 and D2 that are the creation sources are deleted.

FIG. 10 shows an example of a test pattern file created from the test pattern template file.
By adding a switching flag (0, 1) to the “read function” to the test pattern files D1 and D2 created so far, eight test pattern files are created from the test pattern template file of FIG. .

Hereinafter, with reference to FIG. 11, an operation in the case of executing a test along the test pattern file will be described.
(1) Step S401
The test source file 1 is acquired from the source file storage unit.
(2) Step S402
A test pattern file is acquired from the test pattern file storage unit 8.
(3) Step S403
The stub file 3 is acquired from the stub file storage unit 4.
(4) Step S404
Source file 1 and stub file 3 are built, and step execution is performed in an environment where step execution such as a debugger and control branching by user input are possible. Here, when the stub file 3 is built, a prefix is added to the stub function so that the stub function name is not the same as the normal function.
(5) Step S405
Step in order until the final step is reached.
(6) Step S406
It is determined whether or not the final step has been reached.
(7) Step S407
If the final step has not been reached, the next step is executed to check whether a function exists in the execution step.
(8) Step S408
If there is a function in the step, the test pattern file is read in order, and the value of the target function switching flag is acquired.
(9) Step S409
It is determined whether the value of the target function switching flag is 0 or 1.
(10) Step S410
When the value of the target function switching flag is 0, or when the target function switching flag is empty (no stub function), the function is executed as it is.
(11) Step S411
When the value of the target function switching flag is 1, the control is changed to the stub function and the stub function is executed. The control change to the stub function is performed using the control branch function of the debugger.

FIG. 12 shows an example when the source file of FIG. 3 is tested according to the test pattern of FIG.
When step execution is repeated, a “read function” exists in step 4 of the “main function”. At this time, the test pattern file is read in order, and the switching flag of the corresponding function is acquired. Since the switching flag is 0, the “read function” is executed as it is. Next, when step execution proceeds, a “read function” exists in step 6 of the “main function”. Read the test pattern file and get the switching flag. Since the switching flag is 1, the “read function” transfers control to the stub function. Next, when step execution proceeds, a “read function” exists in step 8 of the “main function”. Since the value of the corresponding switching flag is 0, the “read function” is executed as it is. Further, when the step execution is advanced, a “print function” exists in step 11 of the “main function”. Since there is no stub function in the “print function” and there is no corresponding switching flag value, the “print function” is executed as it is.

  The test is executed by the process shown in FIG. 11, and the test execution result is output as a test execution file. The output of the test execution result uses an existing test method, and this patent does not particularly specify the method.

Next, a second embodiment of the present invention will be described in detail.
In the present embodiment, when the test execution processing unit performs a test according to the test pattern file, not all the tests are performed but unnecessary tests are not performed.

Hereinafter, with reference to FIG. 13, an operation when a test is performed by the test execution processing unit will be described.
(1) Step S501
First, a test according to the test pattern is executed. At this time, the executed function is stored.
(2) Step S502
Check if there are any unexecuted functions among the executed test patterns. If there is no function that has not been executed, the test according to the following test pattern is executed.
(3) Step S503
If there are unexecuted functions, all test patterns with the same function switching flag are deleted.

FIG. 14 shows an example of deleting unnecessary test patterns.
In the test pattern 1 of FIG. 14, the “read function” is called three times, and the “print function” is called once. The first and second “read functions” have a switching flag value of 1, and the third “read function” has a switching flag value of 0. As a result of testing the test pattern 1, it is assumed that an error occurs before the third “read function” is executed and the program is terminated without performing the subsequent processing. At this time, as shown in FIG. 14, the “read function” is executed twice. Here, since the third “read function” and “print function” of the test pattern 1 are not executed, the process proceeds to a step of performing test pattern deletion processing.

  As shown in FIG. 14, since the executed function and the corresponding switching flag of the test pattern 2 are the same, the test pattern 2 is deleted and this test is not performed.

  Other processes are the same as those in the first embodiment of the present invention.

  According to the present invention, the conventional test process can be used in an application that enables more comprehensive and efficient operation. In addition, it can be used for the purpose of strengthening the conventional test support tool so that more test items can be automatically tested.

  As described above, in the present invention, a pattern of timing that may cause a run-time error is comprehensively created, and a function to be called during test execution and its stub code are automatically generated according to the pattern. By switching between them, a mechanism to efficiently test error handling is provided. As a result, a test for error handling of a program written in C language or the like is automatically performed.

FIG. 1 is a conceptual diagram showing a configuration of an abnormal system test support apparatus of the present invention. FIG. 2 is a flowchart showing the overall processing flow. FIG. 3 is a flowchart for creating a test pattern template file. FIG. 4 is a diagram illustrating an example of a source file. FIG. 5 is a diagram showing the result of stepping the source file. FIG. 6 is a diagram illustrating an example of a test pattern template file in which the executed function is written. FIG. 7 is a flowchart for creating a test pattern file. FIG. 8 is a diagram illustrating an example of a stub file. FIG. 9 is a diagram illustrating an example of creating a new test pattern file. FIG. 10 is a diagram illustrating examples of eight test pattern files. FIG. 11 is a flowchart for executing a test along the test pattern file. FIG. 12 is a diagram illustrating an example when a source file is tested according to a test pattern. FIG. 13 is a flowchart when the test is executed by the test execution processing unit. FIG. 14 is a diagram illustrating an example of deleting unnecessary test patterns.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Source file 2 ... Source file storage part 3 ... Stub file 4 ... Stub file storage part 5 ... Test pattern template creation processing part 6 ... Test pattern template file storage part 7 ... Test pattern creation processing part 8 ... Test pattern file storage part 9 ... Test execution processing part 10 ... Test result file

Claims (12)

Based on the source file to be tested, and the test pattern template creation processing unit for creating a test pattern template file indicating order of function calls in the source file,
Based on the function name of the source file and the stub file indicating the execution result of the function, and the test pattern template file, a full search algorithm is executed, and among the functions described in the test pattern template file, A test pattern creation processing unit that creates a test pattern file in which a switching flag is added to the function for which the stub file is prepared for each switching pattern ;
The source file and the stub file are read and executed, and during execution , a standard function and error processing are performed at a timing according to the test pattern in the test pattern file based on the switching flag in the test pattern file. An abnormal system test support apparatus comprising: a test execution processing unit that executes a test for simulating a normal system / abnormal system of a function by switching a stub function for generating in a pseudo manner . The abnormal system test support device according to claim 1,
The test pattern template creation processing unit traces the function name while stepping through the executable file program created by building the source file, and if there is a function in the steps until the execution step ends, An abnormal test support device that writes the function name to the test pattern template file during tracing. The abnormal system test support device according to claim 1 or 2,
The test pattern creation processing unit creates a test pattern file in which a switching flag is added to a function for which the stub file is prepared among the functions described in the test pattern template file,
The test execution processing unit executes the executable file program created by building the source file and the stub file, and if there is a function in the steps up to the final step, the test pattern file In order of creation, get the value of the target function switching flag, execute the function as it is when the target function switching flag value is 0 or empty, and execute the stub function when it is 1, abnormal test Support device. The abnormal system test support device according to claim 3,
The abnormal test support apparatus, wherein the test execution processing unit deletes a test pattern having the same function switching flag. The abnormal system test support device according to any one of claims 1 to 4,
The test pattern creation processing unit comprehensively creates a test pattern for generating an error depending on an environment at the time of occurrence at a timing that may occur,
The test execution processing unit replaces a function with a stub function at a timing according to the created test pattern, and automatically performs an error processing test. Based on the source file to be tested, a step of creating a test pattern template file indicating order of function calls in the source file,
Based on the function name of the source file and the stub file indicating the execution result of the function, and the test pattern template file, a full search algorithm is executed, and among the functions described in the test pattern template file, Creating a test pattern file with a switching flag added to the function for which the stub file is prepared for each switching pattern ;
The source file and the stub file are read and executed, and during execution , a standard function and error processing are performed at a timing according to the test pattern in the test pattern file based on the switching flag in the test pattern file. An abnormal system test support method comprising a step of executing a test for simulating a normal system / abnormal system of the function by switching a stub function to be generated in a pseudo manner. The abnormal system test support method according to claim 6,
Executing an executable program created by building the source file in an environment capable of stepping;
Tracing the function name while stepping, and
Checking whether the function exists during the execution step;
If a function exists during the execution step, writing the function name to the test pattern template file during tracing; and
Determining whether the final step has been reached;
An abnormal system test support method, further comprising a step of storing the test pattern template file when the final step is reached. The abnormal system test support method according to claim 6 or 7,
Sequentially reading the test pattern template file to the end of the file;
When a function in the test pattern template file exists in the stub file, among the functions described in the test pattern template file, a test pattern in which a switching flag is added to a function for which the stub file is prepared Creating a file;
Determining whether the end of the file has been reached;
An abnormal test support method, further comprising a step of saving the created test pattern file when the end of the file is reached. The abnormal system test support method according to any one of claims 6 to 8,
Stepping an executable file program created by building the source file and the stub file;
When building the stub file, prefixing the stub function so that the normal function and the stub function name are not the same;
Checking whether the function exists during the execution step;
If a function exists during the step, the test pattern file is read in the order of creation, and the value of the target function switching flag is acquired;
Determining whether the value of the switching flag of the target function is 0 or 1,
When the value of the target function switching flag is 0, or when the target function switching flag is empty (when there is no stub function), executing the function as it is;
An abnormal system test support method, further comprising: when the value of the target function switching flag is 1, changing the control to a stub function and executing the stub function. An abnormal system test support method according to any one of claims 6 to 9,
When executing a test according to a test pattern, storing the executed function;
A step of checking whether there are unexecuted functions among the executed test patterns;
And a step of deleting a test pattern having the same switching flag of the executed function when there is an unexecuted function. An abnormal system test support method according to any one of claims 6 to 10,
A step of comprehensively creating a test pattern that generates an error that depends on the environment at the time of occurrence,
An abnormal test support method, further comprising: a step of replacing a function with a stub function at a timing according to the created test pattern and automatically performing an error processing test.   A program for causing a computer to execute the abnormal system test support method according to any one of claims 6 to 11.

Images