JP7238439B2 - Information processing device, test method, and test program - Google Patents

Description

The present invention relates to an information processing device, test method, and test program.

A test system that automatically tests software to be tested and outputs test results is known.

Also known is a bug analysis method and apparatus for analyzing bugs in a software development project and selecting priority issues from the number of rework due to bugs, etc. (see, for example, Patent Document 1).

However, in a test system that automatically tests the software under test, for example, even if multiple tests fail due to the same defect (bug), it is difficult to determine whether the cause is the same defect. It is difficult to distinguish automatically. Therefore, the test system raises a problem ticket for every test that fails, even if multiple tests fail due to the same problem, and the designer is responsible for the multiple problems raised. There is the problem of having to parse the ballot to determine the fault that caused the failure.

SUMMARY OF THE INVENTION An embodiment of the present invention has been made in view of the above problems. , to make it possible to efficiently analyze failures that have caused failures.

To achieve the above object, an information processing apparatus according to one embodiment of the present invention is an information processing apparatus for testing software to be tested, comprising a test control unit for controlling execution of a plurality of tests on the software; a first judgment unit for judging whether or not the software has functional characteristics in a failed test among the plurality of tests; a second judging section for judging; and an output section for outputting a test result related to a fault that caused the failure based on judgment results of the first judging section and the second judging section. .

According to one embodiment of the present invention, in an information processing system for automatically testing software, when multiple tests fail due to the same failure, the failure that caused the failure is analyzed. be able to do so efficiently.

1 is a diagram illustrating an example of a system configuration of an information processing system according to an embodiment; FIG. It is a figure which shows the structural example of the test tool which concerns on one Embodiment. It is a figure which shows the example of the hardware constitutions of the information processing apparatus which concerns on one Embodiment. It is a figure showing an example of functional composition of an information processor concerning one embodiment. FIG. 4 is a diagram for explaining test specifications according to one embodiment; FIG. 4 is a diagram for explaining an example of test cases according to an embodiment; FIG. 4 is a flow chart showing an example of processing of the information processing apparatus according to the first embodiment; 9 is a flowchart showing an example of processing of an information processing apparatus according to the second embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<System configuration>
(System configuration of information processing system)
FIG. 1 is a diagram illustrating an example of the system configuration of an information processing system according to one embodiment. The information processing system 100 is a system in which a designer 101 or the like who designs software tests (tests) software to be tested.

In software development in recent years, frequent repetition of builds and tests, which is called continuous integration (CI), is used to detect problems early, improve development efficiency, save power, and shorten delivery times. etc. is used. In the CI system, procedures are automated, and the workflow as a whole is made more efficient by cooperating with the change management tool 102, test management tool 106, fault management tool 109, and the like. Note that the CI system is an example of the information processing system 100 .

In the information processing system 100 shown in FIG. 1, a designer 101 creates a source code 103 of software to be tested and a test code 107 for testing the software, and registers them in the information processing system 100 .

The build tool 104 and test tool 108 automatically build and test according to a predetermined schedule, and if the test result fails, the fault management tool 109 reports the failed test result. etc. is notified to the designer 101 . At this time, the failure management tool 109 can track the status of failure countermeasures by cooperating with the change management tool 102, the test management tool 106, and the like.

If the test result fails, the designer 101 analyzes the failure (bug) and corrects the source code 103 or the test code 107 as necessary. In analyzing the failure, the designer 101 identifies the failure based on, for example, what kind of error occurred in which test case, log information when the error occurred, and the like, and takes countermeasures.

By repeatedly executing such a flow, the designer 101 always maintains a state in which the test results pass. As a result, the designer 101 can keep the software (product) to be tested in a state ready for release at any time, so that the software can be released to the market in a timely manner.

(Test tool configuration example)
FIG. 2 is a diagram illustrating a configuration example of a test tool according to one embodiment. Here, as an example, a test tool in embedded software development will be described. Automatic testing in embedded software development is basically performed by executing the test target object 105 on the target device 202 . Here, the “target device” includes not only an electronic device on which software is actually installed (hereinafter referred to as “real device”), but also an emulator that emulates the operation of the real device.

In addition, as an example of the method of executing the test target object 105 on the target device 202, as shown in FIG. There is a way to run 105 and test code 107 . Also, as another example, as shown in FIG. There are also methods for manipulating the object 105 . The method shown in FIG. 2A is mainly used for white-box tests, and the method shown in FIG. 3B is mainly used for black-box tests.

In the above configuration, the information processing system 100, for example, automatically executes a test, and when the test fails, automatically issues a trouble ticket and notifies the designer 101 of it. In such an information processing system 100, for example, when a plurality of tests fail due to the same fault (bug), the information processing system 100 automatically determines that the cause is the same fault. is difficult. Therefore, the information processing system 100 issues a trouble ticket for all the failed tests, and the designer 101 analyzes the plurality of trouble tickets and analyzes the plurality of failed tests. There is the problem of having to determine that the cause is due to the same fault.

At this time, for example, the information processing system 100 may group faults of the same phenomenon as "same faults" based on the phenomenon of the faults that have occurred, and notify the designer 101 of them. However, with this method alone, failures of the same phenomenon are not necessarily caused by the same cause, so latent failures (bugs) may be overlooked.

As another method, the information processing system 100 separates the basic function test from the branch function test, and performs the basic function test first, thereby generating a large number of trouble tickets. A method of suppressing votes is also conceivable. However, in this method, it is necessary to classify which functions are basic functions and which functions are secondary functions. Another problem with this method is that if the basic function test fails, other functions cannot be tested.

Therefore, in the information processing system 100 according to the present embodiment, the test is automatically executed, and when the test fails, based on the phenomenon characteristics and the functional characteristics of the failed test, , outputs the test results for the fault that caused the failure.

Preferably, the information processing system 100 divides the failed tests (for example, trouble reports, etc.) into groups based on the characteristics of the phenomena and the features of the functions of the failed tests, and displays the test results. to output

Here, the functional features include, for example, features related to "functions" described in the functional specifications of the software to be tested. For example, the information processing system 100 performs the failed tests based on the number of failed test items among one or more test items related to a predetermined function described in the functional specifications. It determines whether there is a specific characteristic (for example, whether there is a problem with a predetermined function).

Further, the characteristics of the phenomenon include, for example, characteristics related to "phenomenon" generated by a predetermined command used in the test. For example, the information processing system 100 determines whether the failed test has phenomenological characteristics (errors of the same or not). Note that the command may include various instructions such as an instruction to execute a predetermined procedure, an instruction to access predetermined hardware, and the like.

As a specific example, if there is a characteristic of the phenomenon such that an error occurs multiple times in a predetermined command, the information processing system 100 prioritizes the characteristic of the phenomenon over the characteristic of the function. is notified to the designer 101 . This is because if errors occur frequently in a specific command, there is a high possibility that the function to be tested cannot be properly tested.

For example, it is assumed that the device to be tested is an MFP (Multifunction Peripheral) having a scan function, a copy function, a printer function, a facsimile function, etc. in one housing. In the MFP software test, it is assumed that the scan function test failed, but an error actually occurred due to the address book registration command included in the preparation work. In such a case, according to the information processing system 100 according to the present embodiment, it becomes easier to request failure analysis to the person in charge of the address book function instead of the person in charge of the scan function. Note that the specific processing contents will be described later.

<Hardware configuration>
FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing apparatus according to one embodiment; The change management tool 102, build tool 104, test management tool 106, test tool 108, fault management tool 109, etc. shown in FIG. is executed by the information processing apparatus 300 of

The information processing apparatus 300 includes, for example, a CPU (Central Processing Unit) 301 , a ROM (Read Only Memory) 302 , a RAM (Random Access Memory) 303 , an HD (Hard Disk) 304 , an HDD (Hard Disk Drive) controller 305 , a display 306 , an external device connection I/F 307, a network I/F 308, a keyboard 309, a pointing device 310, a DVD-RW (Digital Versatile Disk Rewritable) drive 312, a media I/F 314, a bus line 315, and the like.

Among these, the CPU 301 controls the overall operation of the information processing apparatus 300 . The ROM 302 stores a program such as an IPL (Initial Program Loader) that is used to start the CPU 301 . A RAM 303 is used as a work area for the CPU 301 . The HD 304 stores various data such as programs. The HDD controller 305 controls reading or writing of various data to/from the HD 304 under the control of the CPU 301 .

A display 306 displays various information such as cursors, menus, windows, characters, and images. The external device connection I/F 307 is an interface for connecting various external devices. The external device includes, for example, a USB (Universal Serial Bus) memory, a printer, and the like, as well as the target device 202 to be tested. A network I/F 308 is an interface for data communication using a communication network. Note that the information processing apparatus 300 may control the target device 202 via the network I/F 308 .

A keyboard 309 is a type of input means having a plurality of keys for inputting characters, numerical values, various instructions, and the like. The pointing device 310 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. A DVD-RW drive 312 controls reading or writing of various data to a DVD-RW 311 as an example of a removable recording medium. The DVD-RW 311 is not limited to DVD-RW, and may be DVD-R or the like. A media I/F 314 controls reading or writing (storage) of data to a media 313 such as a flash memory. A bus line 315 includes an address bus, a data bus, various control signals, and the like for electrically connecting the components described above.

<Functional configuration>
Next, the functional configuration of the information processing device 300 will be described.

FIG. 4 is a diagram illustrating an example of a functional configuration of an information processing device according to an embodiment; The information processing apparatus 300, for example, executes a predetermined test program in the CPU 301 of FIG. , and the storage unit 406 and the like. It should be noted that at least part of the functional configurations described above may be realized by hardware.

The test control unit 401 is implemented, for example, by the test tool 108 or the like executed by the CPU 301 in FIG. 3, and controls execution of a plurality of tests regarding software to be tested.

The first determination unit 402 is implemented, for example, by a test program such as the failure management tool 109 executed by the CPU in FIG. determine whether there is a characteristic of For example, based on the number of failed test items among one or more test items related to a predetermined function, the first determination unit 402 determines whether the failed tests have functional characteristics. (For example, whether there is a problem with a predetermined function) or not.

FIG. 5 is a diagram for explaining test specifications according to one embodiment. For example, when creating a test case for verifying functional compatibility, the designer 101 designs the test case for the functional specification 501 of the software to be tested. For example, when testing sub-function A-1 of function A in functional specification 501 of FIG. do. Similarly, when testing sub-function A-2 of function A in functional specification 501, one or more test items relating to sub-function A-2 of function A are tested, as shown in test specification 411. FIG.

Preferably, for example, when the test specification 411 defines a plurality of test items related to a predetermined function, the first determination unit 402 sets the number of failed test items to a preset first is equal to or greater than the threshold value, it is determined that the predetermined function has functional characteristics. If there is only one test item for a given function, it may be determined that the given function has a functional feature when the test item for the given function fails.

The second determination unit 403 is implemented by, for example, a test program such as the fault management tool 109 executed by the CPU in FIG. determine whether there is a characteristic of For example, the second determination unit 403 determines whether the failed test has characteristics in terms of phenomena (for example, whether there is a problem with the predetermined command), based on the number of times errors have occurred in the predetermined command. determine whether or not

FIG. 6 is a diagram for explaining an example of test cases according to one embodiment. For example, when testing a given function, preconditions and postconditions are clarified prior to execution of the test scenario, and the test is executed. For example, as shown in a test case 601 in FIG. 6, a "precondition", a "postcondition", etc. are automatically executed before and after a "test scenario" for testing a predetermined function.

Various methods can be used to automatically execute each procedure included in the test case 601 . For example, in many cases, the manufacturer that provides the target device 202 provides various tools for testing as well as the build tool 104, so these tools can be used to build the test target object 105. can be manipulated. Further, for example, an I/F provided externally by the object under test 105 can be used for automatic execution of each procedure included in the test case 601 . Note that the I/F may be tested in another test case.

Preferably, the second judging unit 403 determines that the software test fails when the number of errors occurring in a predetermined command (for example, an execution instruction of procedure B1) is equal to or greater than a second threshold. Judging that the software has the characteristics of the phenomenon.

The output unit 404 is implemented by, for example, a test program such as the fault management tool 109 executed by the CPU in FIG. The output unit 404 outputs a test result related to the failure that caused the failure based on the phenomena and functional features of the failed test among the plurality of tests.

For example, if a failed test has functional features and phenomenal features, the output unit 404 gives priority to the functional features over the functional features, and executes a predetermined command (procedure). output a test result indicating that the

In addition, if the failed test has functional characteristics but no phenomenon characteristics, the output unit 404 outputs the test result regarding a predetermined function. Furthermore, if the failed test has phenomenal characteristics but no functional characteristics, the output unit 404 outputs the test results regarding the predetermined command.

Preferably, the output unit 404 divides the failed tests (or trouble reports, etc.) into groups based on the characteristics of the phenomena and the features of the failed tests, and outputs the results as test results. Output. For example, the output unit 404 groups and outputs a plurality of trouble tickets that are considered to be caused by the same trouble based on the characteristics of the phenomena and the characteristics of the failed tests.

The setting reception unit 405 is implemented by, for example, a test program such as the failure management tool 109 executed by the CPU in FIG. 3, and receives the setting of the above-described first threshold or second threshold. For example, the setting reception unit 405 displays a setting screen for setting the first threshold value or the second threshold value on the display unit such as the display 306 in FIG. Alternatively, an operation for setting the second threshold is accepted. The setting reception unit 405 also stores the received first threshold value or second threshold value in the storage unit 406 as a setting value 413 . Note that the setting reception unit 405 may receive a request for setting the first threshold value or the second threshold value from another information processing apparatus or the like.

The storage unit 406 is implemented by, for example, the program executed by the CPU in FIG. 3, the HDD controller 305, the HD 304, etc., and stores the test code 107, the test specifications 411, the test results 412, the setting values 413, and the like.

In the example of FIG. 4, the functional configuration of the test tool 108 and fault management tool 109 of FIG. 1 has been described. Tools such as 106 may also be executed. The functional configurations of the change management tool 102, the build tool 104, the test management tool 106, and the like may be the same as those of general tools, so description thereof will be omitted here.

<Process flow>
Next, the processing flow of the test method according to each embodiment will be described.

[First Embodiment]
7 is a flowchart illustrating an example of processing of the information processing apparatus according to the first embodiment; FIG. This processing shows an example of processing executed by the information processing apparatus 300 when there is a failed test after executing a plurality of tests on software to be tested by the test control unit 401 .

In step S701, the first determination unit 402 of the information processing apparatus 300 extracts functional features of the failed test. For example, the first determination unit 402 refers to the test specification 411 as shown in FIG. 5 and extracts the functions corresponding to the failed test. As an example, in the test specification 411 shown in FIG. 5, when the test item "boundary value test normal system" fails, the first judging unit 402 determines the sub-function " A-1” etc. are extracted.

In step S702, the second determination unit 403 of the information processing apparatus 300 extracts the features of the phenomena in the failed test. For example, the second determination unit 403 extracts commands in which errors have occurred in failed tests, the number of times errors have occurred in the commands, and the like.

In step S703, the first determination unit 402 determines whether the failed test has functional characteristics. For example, when the number of failed test items among the one or more test items related to the function extracted in step S701 is equal to or greater than a preset first threshold, the first determination unit 402 A test that passes is judged to have functional characteristics.

If the failed test has functional characteristics, the information processing apparatus 300 shifts the process to step S704. On the other hand, if the failed test has no functional characteristics, the information processing apparatus 300 shifts the process to step S705.

After proceeding to step S704, the second determination unit 403 determines whether or not the failed test has a feature in terms of phenomena. For example, when the number of errors that occurred in the command extracted in step S702 is equal to or greater than a second preset threshold value, the second determination unit 403 determines that the failed test has a phenomenon feature. to decide.

If the failed test does not have any phenomenal features, the information processing apparatus 300 shifts the process to step S705. On the other hand, if the failed test has a phenomenon, the information processing apparatus 300 shifts the process to step S707.

After moving to step S705, the output unit 404 outputs a test result pointing out the degradation of the specific function. For example, the output unit 404 outputs a trouble report or the like pointing out the deterioration of the function judged to have functional characteristics in step S703.

When the process proceeds from step S703 to step S706, the second determination unit 403 determines whether or not the failed test has characteristics in terms of phenomenon, for example, in the same manner as in step S704.

If the failed test has a phenomenon, the information processing apparatus 300 shifts the process to step S707. On the other hand, if the failed test does not have any phenomenal features, the information processing apparatus 300 terminates the process.

After moving to step S707, the output unit 404 of the information processing apparatus 300 outputs a test result pointing out the deterioration of the specific procedure. For example, the output unit 404 outputs a trouble report or the like pointing out the deterioration of the command (an example of the specific procedure) extracted in step S702.

In this way, the output unit 404 of the information processing apparatus 300 outputs the test results regarding the fault that caused the failure based on the phenomenal features and functional features of the failed test.

As a result, according to the information processing system 100 according to the present embodiment, the order of priority to be analyzed is presented to the designer 101 based on the functional characteristics and phenomenon characteristics of the failed test. can be performed, the failure analysis by the designer 101 can be made more efficient.

[Second embodiment]
FIG. 8 is a flowchart illustrating an example of processing of the information processing apparatus according to the second embodiment. This process is another example of the process executed by the information processing device 300 when there is a failed test after the test control unit 401 has executed a plurality of tests on the software to be tested. there is Note that the basic processing content is the same as the processing of the information processing apparatus according to the first embodiment shown in FIG. 7, so detailed description of the processing similar to that of the first embodiment will be omitted here.

In step S801, the first determination unit 402 of the information processing apparatus 300 extracts the functional features of the failed test. Note that when a plurality of tests fails, the first determination unit 402 extracts functional features of each of the plurality of failed tests.

In step S802, the second determination unit 403 of the information processing apparatus 300 extracts features of phenomena in the failed test. Note that when a plurality of tests fails, the second determination unit 403 extracts the features of the phenomena for each of the plurality of failed tests.

In step S803, the first determination unit 402 determines whether or not each failed test has functional characteristics.

Here, the information processing apparatus 300 executes the process of step S804 for the test determined to have functional characteristics. Further, the information processing apparatus 300 executes the process of step S806 for the test determined to have no functional feature.

In step S804, the second determination unit 403 determines whether or not each of the failed tests determined to have functional characteristics has phenomenal characteristics.

Here, the information processing apparatus 300 executes the process of step S805 for the test determined to have no feature in terms of phenomena. Further, the information processing apparatus 300 executes the process of step S807 for the test determined to have the feature of the phenomenon.

In step S805, the output unit 404 of the information processing apparatus 300 creates a trouble ticket (an example of test results) pointing out the deterioration of the specific function. For example, the output unit 404 determines that the test items above the first threshold fail for each of the tests determined to have functional characteristics in step S803 and to have no phenomenon characteristics in step S804. Create a trouble ticket indicating that the missing function is at fault.

On the other hand, in step S806, the second judging unit 403 judges whether or not each of the failed tests judged to have no functional feature has a phenomenon feature. .

Here, the information processing apparatus 300 executes the process of step S807 for the test determined to have the feature of the phenomenon. Note that the information processing apparatus 300 shifts the process to step S808 without performing any particular process for the test determined to have no feature in terms of phenomena.

In step S807, the output unit 404 creates a trouble ticket pointing out the deterioration of the specific procedure.

For example, the output unit 404 outputs commands (specific Create a trouble ticket pointing out the degradation of the procedure (example).

In addition, the output unit 404 outputs commands (specific Create a trouble ticket pointing out the degradation of the procedure (example).

In step S808, the output unit 404 outputs the trouble tickets created in steps S805 and S807 by grouping them according to the trouble details. For example, the output unit 404 collects and outputs a plurality of trouble tickets judged to have a trouble in a predetermined function into one group. In addition, the output unit 404 outputs a plurality of trouble tickets judged to have a trouble in a predetermined phenomenon as one group.

In this manner, the output unit 404 of the information processing apparatus 300 groups the failed test results (trouble reports) according to the type of failure based on the characteristics of the phenomenon and the characteristics of the function. output.

As a result, according to the information processing system 100 according to the present embodiment, the order of priority to be analyzed is presented to the designer 101 based on the functional characteristics and phenomenon characteristics of the failed test. can be performed, the failure analysis by the designer 101 can be made more efficient.

<Application example>
Next, a specific application example of the software testing method according to the second embodiment will be described.

Here, as an example, the following description is given assuming that the target device 202 is an image forming apparatus such as an MFP (Multifunction Peripheral) that has a scan function, a copy function, a printer function, a facsimile function, etc. in one housing. conduct.

However, the target device 202 is not limited to the image forming apparatus, and may be various devices that execute software to be tested by the information processing system 100 . For example, the target device 202 is an output device such as a PJ (Projector), an IWB (Interactive White Board), a digital signage, or a HUD (Head Up Display) device. It can be. Also, the target device 202 may be, for example, an industrial machine, an imaging device, a sound collector, a medical device, a network appliance, a car (connected car), a digital camera, a wearable terminal, or the like. Furthermore, the target device 202 may be a general-purpose information processing device such as a notebook PC, mobile phone, smart phone, tablet terminal, game machine, PDA (Personal Digital Assistant), digital camera, wearable PC, or desktop PC.

For example, when considering the test of software that realizes the scan function in MFP, the scan function includes various functions such as "scan to mail function", "scan to folder function", "OCR (Optical Character Recognition) function", etc. functions.

The "scan to mail function" also includes a plurality of sub-functions such as "scan condition setting" and "mail destination setting". For example, when testing functional compatibility, it is common to create test cases corresponding to these functional specifications.

As an example, when creating a test case for "scanning condition setting" of the "scan to email function", it is possible to test whether or not the scan results are actually sent by email under various scanning conditions. At this time, what I want to test is whether the settings such as "Double-sided specification", "Color specification", and "Resolution specification" as "Scanning conditions" operate as set. In order to do so, there is a procedure to be executed in advance as a precondition. For example, the preconditions include "the mail transmission server is set" and "the destination mail address is registered".

In the automated test environment, it is necessary to automatically perform the processing to prepare these preconditions. procedure may cause the test to fail. In this example, the causal relationship between the preconditions and the test target is relatively clear. It is difficult to be clear on

In such cases, each embodiment of the present invention is applied to design test results that indicate the order of priority to be analyzed, taking into account both the functional characteristics and the phenomena characteristics of the failed tests. By presenting it to the person 101, failure analysis can be made more efficient.

For example, it is assumed that there are 100 test cases for "scan condition setting" of "scan to mail function" and all the tests passed in the previous build. Also, in this build, it is assumed that 30 or more tests out of the test cases have failed. In such a case, the information processing apparatus 300 may extract, as a functional feature, the possibility that the "scan condition setting" function has caused some kind of degradation. Note that the above "30 cases" is just an example, and the first threshold for determining that there is a functional feature can be set to any value.

On the other hand, it is assumed that the failure phenomena in the 30 test cases are totaled, and that all of the 30 cases have an error in the "register address book" command, for example. In such a case, the information processing apparatus 300 may extract the fact that the "register address book" command may have caused some kind of degradation as a feature of the phenomenon.

In addition, when the information processing apparatus 300 can extract such functional features and phenomenon features, the information processing apparatus 300 groups these 30 failures, ” to the designer. As a result, a more appropriate designer 101 (for example, a designer of the address book registration command, etc.) can be urged to perform the analysis, so failure analysis can be made more efficient.

In addition, in the above example, the test is described as being a functional compatibility test, but it is not limited to this. For example, if the internal structure of software is to be tested, it is sufficient to extract the structural features of the target. For example, if a failure is detected in a state transition exhaustive test, it is conceivable to extract, as a feature, the rate of failure indicated in the exhaustive test.

As described above, according to the embodiments of the present invention, in the information processing system 100 that automatically tests software, if a plurality of tests fail due to the same failure, the failure that caused the failure analysis can be performed efficiently.

<Supplement>
Each function of each embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in this specification means a processor programmed by software to perform each function, such as a processor implemented by an electronic circuit, or a processor designed to perform each function described above. ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays) and devices such as conventional circuit modules.

100 information processing system 300 information processing device 401 test control unit 402 first determination unit 403 second determination unit 404 output unit 405 setting reception unit

Japanese Patent Application Laid-Open No. 2005-222108

Claims (10)

An information processing device for testing software to be tested,
a test controller that controls execution of a plurality of tests for the software;
Among the plurality of tests, in the failed test,
a first determination unit that determines whether the software has functional features;
a second judging unit for judging whether the software has phenomenal features;
an output unit that outputs a test result related to a failure that caused the failure based on the determination results of the first determination unit and the second determination unit ;
An information processing device. 2. The information processing apparatus according to claim 1, wherein the output unit groups the failed tests based on the characteristics of the phenomenon and the characteristics of the function, and outputs the test results. The plurality of tests includes one or more test items related to a predetermined function,
The first determination unit determines, based on the number of failed test items among one or more test items related to the predetermined function, that the failed test has the functional feature. 3. The information processing apparatus according to claim 1 , which determines whether or not . The second determination unit determines whether the failed test has the characteristic of the phenomenon based on the number of times an error occurs in a predetermined command used in the plurality of tests . The information processing apparatus according to claim 3. 5. The output unit according to claim 4, wherein, if the failed test includes the characteristics of the phenomenon and the characteristics of the function, the output unit outputs the test result with priority given to the characteristics of the phenomenon. information processing equipment. 6. The information according to claim 4, wherein said output unit outputs said test result related to said predetermined command when said failed test has said phenomenon aspect feature and said function aspect feature. processing equipment. The output unit
outputting the test result for the predetermined function if the failed test has the functional feature and does not have the phenomenon feature;
outputting the test result for the predetermined command if the failed test has the phenomenon feature and does not have the functional feature;
The information processing apparatus according to any one of claims 4 to 6. 8. The apparatus according to any one of claims 5 to 7, further comprising a setting reception unit that receives a setting of a first threshold for determining that the function is characteristic or a second threshold for determining that the phenomenon is characteristic. The information processing device according to . A test method for testing software to be tested,
the computer
a process of controlling execution of a plurality of tests for the software;
Among the plurality of tests, in the failed test,
a first determination process for determining whether the software has functional characteristics;
a second determination process for determining whether the software has phenomenal characteristics;
a process of outputting a test result related to the failure that caused the failure based on the determination results of the first determination process and the second determination process ;
test method. On the computer that tests the software under test,
a process of controlling execution of a plurality of tests for the software;
Among the plurality of tests, in the failed test,
a first determination process for determining whether the software has functional characteristics;
a second determination process for determining whether the software has phenomenal characteristics;
a process of outputting a test result related to the failure that caused the failure based on the determination results of the first determination process and the second determination process ;
A test program that runs the

Images