JP6248425B2 - Program, process selection method, and information processing apparatus - Google Patents

Description

  The present invention relates to a program, a process selection method, and an information processing apparatus.

  Currently, in software development, tests are performed to confirm the operation of software in order to improve the quality of software. In the test, for example, software to be tested is executed by an information processing apparatus such as a computer. Then, it is checked whether the software can properly perform the designed operation for the planned input. If the operation is invalid (error), information related to the error can be collected, the cause of the error can be analyzed from the information, and countermeasures can be taken.

  For example, what kind of information is useful for investigating the cause for each error type is set in advance, and failure investigation information collected in the main memory based on this setting is stored in a storage medium such as a magnetic disk device. There has been proposed a method for easily obtaining the trouble investigation information by outputting. In this proposal, as fault investigation information, logs indicating the status of hardware and software at the time of error, running contents of various programs and the status of hardware and software resources that change in relation to the operation are dynamically and continuously Collected trace information is listed.

Japanese Patent Laid-Open No. 10-260861

  By the way, the execution state of software can be managed in units called processes on the information processing apparatus. Therefore, it is conceivable to specify information to be collected at the time of error based on the information of the process. For example, by narrowing down information to be collected by specifying processes related to the test in advance, it is possible to make the analysis work for finding the cause more efficient than collecting unnecessary information.

  However, only software that operates in association with software to be tested is not always executed on the information processing apparatus. Software unrelated to the software under test can also be executed. Thus, there is a problem of how to efficiently select a process for collecting information at the time of testing from among a large number of existing software processes.

  In one aspect, an object of the present invention is to provide a program, a process selection method, and an information processing apparatus that can improve test efficiency.

In one aspect, a program executed by a computer capable of executing a plurality of processes is provided. This program obtains first information indicating the accumulated time executed for each existing process when starting a software test on the computer, and exists when the software test ends. For each process, the second information indicating the accumulated time of execution is acquired, and information is collected when the software is tested again based on the first and second information acquired for each process. of selecting a process of interest, the first history during the test on the process intended for information gathering, when an abnormality of the second process communicating with the first process is recorded, a second process Execute the process that targets information collection .

Further, in one aspect, a process selection method executed by an information processing apparatus that can execute a plurality of processes is provided. In this process selection method, when the information processing apparatus starts a software test, the information processing apparatus acquires first information indicating the accumulated time for each existing process, and ends the software test. In this case, for each existing process, the second information indicating the accumulated time of execution is acquired, and the software test is performed again based on the first and second information acquired for each process. When a process to be collected information is selected and an abnormality of the second process communicating with the first process is recorded in the history of the test related to the first process to be collected information, The second process is the target of information collection .

In one aspect, an information processing apparatus capable of executing a plurality of processes is provided. This information processing apparatus includes a storage unit and a calculation unit. The storage unit stores the accumulated time for each existing process. When starting the software test, the arithmetic unit refers to the storage unit, obtains first information indicating the accumulated time for each existing process, and ends the software test. At the time, referring to the storage unit, the second information indicating the accumulated execution time is obtained for each existing process, and based on the first and second information obtained for each process. When the software test is performed again, the process to be collected information is selected , and the abnormality in the second process communicating with the first process is recorded in the history of the test related to the first process to be collected. Is recorded, the second process is the target of information collection .

  In one aspect, testing can be made more efficient.

It is a figure which shows the information processing apparatus of 1st Embodiment. It is a figure which shows the hardware example of the test apparatus of 2nd Embodiment. It is a figure which shows the example of a function of a test apparatus. It is a figure which shows the state transition of a process. It is a figure which shows the example of the execution pattern of the process at the time of a test. It is a figure which shows the example of a candidate process table. It is a figure which shows the example of an exclusion process table. It is a figure which shows the example of a test management table. It is a figure which shows the example of a start table. It is a figure which shows the example of an end table. It is a figure which shows the example of a comparison table. It is a figure which shows the example of a process history table. It is a figure which shows the example of a process table. It is a flowchart which shows the process example of a software test. It is a flowchart which shows the example of a process selection process. It is a flowchart which shows the example of the process at the time of a test start. It is a flowchart which shows the example of the process at the time of a test completion | finish. It is a flowchart which shows the example of a comparison process. It is a flowchart which shows the example of a memory dump acquisition process. It is a flowchart which shows the example of a trace data acquisition process. It is a sequence diagram which shows the specific example of a process selection process. It is a figure which shows the specific example of process selection. It is a sequence diagram which shows the specific example of an information collection process. It is a flowchart which shows the verification example of the effectiveness of a test item. It is a flowchart which shows the example of the extraction process of a test item.

Hereinafter, the present embodiment will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram illustrating the information processing apparatus according to the first embodiment. The information processing apparatus 1 executes test target software and performs an operation test on the software. The information processing apparatus 1 can execute a plurality of processes. The process is one of execution units of various software programs executed by the information processing apparatus 1. For example, an operating system (OS: Operating System) executed by the information processing apparatus 1 manages generation, waiting for execution, execution, and termination of each software process.

  The information processing apparatus 1 includes a storage unit 1a and a calculation unit 1b. The storage unit 1a may be a volatile storage device such as a RAM (Random Access Memory) or a non-volatile storage device such as an HDD (Hard Disk Drive) or a flash memory. The calculation unit 1b may include a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), and the like. The calculation unit 1b may be a processor that executes a program. A “processor” may include a set of multiple processors (multiprocessor). The information processing of the first embodiment may be realized by the calculation unit 1b executing a program stored in the storage unit 1a.

  The storage unit 1a stores management information 2 indicating accumulated time (sometimes referred to as accumulated execution time) executed for each existing process. For example, the OS executed by the information processing apparatus 1 records the accumulated time executed for each process currently existing in the management information 2. An existing process is a generated process. The existing process can be said to be a process in which resources such as a processor and a RAM are allocated by the OS. A non-existing process is a process that has existed in the past but is now terminated and deleted. A process that does not exist can be said to be a process in which resources allocated by the OS are released. The accumulated execution time is the accumulation of the time when the corresponding process is in the execution state.

  When starting the test of the software to be tested, the calculation unit 1b acquires first information indicating the accumulated time for each existing process. The computing unit 1b can acquire the first information for each process existing at the start of the test with reference to the storage unit 1a. For example, the calculation unit 1b registers the acquired first information in the recording information 3 stored in the storage unit 1a in association with the process.

  When completing the test of the software to be tested, the arithmetic unit 1b acquires second information indicating the accumulated time for each existing process. The computing unit 1b can obtain the second information for each process existing at the end of the test with reference to the storage unit 1a. For example, the calculation unit 1b registers the acquired second information in the recording information 3 in association with the process.

Based on the first and second information acquired for each process, the computing unit 1b selects a process for which information is to be collected when a test target software is tested.
Here, the management information 2 and the record information 3 in FIG. 1 exemplify registration contents when a predetermined time has elapsed after a test of a certain software. For example, at the time, the next cumulative execution time is registered in the management information 2 for each process. Accumulated execution time t12 of process A1. Accumulated execution time t22 of process A2. Accumulated execution time t42 of process A4. Accumulated execution time t52 of process A5.

  Further, for example, in the recording information 3, the accumulated execution time at the start of the test is registered as follows for the process that existed at the start of the test. No registration (denoted with a hyphen “-”) indicates that the cumulative execution time could not be acquired because it did not exist at that time. Accumulated execution time t10 of process A1. Accumulated execution time t20 of process A2. Accumulated execution time t30 of process A3. Similarly, the accumulated execution time at the end of the test is registered for the process that existed at the end of the test as follows. Accumulated execution time t11 of process A1. Accumulated execution time t20 of process A2. Accumulated execution time t41 of process A4.

  For example, the computing unit 1b refers to the recorded information 3 and compares the accumulated execution time at the start of the test (first information) with the accumulated execution time at the end of the test (second information) for each process. For example, the calculation unit 1b selects, as information collection targets, processes whose accumulated execution time (first information) at the start of the test and the accumulated execution time (second information) at the end of the test are different. This is because if the accumulated execution time fluctuates during the test period, the process is likely to have been executed in connection with the processing of the software under test.

  In the example of the record information 3, the accumulated execution time of the process A1 is different at the start of the test and at the end of the test. Therefore, the calculation unit 1b sets the process A1 as an information collection target. On the other hand, the calculation unit 1b does not collect information on the process A2 having the same cumulative execution time at the start of the test and at the end of the test.

  The computing unit 1b may select a process in which only one of the first and second information is acquired as a target for collecting software information. For example, the process A4 that did not exist at the start of the test and existed at the end of the test may have been generated in connection with the processing of the software under test. This is because the process A3 that exists at the start of the test and does not exist at the end of the test may have ended in connection with the processing of the software under test. Therefore, the calculation unit 1b sets the processes A3 and A4 as information collection targets.

  According to the information processing apparatus 1, when the software test is started, the calculation unit 1 b acquires first information indicating the accumulated time for each existing process. When the software test is finished, the computing unit 1b acquires second information indicating the accumulated time for each existing process. Based on the first and second information acquired for each process, the calculation unit 1b selects a process for which information is to be collected.

  This can improve the efficiency of the test. Here, for example, if you select the process for which information is to be collected and narrow down the information to be collected in the event of an error in advance, it will be better than collecting information on the extra process (which is not clearly related to software testing). , Analysis of cause investigation can be made more efficient. Therefore, in the software test, there is a problem of how to efficiently select a process as an information collection target from among many software processes existing on the information processing apparatus 1.

  For example, during software testing, processes such as logs and trace data are comprehensively acquired for all existing processes, and the acquired information is analyzed and executed in connection with the tests. It is also possible to select. However, when trying to acquire information comprehensively, the load on the information processing apparatus 1 associated with the process of acquiring the information becomes a problem. When the load on the information processing apparatus 1 increases, there is a possibility that the software to be tested is not properly processed. Moreover, if information is acquired comprehensively, the amount of information acquired can also increase. If the amount of information to be analyzed increases, the burden on the user in the analysis work may become excessive.

  Therefore, the information processing apparatus 1 acquires the accumulated execution time of each process at the start of the test and at the end of the test, and selects a process for which information is to be collected based on the accumulated execution time. Since the acquired change in the accumulated execution time indicates the actual performance of the process, it can be inferred that the process in which the accumulated execution time has changed during the test is the process executed in connection with the test. Therefore, if a process is selected based on the accumulated execution time and information is collected, a process that is clearly unrelated to the test can be excluded from the information collection target.

  Further, for example, the cumulative execution time is information stored in the storage unit 1 a for each process by the operating system on the information processing apparatus 1. The calculation unit 1b may read the cumulative execution time of the corresponding process from the storage unit 1a in order to acquire the cumulative execution time. For this reason, it is not necessary to acquire information comprehensively as described above, and the load accompanying acquisition of information for process selection can be kept relatively small. Therefore, it is possible to reduce the influence of the load of the information processing apparatus 1 on the test. In addition, since the process to be collected is selected based on the accumulated execution time of each process at the start of the test and at the end of the test, the user is not forced to select the process. Therefore, labor saving of the user's work can be achieved.

  In particular, when the software to be tested realizes a predetermined function in cooperation with other software, it may be tested whether the cooperation is properly performed (sometimes called a system test). In the system test, an error generated in cooperation with other software is analyzed. However, other software may be created by a developer or vendor different from itself. For this reason, if other software is black-boxed, it is not easy to know in advance what process will be executed. Therefore, in such a case, it is often the case that the software to be tested is actually moved to confirm the cooperation with other software. According to the information processing apparatus 1, even when performing such a black box test, it is possible to efficiently select a process for which information is to be collected, and to improve the efficiency of the test.

  The information processing apparatus 1 can collect information such as a trace log and a memory dump by specifying the selected process when the test target software is tested again. That is, the information collection range can be narrowed down to the selected process. For this reason, the load required for the information collection can be reduced. Therefore, the influence of the load of the information processing apparatus 1 on the test can be reduced during the test. Also, do not collect information about processes that are not clearly related to software testing (extra information). Therefore, the amount of information to be analyzed can be reduced, and the labor of analysis work can be saved.

[Second Embodiment]
FIG. 2 is a diagram illustrating a hardware example of the test apparatus according to the second embodiment. The test apparatus 100 is a computer that performs a test on software to be tested. In the second embodiment, a system test for the software is assumed. In the system test, for example, it is confirmed that the software to be tested and other software (including the OS and device driver) can cooperate to realize the designed function. However, this does not prevent the following functions from being applied to other than system tests. For example, it can be applied to unit tests and integration tests.

  The test apparatus 100 includes a processor 101, a RAM 102, an HDD 103, a communication unit 104, an image signal processing unit 105, an input signal processing unit 106, a disk drive 107, and a device connection unit 108. Each unit is connected to the bus of the test apparatus 100.

  The processor 101 controls information processing of the test apparatus 100. The processor 101 is, for example, a CPU, DSP, ASIC, or FPGA. The processor 101 may be a multiprocessor. The processor 101 may be a combination of two or more elements of CPU, DSP, ASIC, or FPGA.

  The RAM 102 is a main storage device of the test apparatus 100. The RAM 102 temporarily stores at least part of an OS program and application programs to be executed by the processor 101. The RAM 102 stores various data used for processing by the processor 101.

  The HDD 103 is an auxiliary storage device of the test apparatus 100. The HDD 103 magnetically writes and reads data to and from the built-in magnetic disk. The HDD 103 stores an OS program, application programs, and various data. The test apparatus 100 may include other types of auxiliary storage devices such as flash memory and SSD (Solid State Drive), or may include a plurality of auxiliary storage devices.

The communication unit 104 is an interface that can perform wireless communication with another computer via the network 10. The communication unit 104 may be a wired interface.
The image signal processing unit 105 outputs an image to the display 11 connected to the test apparatus 100 in accordance with an instruction from the processor 101. As the display 11, a CRT (Cathode Ray Tube) display, a liquid crystal display, or the like can be used.

  The input signal processing unit 106 acquires an input signal from the input device 12 connected to the test apparatus 100 and outputs the input signal to the processor 101. As the input device 12, for example, a pointing device such as a mouse or a touch panel, a keyboard, or the like can be used.

  The disk drive 107 is a drive device that reads a program and data recorded on the optical disk 13 using a laser beam or the like. As the optical disc 13, for example, a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc Read Only Memory), a CD-R (Recordable) / RW (ReWritable), or the like can be used. For example, the disk drive 107 stores the program and data read from the optical disk 13 in the RAM 102 or the HDD 103 in accordance with an instruction from the processor 101.

  The device connection unit 108 is a communication interface for connecting peripheral devices to the test apparatus 100. For example, the memory device 14 and the reader / writer device 15 can be connected to the device connection unit 108. The memory device 14 is a recording medium equipped with a communication function with the device connection unit 108. The reader / writer device 15 is a device that writes data to the memory card 16 or reads data from the memory card 16. The memory card 16 is a card-type recording medium. For example, the device connection unit 108 stores a program or data read from the memory device 14 or the memory card 16 in the RAM 102 or the HDD 103 in accordance with an instruction from the processor 101.

  FIG. 3 is a diagram illustrating an example of functions of the test apparatus. The test apparatus 100 includes a storage unit 110, an OS 120, a test processing unit 130, an information collection unit 140, test target software 150, and a software group 160. The storage unit 110 can be realized using a predetermined storage area of the RAM 102 or the HDD 103. The test processing unit 130 and the information collecting unit 140 may be modules of a program executed by the processor 101.

  The storage unit 110 stores information used for processing of each unit of the test apparatus 100. The storage unit 110 stores an accumulated time (accumulated execution time) executed for each process existing at the present time. The information stored in the storage unit 110 includes a test management table, a start table, and an end table.

  The test management table is a table showing the correspondence between the software to be tested and the test contents. The test content includes one or more test items and the execution order of each test item. The start table and the end table are tables used for recording the accumulated execution time for each process.

  The OS 120 is basic software that manages the execution of various types of software. For example, the OS 120 manages creation, waiting for execution, execution, and termination of each software process. A process is one of execution units by the OS 120 of a program included in software. Further, the OS 120 stores the accumulated execution time for each process existing at the present time in the storage unit 110. The OS 120 constantly measures the accumulated execution time of the existing process and updates the accumulated execution time recorded in the storage unit 110.

For example, Unix (registered trademark), Linux (registered trademark), Windows (registered trademark), or the like can be used as the OS 120.
The test processing unit 130 refers to the test management table and executes a test according to the test target software 150. For example, the test processing unit 130 inputs predetermined data to the test target software 150 and obtains output from the test target software 150 or the software group 160, thereby sequentially executing the test items registered in the test management table. . The test processing unit 130 determines whether or not the test is successful by determining whether or not the output corresponding to the input is appropriate. The test processing unit 130 proceeds with the test as follows. First, a test for selecting a process for collecting information is performed (first time). Next, a test for collecting information on the selected process is performed (from the second time on).

  In the first test by the test processing unit 130, the information collection unit 140 selects a process to be collected information for each test item. Specifically, the information collection unit 140 acquires the cumulative execution time for the process that existed at the start of the test for each test item. The information collecting unit 140 records the accumulated execution time for each process at the time of starting the test in the start table stored in the storage unit 110. The information collection unit 140 acquires the accumulated execution time for the process that existed at the end of the test for each test item. The information collection unit 140 records the accumulated execution time for each process at the end of the test in the end table stored in the storage unit 110.

  The accumulated execution time at each time point of each process is stored in the storage unit 110 by the OS 120 as described above. For example, the information collection unit 140 can acquire the cumulative execution time of the process by issuing a predetermined command designating the process to the OS 120 at each time point. For example, if the OS 120 is Unix or Linux, the accumulated execution time for each process may be acquired using the ps command. Alternatively, if the OS 120 is Windows, the accumulated execution time for each process may be acquired using the GetProcessTimes command.

The information collection unit 140 selects a process for which information is to be collected for each test item based on the accumulated execution time at the start and end of the test for each process.
The information collection unit 140 collects error analysis information for the selected process for each test item in the second and subsequent tests by the test processing unit 130. Specifically, the information collection unit 140 collects a memory dump for each process (an arbitrary area on the RAM 102 output as a file on the HDD 103) and trace data as error analysis information. For example, if the OS 120 is Unix, trace data for each process may be acquired using a truss command. Alternatively, if the OS 120 is Linux, trace data for each process may be acquired using a trace command.

  The test target software 150 is software to be tested. The test target software 150 realizes a predetermined function in cooperation with the OS 120 and the software group 160.

  The software group 160 is a set of software other than the test target software 150 that is executed on the OS 120. The software group 160 includes various device drivers that control the hardware of the test apparatus 100. The software group 160 includes middleware serving as a base for executing the test target software 150. The software group 160 includes other applications that communicate with the test target software 150.

  FIG. 4 is a diagram showing process state transitions. The state of the process is managed by the OS 120 as follows. For example, the process states include a running state 21 (Running), a blocked state 22 (Blocked), and an executable state 23 (Ready). The executing state 21 is a state in which the process is being executed by the processor 101. The block state 22 is a state in which the process cannot be executed until a predetermined external event occurs. The executable state 23 is a state waiting for the allocation time of the processor 101 scheduled for the process. However, in addition to the above states, processes may be swapped to virtual memory.

  For example, (1) in a series of steps of a process, when it becomes a stage waiting for occurrence of a predetermined external event (resource reservation, input from another process, etc.), a transition is made from the running state 21 to the block state 22. (2) When the first process is in the executing state 21 and the processor allocation time of the second process is reached, the first process transits to the executable state 23. (3) The second process that was in the executable state 23 transitions to the executing state 21. (4) A process in the block state 22 transitions from the block state 22 to the executable state 23 when a predetermined external event occurs.

  For example, the generated process is in an executable state 23. When a series of steps of the process ends, resources such as the processor 101, the RAM 102, and the HDD 103 for the process are released, and the process disappears.

  The block state 22, the executable state 23, and the swapped state exist on the RAM 102 and the HDD 103 as processes, but can be said to be waiting for execution. For example, the accumulated execution time described above may be a time when each process is in the executing state 21. Alternatively, the cumulative execution time may be the sum of the times in the executing state 21 and the executable state 23.

  FIG. 5 is a diagram illustrating an example of a process execution pattern during a test. In FIG. 5, the solid line indicates that the process has been executed continuously or intermittently (it was in the running state 21). A broken line indicates that the process is not being executed (the state 21 is not being executed).

  The execution pattern P1 is a pattern in which a process is executed from the start of the test to the end of the test. It can be determined that the process of the execution pattern P1 is highly likely to have been executed in connection with the test of the test target software 150.

  The execution pattern P2 is a pattern in which a process is executed from the start of the test to a certain point in the middle of the test, but the process is ended by the end of the test. The process of the execution pattern P <b> 2 may have been terminated in connection with the test of the test target software 150.

  The execution pattern P3 is a pattern that does not exist at the start of the test, a process is generated at a certain point during the test, and the process is executed until the end of the test. The process of the execution pattern P3 may have been generated and executed in connection with the test of the test target software 150.

  The execution pattern P4 does not exist at the start of the test and at the end of the test, is a pattern that is generated during the test and ends by the end of the test. The process of the execution pattern P4 may have been generated and executed in connection with the test of the test target software 150.

  The execution pattern P5 is a pattern in which the process is not executed between the start of the test and the end of the test. It can be determined that the process of the execution pattern P5 is not executed in relation to the test of the test target software 150.

  FIG. 6 is a diagram illustrating an example of a candidate process table. The candidate process table 111 is information in which process candidates for information collection are registered for each software to be tested. For example, when the user wants to specify a candidate process for which information is to be collected, the user registers it in the candidate process table 111. Candidate process table 111 is stored in storage unit 110 in advance. The candidate process table 111 includes items of software name and process name.

The name of the software to be tested is registered in the software name item. In the process name item, the name of a process as a candidate for information collection is registered.
For example, in the candidate process table 111, information that the software name is “SW1” and the process name is “smss, providerMGR, providermonitor, subscriber, ccSvcHst, printMGR,...” Is registered. This indicates that when the test target software 150 is “SW1”, the target process for information collection is “smss, providerMGR, providermonitor, stubprovider, ccSvcHst, printMGR,. Here, the software name “SW1” is the software name of the test target software 150.

  FIG. 7 is a diagram illustrating an example of the exclusion process table. The excluded process table 112 is information in which process candidates to be excluded from information collection are registered for each piece of software to be tested. For example, the user registers in the excluded process table 112 when he / she wants to designate a process candidate to be excluded from information collection. The exclusion process table 112 is stored in the storage unit 110 in advance. The excluded process table 112 includes items of software name and process name.

The name of the software to be tested is registered in the software name item. In the process name field, the name of a process that is not subject to information collection is registered.
For example, in the excluded process table 112, information that the software name is “SW1” and the process name is “System Idle Process, System, svhost,...” Is registered. This indicates that when the test target software 150 is “SW1”, the process that is not subject to information collection is “System Idle Process, System, svhost, System, svhost,...”.

  FIG. 8 is a diagram illustrating an example of a test management table. The test management table 113 is information indicating a correspondence relationship between test items for each software to be tested and information used in the test items. The test management table 113 is stored in the storage unit 110 in advance. The test management table 113 includes software names, test items, and test information items.

  In the software name item, the name of the software to be tested is registered. Identification information of the test item is registered in the item of the test item. In the test information item, input data used in the test item, information for generating an event, information for determining whether or not the test is successful, and the like are registered.

  For example, in the test management table 113, information that the software name is “SW1”, the test item is “T1”, and the test information is “test1” is registered. In addition, information that the software name is “SW1”, the test item is “T2”, and the test information is “test2” is registered. This indicates that the test item “T1” and the test item “T2” are sequentially tested for the test target software 150. In addition, when the test item “T1” is tested, “test1” is used as the test information. When the test item “T2” is tested, “test2” is used as the test information.

  FIG. 9 is a diagram illustrating an example of the start table. The start table 114 is used for recording the accumulated execution time for each process at the start of the test. The start table 114 is stored in the storage unit 110. The start table 114 includes items of software name, test item, process name, and cumulative execution time.

  The name of the software to be tested is registered in the software name item. Identification information of the test item is registered in the item of the test item. In the process name item, the process name for which the cumulative execution time is to be acquired is registered. The accumulated execution time acquired at the start of the test is registered in the item of accumulated execution time.

  For example, in the start table 114, information that the software name is “SW1”, the test item is “T1”, the process name is “smss”, and the cumulative execution time is “00: 05: 45.126” is registered. This indicates that the cumulative execution time of the process “smss” at the start of the test of the test item “T1” of the test target software 150 was “00: 05: 45.126”.

  FIG. 10 is a diagram illustrating an example of the end table. The end table 115 is used to record the accumulated execution time for each process at the end of the test. The end table 115 is stored in the storage unit 110. The end table 115 includes items of software name, test item, process name, and accumulated execution time.

  The name of the software to be tested is registered in the software name item. Identification information of the test item is registered in the item of the test item. In the process name item, the process name for which the cumulative execution time is to be acquired is registered. The accumulated execution time acquired at the end of the test is registered in the item of accumulated execution time.

  For example, information indicating that the software name is “SW1”, the test item is “T1”, the process name is “smss”, and the cumulative execution time is “00: 12: 45.877” is registered in the end table 115. This indicates that the accumulated execution time of the process “smss” at the end of the test of the test item “T1” of the test target software 150 was “00: 12: 45.877”.

  FIG. 11 is a diagram illustrating an example of a comparison table. The comparison table 116 is information that combines the start table 114 and the end table 115, and is used for comparison between the two. The comparison table 116 is stored in the storage unit 110. The comparison table 116 includes items of software name, test item, process name, and cumulative execution time (at start time and end time).

  The name of the software to be tested is registered in the software name item. Identification information of the test item is registered in the item of the test item. In the process name item, the process name for which the accumulated execution time is acquired is registered. In the item of accumulated execution time (at the start), the accumulated execution time acquired at the start of the test is registered. The accumulated execution time acquired at the end of the test is registered in the item of accumulated execution time (at the end).

  For example, in the comparison table 116, the software name is “SW1”, the test item is “T1”, the process name is “smss”, the start time of the cumulative execution time is “00: 05: 45.126”, and the cumulative execution time is Information that the end time is “00: 12: 45.877” is registered. This is because the accumulated execution time at the start of the test of the process “smss” when the test of the test item “T1” of the test target software 150 is performed is “00: 05: 45.126”, and the accumulated execution time at the end of the test. Is “00: 12: 45.877”. Since the accumulated execution time at the start of the test is different from the accumulated execution time at the end of the test, it can be seen that the process “smss” was executed during the test. That is, the process “smss” is a process of the execution pattern P1.

  Further, for example, in the comparison table 116, the software name is “SW1”, the test item is “T1”, the process name is “providerMGR”, the cumulative execution time starts at “00: 01: 12.122”, and the cumulative execution Information indicating that there is no registration at the end of the time ("-(hyphen)") is registered. This indicates that the cumulative execution time at the start of the test of the process “providerMGR” when the test of the test item “T1” of the test target software 150 is performed is “00: 01: 12.122”. Further, since the process “providerMGR” has been completed (has not existed) at the end of the test, the information collection unit 140 cannot acquire the accumulated execution time. The process “providerMGR” was executed at the start of the test, but it can be determined that the process has ended during the test. That is, the process “providerMGR” is a process of the execution pattern P2.

  Further, for example, in the comparison table 116, the software name is “SW1”, the test item is “T1”, the process name is “providermonitor”, the cumulative execution time start time is not registered, and the cumulative execution time end time is “00”. : 00: 00.098 "is registered. This indicates that the accumulated execution time at the end of the test of the process “providermonitor” when the test of the test item “T1” of the test target software 150 is performed is “00: 00: 0.098”. Further, since the process “providermonitor” was not generated at the start of the test (it did not exist), the information collection unit 140 could not acquire the accumulated execution time. The process “providermonitor” was not generated at the start of the test, but can be determined to have been generated and executed during the test. That is, the process “providermonitor” is a process of the execution pattern P3.

  Further, for example, in the comparison table 116, the software name is “SW1”, the test item is “T1”, the process name is “ccSvcHst”, and the start time of the cumulative execution time is “00: 00: 08.356”. Information that the time ends is “00: 00: 08.356” is registered. This is because the accumulated execution time at the start of the test of the process “ccSvcHst” when the test of the test item “T1” of the test target software 150 is performed is “00: 00: 08.356”, and the accumulated execution time at the end of the test. Is “00: 00: 08.356”. Since the cumulative execution time at the start of the test is the same as the cumulative execution time at the end of the test, it can be determined that the process “ccSvcHst” has not been executed during the test. That is, the process “ccSvcHst” is a process of the execution pattern P5.

  FIG. 12 is a diagram illustrating an example of a process history table. The process history table 117 is stored in the storage unit 110. The process history table 117 includes items of process name, log ID (IDentifier), log generation time, type, and detailed information.

  In the process name item, the name of the process is registered. In the log ID item, a log ID is registered. The log generation time is registered in the log generation time item. A log type is registered in the type item. For example, when process generation is indicated, the type is “generation”. When indicating an error relating to a process, the type is “error”. When indicating the end of the process, the type is “End”. In the detailed information item, information indicating the contents when other processes communicating with the generated process cause an abnormality is registered.

  The detailed information includes an error code, a related process ID, a related process name, and a detection location code. The error code is information for identifying the abnormal content. The related process ID is the process ID of the other process. The related process name is a process name corresponding to the process ID. The detection location code is information indicating a step in which an abnormality is detected in the other process.

  For example, in the process history table 117, information that the process name is “stubprovider”, the log ID is “L1”, the log generation time is “15: 16: 17.203”, and the type is “generation” is registered. This indicates that a process with the process name “stubprovider” is generated at time “15: 16: 17.203”.

  Further, for example, in the process history table 117, the process name is “stubprovider”, the log ID is “L2”, the log generation time is “15: 16: 18.358”, the type is “error”, and the error code is “1275”. ", The information that the related process ID is" 1054 ", the related process name" rcvy ", and the detection location code" 3388 "is registered.

  This indicates that an error occurred at the time “15: 16: 18.358” in the process with the process name “stubprovider”. Further, in the error, the process “rcvy” communicating with the process “stubprovider” has caused an abnormality indicated by the error code “1275”, and the step having the abnormality is indicated by “3388”. It shows that.

  The log recorded in the process history table 117 may be generated by the information collection unit 140, or may be a log collected by the OS 120 or other software.

  FIG. 13 is a diagram illustrating an example of a process table. The process table 118 is a table for registering a process selected as an information collection target. The process table 118 is stored in the storage unit 110. The process table 118 includes items of item number, test start time, test end time, software name, test item, and process name.

  In the item number item, a number for identifying the record is registered. In the test start time item, the time when the test of the corresponding test item is started is registered. In the test end time item, the time when the test of the corresponding test item is ended is registered. Although the hour, minute, and second are recorded as the test start time and the test end time, the date may be recorded. The name of the software to be tested is registered in the software name item. Identification information of the test item is registered in the item of the test item. In the process name item, the name of the process whose information is to be collected is registered.

  For example, in the process table 118, the item number is “1”, the test start time is “15:16:16”, the test end time is “15:19:17”, the software name is “SW1”, and the test item is “ Information of “T1” and process names “smss, providerMGR, providermonitor, stubprovider, rcvy,...” Is registered.

  This is because, in the test of the test item “T1” of the test target software 150 performed at the time “15:16:16 to 15:19:17”, “smss, providerMGR, providermonit, tubeprovider, rcvy,... "is selected.

FIG. 14 is a flowchart illustrating an example of a software test process. In the following, the process illustrated in FIG. 14 will be described in order of step number.
(Step S11) The test processing unit 130 accepts designation of the test target software 150 as a test target. For example, the software to be tested is specified by the user.

  (Step S12) The information collection unit 140 accepts designation of a process selection mode. The process selection mode indicates a method for selecting a process for collecting information. There are two types of process selection modes: “process designation” and “all process designation”. “Process designation” is a mode for selecting a process for collecting information by narrowing down processes from candidate processes designated by the user (candidate process table 111). “Specify all processes” is a mode for selecting all processes presumed to be executed in connection with the test from all processes. In “specify all processes”, it is possible to set processes to be excluded from information collection by the excluded process table 112. The user can select either method according to the test.

  (Step S <b> 13) The test processing unit 130 performs a first test on the test target software 150. The information collection unit 140 performs process selection processing. In the process selection process, a process for which information is to be collected is selected in subsequent steps.

  (Step S14) The test processing unit 130 performs a second test on the test target software 150. The information collection unit 140 acquires a memory dump for the process selected in step S13.

  (Step S15) The test processing unit 130 performs a third test on the test target software 150. The information collection unit 140 acquires trace data for the process selected in step S13.

Next, the procedure of the process selection process in step S13 will be described.
FIG. 15 is a flowchart illustrating an example of process selection processing. In the following, the process illustrated in FIG. 15 will be described in order of step number.

(Step S21) The test processing unit 130 substitutes 1 for the variable n.
(Step S22) The test processing unit 130 refers to the test management table 113 stored in the storage unit 110, and acquires test information of the test item Tn. For example, if the test item “T1” of the test target software 150 (“SW1”), the test information “test1” is acquired.

(Step S <b> 23) The test processing unit 130 notifies the information collecting unit 140 of the test start of the test item Tn for the test target software 150.
(Step S24) The information collection unit 140 performs a test start process. Details will be described later. The information collecting unit 140 registers the test start time, software name, and test item in the process table 118 stored in the storage unit 110.

(Step S25) The test processing unit 130 tests the test item Tn for the test target software 150 using the test information acquired in Step S22.
(Step S <b> 26) The information collection unit 140 collects logs of processes that have been generated, errored, and ended during the test, and records them in the process history table 117 stored in the storage unit 110. The information collection unit 140 may record the log in the process history table 117 in association with the software name or the test item.

  (Step S27) The test processing unit 130 ends the test of the test item Tn for the test target software 150. The test processing unit 130 notifies the information collection unit 140 of the test end of the test item Tn.

  (Step S28) The information collection unit 140 performs a test end process. Details will be described later. The information collection unit 140 registers the test end time of the test in the process table 118.

(Step S29) The information collection unit 140 performs a comparison process of the accumulated execution time for each process acquired in steps S24 and S28. Details will be described later.
(Step S30) The test processing unit 130 refers to the test management table 113 and determines whether all the test items have been tested. If all have been tested, the process ends. If there is an unexecuted test, the process proceeds to step S31. For example, in the case of the test target software 150 (“SW1”), with reference to the test management table 113, if both of the test items “T1” and “T2” have been tested, all have been tested.

(Step S31) The test processing unit 130 adds 1 to the variable n. Then, the process proceeds to step S22.
Next, the procedure of the test start process in step S24 will be described.

FIG. 16 is a flowchart illustrating an example of a test start process. In the following, the process illustrated in FIG. 16 will be described in order of step number.
(Step S41) The information collection unit 140 acquires a process that exists at the time of step S41. For example, the information collection unit 140 can acquire the process ID and process name of an existing process by issuing a predetermined command to the OS 120.

  (Step S42) The information collection unit 140 determines a process selection mode. In the case of “process designation”, the process proceeds to step S43. If “all processes are specified”, the process proceeds to step S44.

  (Step S43) The information collection unit 140 refers to the candidate process table 111 stored in the storage unit 110, and acquires a process corresponding to the test target software. The information collection unit 140 narrows down to processes that exist in both the process acquired in step S41 and the process acquired from the candidate process table 111. For example, it is assumed that the process acquired in step S41 is a process “smss, providerMGR, ccSvcHst,. According to the candidate process table 111, candidate processes corresponding to the software name “SW1” are “smss, providerMGR, providermonitor, subscriber, ccSvcHst, printMGR,. Therefore, the information collection unit 140 narrows down to processes “smss, providerMGR, ccSvcHst,. Then, the process proceeds to step S45.

  (Step S44) The information collection unit 140 refers to the exclusion process table 112 stored in the storage unit 110, and acquires an exclusion target process. The information collection unit 140 narrows down the processes by excluding the processes to be excluded from the processes identified in step S41. According to the exclusion process table 112, the exclusion process corresponding to the software name “SW1” is “System Idle Process, System, svhost,...”. Therefore, the information collection unit 140 excludes the exclusion process from the process acquired in step S41. Then, the process proceeds to step S45.

(Step S45) The information collection unit 140 registers the processes narrowed down in step S43 or step S44 in the start table 114 stored in the storage unit 110.
(Step S46) The information collection unit 140 acquires the accumulated execution time for each process registered in the start table 114 in step S45, and registers it in the start table 114. The OS 120 stores the accumulated execution time of the existing process in the storage unit 110. The information collection unit 140 can acquire the accumulated execution time for each process by referring to the storage unit 110. The information collection unit 140 may acquire a cumulative execution time for each process by issuing a predetermined command to the OS 120 as described above.

Next, the procedure of the test end process in step S28 of FIG. 15 will be described.
FIG. 17 is a flowchart illustrating an example of test end processing. In the following, the process illustrated in FIG. 17 will be described in order of step number.

  (Step S51) The information collection unit 140 acquires a process existing at the time of step S51. For example, the information collection unit 140 can acquire the process ID and process name of an existing process by issuing a predetermined command to the OS 120.

  (Step S52) The information collection unit 140 determines a process selection mode. In the case of “process designation”, the process proceeds to step S53. If “all processes are specified”, the process proceeds to step S54.

  (Step S53) The information collection unit 140 refers to the candidate process table 111 stored in the storage unit 110, and acquires a process corresponding to the test target software. The information collection unit 140 narrows down the processes that exist in both the process acquired in step S51 and the process acquired from the candidate process table 111. A specific method is the same as that in step S43. Then, the process proceeds to step S55.

  (Step S <b> 54) The information collection unit 140 refers to the exclusion process table 112 stored in the storage unit 110 and acquires an exclusion target process. The information collection unit 140 narrows down the processes by excluding the processes to be excluded from the processes identified in step S51. A specific method is the same as that in step S44. Then, the process proceeds to step S55.

(Step S55) The information collection unit 140 registers the processes narrowed down in step S53 or step S54 in the end table 115 stored in the storage unit 110.
(Step S56) The information collection unit 140 acquires the accumulated execution time for each process registered in the end table 115 in step S55, and registers it in the end table 115. The OS 120 stores the accumulated execution time of the existing process in the storage unit 110. The information collection unit 140 can acquire the accumulated execution time for each process by referring to the storage unit 110. The information collection unit 140 may acquire a cumulative execution time for each process by issuing a predetermined command to the OS 120 as described above.

Next, the comparison processing procedure in step S29 in FIG. 15 will be described.
FIG. 18 is a flowchart illustrating an example of comparison processing. Hereinafter, the process illustrated in FIG. 18 will be described in order of step number.

(Step S61) The information collection unit 140 creates the comparison table 116 based on the start table 114 and the end table 115.
(Step S <b> 62) The information collection unit 140 extracts one process from the comparison table 116.

  (Step S63) The information collection unit 140 determines whether the accumulated execution time is registered at both the test start time and the test end time for the extracted process. If registered, the process proceeds to step S64. If not registered, the process proceeds to step S65.

  (Step S64) The information collection unit 140 determines whether the accumulated execution time at the test start point and the test end point are the same. If they are the same, the process proceeds to step S66. If not, the process proceeds to Step S65.

  (Step S65) The information collection unit 140 registers the process extracted in step S62 in the process table 118. In step S65, processes of execution patterns P1, P2, and P3 can be registered.

  (Step S66) The information collection unit 140 determines whether all processes in the comparison table 116 have been confirmed for the current test of the test item Tn of the test target software 150. If it has been confirmed, the process proceeds to step S67. If not confirmed, the process proceeds to step S62.

  (Step S67) The information collection unit 140 refers to the process history table 117 and determines whether or not there is a process log generated during the test. If there is a process generation log, the process proceeds to step S68. If there is no log, the process is terminated. In the “process designation” mode, when the process generated during the test is also a candidate process (registered in the candidate process table 111), the process proceeds to step S68 (if it is not a candidate process, End processing). In the “all process designation” mode, if the process generated during the test is not an excluded process (not registered in the excluded process table 112), the process proceeds to step S68 (if it is an excluded process, the process is performed). Exit). The time zone under test is registered in the process table 118.

  (Step S68) The information collection unit 140 registers the process specified in step S67 (process generated during the test) in the process table 118. In step S68, the process of execution pattern P4 is registered.

  (Step S <b> 69) The information collection unit 140 refers to the process history table 117 and identifies a process that has an error during the test among the processes registered in the process table 118. The information collection unit 140 determines whether there is a registration of a related process in the detailed information of the identified process log. If there is registration, the process proceeds to step S70. If there is no registration, the process ends.

(Step S70) The information collection unit 140 registers the related process identified in Step S69 in the process table 118.
Next, the procedure of the memory dump acquisition process in step S14 in FIG. 14 will be described.

FIG. 19 is a flowchart illustrating an example of a memory dump acquisition process. In the following, the process illustrated in FIG. 19 will be described in order of step number.
(Step S71) The test processing unit 130 substitutes 1 for the variable n.

(Step S <b> 72) The test processing unit 130 refers to the test management table 113 stored in the storage unit 110 and acquires test information of the test item Tn.
(Step S <b> 73) The test processing unit 130 starts testing the test item Tn with respect to the test target software 150.

(Step S74) The test processing unit 130 ends the test of the test item Tn for the test target software 150.
(Step S75) The test processing unit 130 determines whether or not the test result is invalid (error) based on the test information acquired in Step S72. If the test result is invalid, the information collection unit 140 is notified of this fact, and the process proceeds to step S76. If the test result is appropriate, the process proceeds to step S77.

  (Step S76) The information collection unit 140 refers to the process table 118 and identifies the process of the test item Tn corresponding to the test target software 150 (“SW1”). The information collection unit 140 outputs a memory dump related to the identified process as a file to a predetermined storage area of the HDD 103.

  (Step S77) The test processing unit 130 refers to the test management table 113 and determines whether all the test items have been tested. If all have been tested, the process ends. If there is an unexecuted test, the process proceeds to step S78.

(Step S78) The test processing unit 130 adds 1 to the variable n. Then, the process proceeds to step S72.
Next, the procedure of the trace data acquisition process in step S15 in FIG. 14 will be described.

FIG. 20 is a flowchart illustrating an example of the trace data acquisition process. In the following, the process illustrated in FIG. 20 will be described in order of step number.
(Step S81) The test processing unit 130 substitutes 1 for the variable n.

(Step S82) The test processing unit 130 refers to the test management table 113 stored in the storage unit 110, and acquires test information of the test item Tn.
(Step S83) The test processing unit 130 instructs the information collecting unit 140 to start acquiring trace data for the test item Tn of the test target software 150. The information collection unit 140 refers to the process table 118 and identifies the process of the test item Tn corresponding to the test target software 150 (“SW1”). The information collection unit 140 starts acquiring trace data for the identified process.

(Step S84) The test processing unit 130 starts a test of the test item Tn with respect to the test target software 150.
(Step S85) The test processing unit 130 ends the test of the test item Tn with respect to the test target software 150.

(Step S86) The test processing unit 130 instructs the information collection unit 140 to end the acquisition of trace data. The information collection unit 140 ends the acquisition of trace data.
(Step S87) The test processing unit 130 determines whether or not the test result is invalid (error) based on the test information acquired in Step S82. If the test result is invalid, the information collection unit 140 is notified of this fact, and the process proceeds to step S88. If the test result is appropriate, the process proceeds to step S89.

  (Step S88) The information collection unit 140 outputs the trace data for each program acquired during steps S83 to S86 as a file to a predetermined storage area of the HDD 103.

  (Step S89) The test processing unit 130 refers to the test management table 113 and determines whether all the test items have been tested. If all have been tested, the process ends. If there is an unexecuted test, the process proceeds to step S90.

(Step S90) The test processing unit 130 adds 1 to the variable n. Then, the process proceeds to step S82.
FIG. 21 is a sequence diagram illustrating a specific example of the process selection process. In the following, the process illustrated in FIG. 21 will be described in order of step number.

(Step ST101) The test processing unit 130 notifies the information collecting unit 140 of the start of the test for the test item T1. The information collection unit 140 receives the test start notification.
(Step ST102) The information collection unit 140 performs a test start process. Specifically, the accumulated execution time is acquired for the process existing at the time.

(Step ST103) The test processing unit 130 starts a test of the test item T1 with respect to the test target software 150.
(Step ST104) The test processing unit 130 ends the test of the test item T1 for the test target software 150.

(Step ST105) The test processing unit 130 notifies the information collecting unit 140 of the test end of the test item T1. The information collection unit 140 receives a notification of the end of the test.
(Step ST106) The information collection unit 140 performs a test end process. Specifically, the accumulated execution time is acquired for the process existing at the time.

  (Step ST107) The information collection unit 140 performs a comparison process. Specifically, based on the accumulated execution time for each process acquired in steps ST102 and ST106, a process to be collected information is selected when testing the test item T1 of the test target software 150.

(Step ST108) The test processing unit 130 notifies the information collection unit 140 of the test start of the test item T2.
Then, the test apparatus 100 repeats the processing from step ST101 to step ST107 until all the test items are completed. In this way, a process for which information is to be collected is selected for each test item.

  FIG. 22 is a diagram illustrating a specific example of process selection. FIG. 22 illustrates a process registered in the process table 118 for the test item T1. The information collection unit 140 executes the procedure of FIG. 18 on the basis of information registered in the start table 114, the end table 115, and the process history table 117, thereby collecting information on the test item T1 of the test target software 150. And get the process.

  For example, the cumulative execution time of the process “smss” is different at the test start time and test end time. That is, the process “smss” is a process of the execution pattern P1. Therefore, the information collection unit 140 selects the process “smss” as an information collection target.

  For the process “providerMGR”, the accumulated execution time at the start of the test is acquired, but the accumulated execution time at the end of the test is not acquired. This indicates that the process “providerMGR” was executed and ended from the test start time to the test end time. That is, the process “providerMGR” is a process of the execution pattern P2. Therefore, the information collection unit 140 selects the process “providerMGR” as an information collection target.

  For the process “providermonitor”, the accumulated execution time at the start of the test is not acquired, but the accumulated execution time at the end of the test is acquired. This indicates that the process “providermonitor” is generated and executed between the test start time and the test end time. That is, the process “providermonitor” is a process of the execution pattern P3. Therefore, the information collection unit 140 selects the process “providermonitor” as an information collection target.

  Further, since the accumulated execution time of the process “ccSvcHst” is the same at the start of the test and at the end of the test, it indicates that the process “ccSvcHst” has not been executed during the test. That is, the process “ccSvcHst” is a process of the execution pattern P5. Therefore, the information collection unit 140 does not set the process “ccSvcHst” as an information collection target.

  Furthermore, the information collection unit 140 refers to the process history table 117 and identifies the process “stubprovider” generated during the test. The process “stubprovider” is not registered in the start table 114 and the end table 115 because the process “stubprovider” does not exist at the start and end of the test. This indicates that the process “providermonitor” is generated, executed, and terminated between the test start time and the test end time. That is, the process “providermonitor” is a process of the execution pattern P4. Therefore, the information collection unit 140 selects the process “stubprovider” as an information collection target.

  Further, the information collection unit 140 refers to the process history table 117 and identifies that the process related to the process “stubprovider” has caused an abnormality during the test. The information collection unit 140 selects the related process “rcvy” as an information collection target.

  The information collection unit 140 selects the processes “smss, providerMGR, providermonitor, stubprovider, rcvy” to be collected as described above, and registers them in the process table 118. In this way, the information collection unit 140 selects a process whose information is to be collected for each test target software and each test item. Then, the information collection unit 140 collects information about the selected process.

FIG. 23 is a sequence diagram illustrating a specific example of information collection processing. In the following, the process illustrated in FIG. 23 will be described in order of step number.
(Step ST111) The test processing unit 130 starts a test of the test item T1 with respect to the test target software 150.

(Step ST112) The test processing unit 130 ends the test of the test item T1 for the test target software 150.
(Step ST113) The test processing unit 130 determines that the test result of the test item T1 is invalid.

(Step ST114) The test processing unit 130 instructs the information collecting unit 140 to output a memory dump. The information collection unit 140 receives the instruction.
(Step ST115) The information collection unit 140 outputs, to the HDD 103, a memory dump related to the process selected as the information collection target for the test item T1 of the test target software 150 as a file. The information collection unit 140 assigns identification information to the file so that the output file is acquired for the test item T1 of the test target software 150.

(Step ST116) The test processing unit 130 starts a test of the test item T2 with respect to the test target software 150.
Then, the processes from step ST111 to step ST115 are repeated until all test items are completed.

  (Step ST117) The test processing unit 130 instructs the information collecting unit 140 to start acquiring trace data for the test item T1 of the test target software 150. The information collection unit 140 starts obtaining trace data for the process selected as the information collection target for the test item T1 of the test target software 150.

(Step ST118) The test processing unit 130 starts testing the test item T1 with respect to the test target software 150.
(Step ST119) The test processing unit 130 ends the test of the test item T1 for the test target software 150.

  (Step ST120) The test processing unit 130 instructs the information collection unit 140 to end the acquisition of trace data. The information collection unit 140 ends the acquisition of trace data.

(Step ST121) The test processing unit 130 determines that the test result of the test item T1 is invalid.
(Step ST122) The test processing unit 130 instructs the information collecting unit 140 to output trace data. The information collection unit 140 receives the instruction.

  (Step ST123) The information collection unit 140 outputs the trace data acquired during steps ST117 to ST120 to the HDD 103 as a file. The information collection unit 140 assigns identification information to the file so that the output file is acquired for the test item T1 of the test target software 150.

(Step ST124) The test processing unit 130 instructs the information collecting unit 140 to start acquiring trace data for the test item T2 of the test target software 150.
Then, the processes from step ST117 to step ST123 are repeated until all the test items are completed.

  Here, for example, if you select a process to collect information and narrow down the information to be collected in the event of an error, the cause is more than the collection of unnecessary process information (which is not related to software testing). The analysis work of investigation can be made efficient. Therefore, in the software test, there is a problem of how to efficiently select a process as an information collection target from among many software processes existing on the test apparatus 100.

  For example, during software testing, comprehensively acquire information such as logs and trace data for all existing processes, and analyze the acquired information to identify the process related to the test. Is also possible. However, when trying to acquire information comprehensively, the load on the test apparatus 100 associated with the information acquisition process becomes a problem. When the load on the test apparatus 100 increases, there is a risk that the software to be tested is not properly processed. In addition, when information is comprehensively acquired, the amount of information can be increased. For this reason, the burden on the user in the analysis work may be excessive.

  Therefore, the test apparatus 100 acquires the accumulated execution time of each process at the start and end of the test, and selects a process related to the software test based on the accumulated execution time. For this reason, it is not necessary to comprehensively acquire information for all processes, and the load associated with the acquisition of information for process selection can be kept relatively small. Therefore, the influence of the load of the test apparatus 100 on the test can be reduced. In addition, since the process to be collected is selected based on the accumulated execution time of each process at the start of the test and at the end of the test, the user is not forced to specify the process. Therefore, labor saving of the user's work can be achieved.

  For example, when the test apparatus 100 re-executes the test, the test apparatus 100 can specify the selected process and collect information such as a trace log and a memory dump. At this time, since the information collection range can be narrowed down to the selected process, the load required for the information collection can be reduced.

  Also, if the software test is not successful, the cause is identified and the test environment is often maintained until the test is successful. When it is desired to effectively use a resource used in the test environment, it is required to finish the test in a short period of time and release the resource. In this regard, by using the second embodiment, the burden of cause analysis can be reduced, so that the possibility that the test can be completed in a short period increases.

  Further, as described above, the information collection unit 140 may acquire the accumulated execution time at the start and end of the test using an API (Application Programming Interface) provided by the OS 120. Then, the information collection unit 140 (or OS 120) can acquire the cumulative execution time for each process with reference to a predetermined area on the RAM 102, for example, and thus compares the load for acquiring information for process selection. Can be made small. That is, an extra load during the test can be reduced and the influence on the original processing for the test can be suppressed.

As described above, according to the test apparatus 100, the test efficiency can be improved.
Next, another usage example of the process table 118 will be described. In the process table 118, processes for which information is collected are registered for each software to be tested. If a test item for the same software is executed a plurality of times and the same process is selected a plurality of times, there is a high possibility that the processing is executed by the same route during the program running. Such a test item is a highly reproducible test item with high reproducibility regarding the use of resources during test execution. This is because if the reproducibility of resource usage cannot be ensured in the black box test, it will be difficult to obtain evidence that causes software errors, and accurate analysis will not be possible.

  If test items with high effectiveness can be extracted, testing efficiency can be improved. For example, a regression test accompanying a software revision is performed. Based on the test results of the software before the revision, it is conceivable to test the software after the revision focusing on highly effective test items. Therefore, a test item validity verification function based on the process table 118 is provided.

FIG. 24 is a flowchart illustrating an example of verifying the effectiveness of a test item. In the following, the process illustrated in FIG. 24 will be described in order of step number.
(Step S <b> 91) The information collection unit 140 accepts designation of software to be verified and a test item by user input. For example, the software name “SW2” and the test item “T1” are designated.

(Step S92) The information collection unit 140 refers to the process table 118 and identifies the specified software and test item.
(Step S <b> 93) The information collection unit 140 determines whether or not the processes for which information is collected match. For example, according to the process table 118, it is determined whether or not the processes included in the records of item numbers “3” and “4” match.

  (Step S <b> 94) The information collection unit 140 outputs the determination result to the display 11. Then, the process ends. The information collection unit 140 may output a matching process or a mismatching process. For example, when all processes match, the information collection unit 140 may evaluate that the test item is valid and decide to adopt the test item for the regression test. If any of the processes do not match, the information collection unit 140 may evaluate that the test item is not valid and determine not to employ the test item for the regression test.

  As described above, the information collection unit 140 tests the test item of the test target software 150 a plurality of times, and evaluates whether or not the test item is valid based on the rate at which the processes selected at each time match. . For example, it is conceivable to obtain the ratio at which the processes selected each time match as follows. First, the number of processes previously selected among the processes selected this time for a certain test item may be obtained by dividing by the total number of processes selected this time. Second, the percentage of the processes selected at each time matches, the number of processes that have been selected up to the previous time among the processes selected this time for a certain test item, and the number of processes that have been selected this time. It may be obtained by dividing by the total number. For example, in addition to determining that it is valid when it is a perfect match, it may be determined that it is valid when it matches at a predetermined ratio (for example, 90%, 95%, etc.) or more.

  As a result, the increase in the number of test items can be suppressed and the test items can be narrowed down to more effective items. Therefore, the software after the revision can be efficiently tested during the regression test.

  In addition, a regression test at the time of software revision may test whether the software is defective. For example, it is conceivable to use the process table 118 for extracting test items during a regression test.

FIG. 25 is a flowchart illustrating an example of test item extraction processing. In the following, the process illustrated in FIG. 25 will be described in order of step number.
(Step S <b> 101) The information collection unit 140 receives a test target software and process designation by a user input. A plurality of processes may be specified. For example, the information collection unit 140 accepts designation of software “SW1” and process “smss”.

  (Step S <b> 102) The information collection unit 140 refers to the process table 118 and identifies a test item based on software and a process designated by the user. The information collection unit 140 refers to the process table 118. The information collecting unit 140 identifies the test items “T1” and “T2” in which the process “smss” is registered with the software “SW1”.

(Step S <b> 103) The information collection unit 140 outputs the specified test item to the display 11. Then, the process ends.
Thus, using the process table 118, test items for the process to be executed can be easily extracted. During regression testing, the same process can be performed before and after software revisions. Therefore, by specifying the test target software and process and extracting the test items at the time of the test before the revision as described above, the test item after the revision can be efficiently created. The information collection unit 140 may generate a test item for the regression test based on the extracted test item and register it in the test management table 113. In this way, it is possible to reduce the burden of creating test items by the user.

As described above, according to the test apparatus 100, the test efficiency can be improved.
As described above, the information processing of the first embodiment can be realized by causing the computing unit 1b to execute a program. The information processing according to the second embodiment can be realized by causing the processor 101 to execute a program. The program can be recorded on a computer-readable recording medium (for example, an optical disc, a memory device, a memory card, etc.).

  When distributing the program, for example, a portable recording medium in which the program is recorded is provided. It is also possible to store the program in a storage device of another computer and distribute the program via a network. The computer stores, for example, a program recorded on a portable recording medium or a program received from another computer in a storage device, and reads and executes the program from the storage device. However, a program read from a portable recording medium may be directly executed, or a program received from another computer via a network may be directly executed.

  In addition, at least a part of the information processing described above can be realized by an electronic circuit such as a DSP, ASIC, or PLD (Programmable Logic Device).

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 1a Storage part 1b Calculation part 2 Management information 3 Record information

Claims (8)

On a computer that can run multiple processes,
When starting a software test, for each existing process, obtain first information indicating the accumulated time executed,
Upon completion of the test of the software, for each existing process, obtain second information indicating the accumulated time executed,
Based on the first and second information obtained in each process, select the process to be subjected to information collection when again performing the test of the software,
When an abnormality of the second process communicating with the first process is recorded in the history of the test related to the first process that is the target of information collection, the second process is the target of information collection. ,
A program that executes processing.   The program according to claim 1, wherein the selection compares the first and second information for each process, and selects a process having different first and second information.   The program according to claim 1 or 2, wherein in the selection, a process in which only one of the first and second information is acquired is selected.   With reference to the information indicating the history of process generation, among the processes for which the first and second information are not acquired, the process generated during the period when the test was performed is the target of information collection. The program according to any one of claims 1 to 3. The said test of the said software is performed in multiple times, and it evaluates whether the said test is effective based on the ratio which the process selected at each time corresponds, The any one of Claim 1 thru | or 4 program. In obtaining the first and second information, the operating system executed by the computer refers to the storage device storing the accumulated time executed for each existing process, and the first and second information for each process. The program according to any one of claims 1 to 5 , wherein the information of 2 is acquired. An information processing apparatus capable of executing a plurality of processes
When starting a software test, for each existing process, obtain first information indicating the accumulated time executed,
Upon completion of the test of the software, for each existing process, obtain second information indicating the accumulated time executed,
Based on the first and second information obtained in each process, select the process to be subjected to information collection when again performing the test of the software,
When an abnormality of the second process communicating with the first process is recorded in the history of the test related to the first process that is the target of information collection, the second process is the target of information collection. ,
Process selection method. An information processing apparatus capable of executing a plurality of processes,
For each existing process, a storage unit that stores the cumulative time that was executed;
When starting a software test, referring to the storage unit, for each existing process, obtain first information indicating the accumulated time of execution,
When ending the test of the software, referring to the storage unit, for each existing process, to obtain second information indicating the accumulated time of execution,
Based on the first and second information obtained in each process, select the process to be subjected to information collection when again performing the test of the software,
When an abnormality of the second process communicating with the first process is recorded in the history of the test related to the first process that is the target of information collection, the second process is the target of information collection. , Arithmetic unit,
An information processing apparatus.

Images