JP6410749B2 - Information processing apparatus, information processing method, and information processing program - Google Patents

Description

  The present invention relates to a technique for improving the efficiency of test result confirmation work.

  In Patent Document 1, when data is input to the device under test, the data is stored as an operation scenario. During the test, the operation scenario is reproduced and sent to the device under test, and a display screen displayed on the device under test. Is captured by the camera, and the captured image on the display screen is collated with the corresponding image of the image transition scenario previously captured and stored by the camera.

JP2013-30969A

  In the technique of Patent Literature 1, an image captured by a camera is collated with an image of an image transition scenario. The image of the image transition scenario is also an image taken with a camera in advance. As described above, in the technique of Patent Document 1, since the display screen is collated with an image captured by the camera, a difference in the display screen cannot be extracted with high accuracy. Therefore, an area where the difference exists is accurately determined. There is a problem that it cannot be done.

  The main object of the present invention is to solve such problems, so that the difference between the two display screens obtained by the two tests can be extracted with high accuracy and the region where the difference exists can be accurately determined. The main purpose is to be.

An information processing apparatus according to the present invention includes:
The data input procedure in the test for the first test object is reproduced for the second test object related to the first test object, the second test object is tested, and the second test object is tested. A test reproduction unit that obtains a screen shot image of a display screen including a plurality of display components generated in the test of the test object of 2 as a second screen shot image;
The first screen shot image, which is a screen shot image of a display screen including the plurality of display components, generated by the test on the first test object is compared with the second screen shot image, and the second screen shot image is compared. The difference between the first screen shot image and the second screen shot image is extracted, the extracted difference is analyzed, and in which display component of the plurality of display components the extracted difference exists. And a comparison and identification unit for identifying.

  According to the present invention, by comparing the screenshot images, the difference between the two display screens can be extracted with high accuracy, and the region where the difference occurs can be accurately determined.

FIG. 3 is a diagram illustrating a hardware configuration example of a GUI test screen collating device according to the first embodiment. FIG. 3 is a diagram illustrating a functional configuration example of a GUI test screen collating apparatus according to the first embodiment. FIG. 6 is a flowchart showing an operation example of the GUI test screen collating apparatus according to the first embodiment. FIG. 9 is a flowchart showing an operation example of a GUI test screen collating apparatus according to the second embodiment. FIG. 9 is a flowchart showing an operation example of a GUI test screen collating apparatus according to the second embodiment. FIG. 10 is a diagram illustrating an example of an image that is a target of pixel comparison according to the second embodiment. FIG. 6 shows an example of pixel comparison according to Embodiment 2. FIG. 6 shows an example of labeling according to the second embodiment. FIG. 10 is a diagram illustrating an example of a difference pattern according to the second embodiment.

Embodiment 1 FIG.
***Overview***
In this embodiment, in the test of an application having a GUI (Graphical User Interface) screen, the screen confirmation is performed by collating the screen shot image of the screen in order to automate and improve the efficiency of the screen confirmation at the time of environment update or regression test. A semi-automated method will be described.
In the field of software development, there is a problem that man-hours are required for manual test operation and screen confirmation at the time of software (also referred to as S / W) renovation or OS (Operating System) or library update. Furthermore, there is a problem that it takes time to confirm what kind of display abnormality has occurred at the time of screen confirmation and summarize the test results in a test report or the like.
In this embodiment, the GUI test screen verification device records the operation at the time of the first test. This initial test is a test that was performed on a certain software and obtained a correct test result. Then, the GUI test screen verification device automatically reproduces the recorded operation when performing the second and subsequent tests. The test after the second time is, for example, a test in an environment different from the test for the software after the update of the software for which the initial test was performed or the initial test for the software for which the initial test was performed.
Also, the GUI test screen collation apparatus according to the present embodiment acquires and records screen shot image data (also simply referred to as a screen shot image or a screen shot) of a display screen generated at the time of the first test. Then, the GUI test screen verification device acquires screen shot images of the display screen generated in the second and subsequent tests, and displays the screen shot image of the display screen in the first test and the screen shot images of the second and subsequent tests. Collate.
In the screen verification, the GUI test screen verification device performs pass / fail determination of the screen, classifies the rejected screens from the characteristics of the differences in the screens into screen difference patterns, and displays the information for each pattern for emphasis and test reports. Used for applications that automatically generate documents.
The software that is the target of the first test corresponds to the first test object, and the software that is the target of the second and subsequent tests corresponds to the second test object. As described above, the software that is the target of the second and subsequent tests is the software after the update of the software that is the target of the first test, or the software that is the target of the first test that is implemented in a different environment from the time of the first test. Thus, the second test object is related to the first test object.

*** Explanation of configuration ***
FIG. 1 shows a hardware configuration example of a GUI test screen collation apparatus 100 according to the present embodiment.
FIG. 2 shows an example of the functional configuration of the GUI test screen verification device 100.
The GUI test screen verification device 100 is an example of an information processing device. The operations performed by the GUI test screen verification device 100 are examples of an information processing method and an information processing program.

As shown in FIG. 1, the GUI test screen verification device 100 includes a processor 101, a memory 102, a secondary storage device 103, a display device 104, and an operation interface 105. The GUI test screen verification device 100 is a computer.
The processor 101 executes a program.
The memory 102 is a volatile storage device.
The secondary storage device 103 is a nonvolatile storage device.
The secondary storage device 103 includes a GUI operation recording / playback processing unit 201 and a screen matching processing unit 206 (automatic matching processing unit 202, feature amount calculation processing unit 203, classification model generation processing unit 204, pattern classification processing unit shown in FIG. 205) is stored.
Then, these programs are loaded from the secondary storage device 103 to the memory 102, and the processor 101 executes these programs, and a screen matching processing unit 206 (automatic matching processing unit 202, feature amount calculation processing unit 203, classification, which will be described later) The model generation processing unit 204 and the pattern classification processing unit 205) are operated.
The memory 102 functions as the screen data recording unit 211 and the classification model recording unit 212 in FIG. That is, the memory 102 stores a data input procedure and screen shot image data at the first test as the screen data recording unit 211. The memory 102 stores a classification model, which will be described later, as the classification model recording unit 212. At the time of operation recording / reproduction, the memory 102 outputs a data input procedure at the first test to the processor 101 in accordance with an instruction from the processor 101. At the time of screen collation, the memory 102 outputs screen shot image data to the processor 101 in accordance with an instruction from the processor 101.
The display device 104 is a display device that displays a display screen.
The operation interface 105 is an interface for operating a mouse or a keyboard.

  Next, the components shown in FIG. 2 will be described.

The GUI operation recording / playback processing unit 201 stores the data input procedure and screen shot image data in the first test in the screen data recording unit 211. That is, the GUI operation recording / playback processing unit 201 stores the data input procedure in the initial test for the software corresponding to the first test object in the screen data recording unit 211. The screen data recording unit 211 includes a memory 102 that is a storage device. The GUI operation recording / playback processing unit 201 includes, as data input procedures, input data input by the user of the GUI test screen verification device 100 using a mouse and a keyboard, input positions (input items) of each input data, input data The input order and the like are stored in the screen data recording unit 211. In addition, the GUI operation recording / playback processing unit 201 stores the screen shot image generated in the first test for the first test object in the screen data recording unit 211. The display screen includes a plurality of display components. FIG. 6 shows an example of a display screen. The display component is an element that forms a display screen. The display screen in FIG. 6 includes a date, a text form, and a button. The display screen may include a title bar, an icon, a list box, and the like shown in FIG. 9 as display components. Note that the screen shot image acquired in the first test is referred to as a first screen shot image.
Further, the GUI operation recording / playback processing unit 201 reads the data input procedure from the screen data recording unit 211 as the second and subsequent tests, and reproduces the read data input procedure for software corresponding to the second test object. Then test the software. Further, the GUI operation recording / playback processing unit 201 acquires a screen shot image of the display screen generated in the test. Note that the screen shot image acquired in this test is referred to as a second screen shot image.
The GUI operation recording / playback processing unit 201 stores the second screen shot image in the screen data recording unit 211.
The GUI operation recording / playback processing unit 201 corresponds to a test reproduction unit. Further, the processing performed by the GUI operation recording / playback processing unit 201 corresponds to a test reproduction step.

The screen verification processing unit 206 reads the first screen shot image and the second screen shot image from the screen data recording unit 211, and compares the first screen shot image and the second screen shot image in units of pixels. Then, the screen matching processing unit 206 extracts the difference between the first screen shot image and the second screen shot image, analyzes the extracted difference, and the extracted difference is any one of the plurality of display components. Identifies if present in the display component. Furthermore, the screen matching processing unit 206 identifies how the display component having the extracted difference changes between the first screen shot image and the second screen shot image. More specifically, the screen matching processing unit 206 uses the classification model to match any type of change between the first screenshot image and the second screenshot image in the display component where the difference exists. Identify what to do. The classification model is definition information in which a plurality of types of changes occurring between the first screen shot image and the second screen shot image are defined in association with the feature amount of the difference for each display component. Hereinafter, the type of change is also referred to as a difference pattern.
The screen collation processing unit 206 corresponds to a comparison and identification unit. Further, the processing performed by the screen matching processing unit 206 corresponds to a comparison and identification step.
As shown in FIG. 2, the screen matching processing unit 206 includes an automatic matching processing unit 202, a feature amount calculation processing unit 203, a classification model generation processing unit 204, and a pattern classification processing unit 205.
Hereinafter, the automatic matching processing unit 202, the feature amount calculation processing unit 203, the classification model generation processing unit 204, and the pattern classification processing unit 205 will be described.

  The automatic matching processing unit 202 reads the first screen shot image and the second screen shot image from the screen data recording unit 211, and performs complete match / mismatch determination by comparing pixel values. Further, the automatic matching processing unit 202 extracts a pixel having a difference from images determined to be inconsistent.

  The feature amount calculation processing unit 203 performs feature amount calculation for the difference pixels extracted by the automatic matching processing unit 202.

  The classification model generation processing unit 204 generates a classification model from the features calculated by the feature amount calculation processing unit 203, and stores the generated classification model in the classification model recording unit 212.

  The pattern classification processing unit 205 reads out the classification model from the classification model recording unit 212 and the screen shot image from the screen data recording unit 211, and classifies the differences in the screen into difference patterns.

*** Explanation of operation ***
The GUI operation recording / playback processing unit 201 stores the data input procedure in the first test and the screen shot image (first screen shot image) of the display screen in the screen data recording unit 211. Also, the GUI operation recording / playback processing unit 201 reads the data input procedure from the screen data recording unit 211, plays back the data input procedure, and displays a screen shot image (second screen shot image) displayed at that time. ) And the second screen shot image is stored in the screen data recording unit 211.

The automatic verification processing unit 202 reads out the first screen shot image and the second screen shot image from the screen data recording unit 211, and compares the pixel values of the two images to determine whether or not the two images are completely matched. judge.
Further, the automatic matching processing unit 202 extracts coordinate information of pixels having different pixel values.
Then, the feature amount calculation processing unit 203 performs feature amount calculation for the difference pixels extracted by the automatic matching processing unit 202. As the feature amount at this time, the width, height, area, maximum, minimum, average, variance, local feature amount, edge feature, or the like of the region where the difference pixel exists may be used.
Next, the classification model generation processing unit 204 generates a classification model for classifying screen differences into difference patterns based on the feature amounts calculated by the feature amount calculation processing unit 203. For each difference pattern, the user may label the difference pattern and generate a classification model by supervised learning based on the above-described feature amount and label of the difference pixel. Alternatively, the classification model may be generated by performing clustering by unsupervised learning without labeling. The classification model generation processing unit 204 stores the generated classification model in the classification model recording unit 212.
The pattern classification processing unit 205 reads out the classification model from the classification model recording unit 212, reads out the first screen shot image and the second screen shot image from the screen data recording unit 211, and outputs the first screen shot image and the second screen shot image. The screen shot images are collated, and the difference is classified into a difference pattern according to the classification model.

  Next, an operation example of the GUI test screen collating apparatus 100 according to the present embodiment will be described with reference to the flowchart of FIG.

First, in step ST301, the GUI operation recording / playback processing unit 201 acquires operation contents (data input procedure) such as a mouse and a keyboard, and stores the acquired operation contents in the screen data recording unit 211.
In step ST302, the GUI operation recording / playback processing unit 201 acquires a screen shot image (first screen shot image) of the display screen. In FIG. 3, one screen shot is acquired at the time of one operation recording, but a plurality of sheets are acquired by one operation recording, such as acquiring a screen shot every time the screen changes. A screenshot may be taken.
Next, in step ST303, the GUI operation recording / playback processing unit 201 plays back the operation content recorded in step ST301. In step ST304, a screen shot image (second screen shot image) of the display screen at that time is acquired. To do.
Next, in step ST305, the automatic matching processing unit 202 determines whether or not the first screen shot image and the second screen shot image completely match. If the first screen shot image and the second screen shot image do not completely match, the automatic matching processing unit 202 obtains the coordinates of the difference pixel in the screen.
Next, in step ST306, the feature amount calculation processing unit 203 extracts the feature amount of the difference in the screen such as the size of the difference region and the average of the difference between the pixel values from the coordinates of the difference pixel.
When the classification model is not generated and learning of the classification model is performed (NO in step ST307), the classification model generation processing unit 204 generates a classification model in step ST308. Then, the classification model generation processing unit 204 stores the generated classification model in the classification model recording unit 212.
When the classification model has already been generated and the difference pattern is classified by the classification model (YES in step ST307), the pattern classification processing unit 205 reads the classification model from the classification model recording unit 212 in step ST309, and performs the difference. Perform pattern classification.

*** Explanation of the effect of the embodiment ***
In this embodiment, the operation at the first test is recorded, and the recorded operation is reproduced for the second and subsequent tests. Further, screen shot images at the time of operation recording / playback are collated, and the screen difference is classified for each difference pattern. For this reason, according to the present embodiment, it is possible to improve the efficiency of the work of creating evidence for test operation such as manual test operation, screen confirmation, test report, etc. at the time of S / W repair or OS / library update.

Embodiment 2. FIG.
In this embodiment, the operation procedure described in Embodiment 1 will be described more specifically. In the present embodiment, a software test of a standard Web application having a screen including basic components such as a text form, a button, a title bar, and a list box is taken as an example.

*** Explanation of configuration ***
A hardware configuration example (FIG. 1) and a functional configuration example (FIG. 2) in the present embodiment are the same as those in the first embodiment.

*** Explanation of operation ***
The GUI operation recording / playback processing unit 201 records the operation content at the first test of the application to be tested, and records a screen shot image (first screen shot image) of the display screen. For example, when transitioning from A screen → B screen → C screen, the GUI operation recording / playback processing unit 201 records screen shots of the A screen, the B screen, and the C screen. Also, the GUI operation recording / playback processing unit 201 plays back the recorded operation content and records a screen shot image (second screen shot image) of the display screen displayed at that time. At this time, the GUI operation recording / playback processing unit 201 records the screen shot image of the display screen corresponding to the display screen from which the screen shot image was acquired during operation recording. The screen shot image of the display screen is stored in the screen data recording unit 211.

The automatic matching processing unit 202 reads the first and second screen shot images from the screen data recording unit 211, and compares both pixel values of the same coordinates in both images as shown in FIG. Determine if the images match perfectly. The automatic matching processing unit 202 compares all the pixel values in the screen, determines that the pixel values are all the same, and determines that the pixel values are completely the same. Also good. Alternatively, the automatic matching processing unit 202 extracts feature points such as edges and local feature amounts from each of the first screen shot image and the second screen shot image and obtains corresponding points (number of corresponding points × 2 ) / (Total number of feature points of two images) may be determined to be inconsistent when it is equal to or less than a threshold value. The threshold at this time may be determined based on past cases.
Further, when the first screen shot image and the second screen shot image do not match, the automatic matching processing unit 202 extracts a different area from the coordinate information of the pixels having different pixel values. Then, the feature amount calculation processing unit 203 performs feature amount calculation for each different area. As the feature amount at this time, the width, height, area, maximum, minimum, average, variance, local feature amount, edge feature, or the like of the region where the difference pixel exists may be used.

An example of the feature quantity extraction procedure will be described.
First, the feature amount calculation processing unit 203 compares the luminance values of pixels with the same coordinates (0 to 255 in RGB) for the first screen shot image and the second screen shot image. Save the coordinates. Next, as shown in FIG. 7, the difference pixels are mapped in the screen, and display components in the GUI screen such as a text form, a title bar, and a button in which the difference pixels are generated are extracted as the difference areas. Then, the feature amount calculation processing unit 203 calculates, for each different region, the maximum value and average value of the region width, height, area, and luminance value difference as the feature amount.

Next, as shown in FIG. 8, the user of the GUI test screen verification device 100 defines display components in the GUI screen such as a text form, a title bar, and a button, and labels (attributes) for each display component. Give.
As shown in FIG. 8, the feature amount calculation processing unit 203 labels “text form” if the difference area is on the text form, and “button” if the difference area is on the button.

The classification model generation processing unit 204 generates a classification model from a label of each different area and the calculated feature amount by a clustering technique based on supervised learning such as SVM (Support Vector Machine). The generated classification model is one in which a label is output when a feature amount is input, and thus, it can be classified into a difference pattern corresponding to the label.
Examples of difference patterns on the GUI screen include a position shift of the entire screen, a difference in displayed characters such as date and time, user ID, a difference in characters in a text form, a combo box and a list box. There are differences in items, whether there is button focus, etc. FIG. 9 shows an example of a difference pattern for each display component. In the classification model, for each display component, a criterion for determining which difference pattern in FIG. 9 corresponds to the difference between the first screenshot image and the second screenshot image is associated with the feature amount. Defined. For each difference area, the user may label the difference pattern and generate a classification model by supervised learning based on the above-described difference pixel feature amount and label. The generated classification model information is stored in the classification model recording unit 212.

  The pattern classification processing unit 205 reads out the classification model generated by the classification model generation processing unit 204 from the classification model recording unit 212, and the first screen screenshot image and the second screen screenshot image from the screen data recording unit 211. Is read. Then, the pattern classification processing unit 205 collates the first screen screenshot image and the second screen screenshot image, and classifies the difference into a difference pattern according to the classification model. That is, the pattern classification processing unit 205 identifies how a display component having a difference changes between the first screenshot image and the second screenshot image.

Next, an operation example of the GUI test screen collating apparatus 100 according to the present embodiment will be described with reference to the flowcharts of FIGS. 4 and 5. In the present embodiment, a test case in which operations on the screen A and the screen B are performed with respect to an application including three screens of the screen A, the screen B, and the screen C is targeted.
Steps ST401 to ST405 represent GUI operation recording, steps ST406 to ST410 represent GUI operation reproduction, and steps ST411 to ST417 represent GUI screen collation processing.

  First, the GUI operation recording / playback processing unit 201 acquires a screen shot of the screen A in step ST401, and records the operation content for the screen A in step ST402. Next, when transitioning to screen B, GUI operation recording / playback processor 201 acquires a screen shot of screen B in step ST403, and records an operation for screen B in step ST404. Next, when transitioning to the screen C, the GUI operation recording / playback processing unit 201 acquires a screen shot of the screen C in step ST405 and ends the operation recording.

  Next, when the operation playback is started, the GUI operation recording / playback processing unit 201 acquires a screen shot of the screen A in step ST406, and the operation on the screen A is played back in step ST407. Next, when transitioning to screen B, GUI operation recording / playback processing section 201 acquires a screen shot of screen B in step ST408, and the operation on screen B is played back in step ST409. Next, when the GUI operation recording / playback processing unit 201 transitions to the screen C, the screen shot of the screen C is acquired in step ST410.

In step ST411, the automatic collation processing unit 202 determines whether or not the acquired operation recording / playback screen shot image is a complete match. When the screen shot image pair does not completely match in any of the screens A to C, the automatic matching processing unit 202 determines that the screen shot image pair that does not completely match is displayed in step ST412. The coordinates of the different pixels are obtained. Next, the feature quantity calculation processing unit 203 extracts a difference area from the coordinate information of the difference pixel in step ST413, and in step ST414, for each difference area, a difference screen such as the size of the difference area, the average of the pixel value differences, and the like. Extract feature values.
When the classification model is not generated and learning of the classification model is performed (NO in step ST415), the classification model generation processing unit 204 generates a classification model in step ST416. Then, the classification model generation processing unit 204 stores the generated classification model in the classification model recording unit 212.
When the classification model has already been generated and the difference pattern is classified by the classification model (YES in step ST415), the pattern classification processing unit 205 reads the classification model from the classification model recording unit 212 in step ST417, and the difference is determined. Perform pattern classification.

*** Explanation of hardware configuration ***
Finally, a supplementary description of the hardware configuration of the GUI test screen verification device 100 will be given.
A processor 101 shown in FIG. 1 is an IC (Integrated Circuit) that performs processing.
The processor 101 is a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or the like.
The memory 102 is a RAM (Random Access Memory).
The secondary storage device 103 is a ROM (Read Only Memory), a flash memory, an HDD (Hard Disk Drive), or the like.

The secondary storage device 103 also stores an OS (Operating System).
At least a part of the OS is executed by the processor 101.
While executing at least a part of the OS, the processor 101 executes a GUI operation recording / playback processing unit 201 and a screen matching processing unit 206 (automatic matching processing unit 202, feature quantity calculation processing unit 203, classification model generation processing unit 204, pattern classification processing, and the like. A program that realizes the functions of the unit 205).
Although one processor is illustrated in FIG. 1, the GUI test screen verification device 100 may include a plurality of processors.
In addition, information, data, signal values, and variable values indicating the processing results of the GUI operation recording / playback processing unit 201 and the screen matching processing unit 206 are stored in the memory 102, the secondary storage device 103, the register in the processor 101, or the cache memory. Remembered.
A program for realizing the functions of the GUI operation recording / playback processing unit 201 and the screen collation processing unit 206 is stored in a portable storage medium such as a magnetic disk, flexible disk, optical disk, compact disk, Blu-ray (registered trademark) disk, or DVD. May be.

Further, “part” of the GUI operation recording / reproducing processing unit 201 and the screen collation processing unit 206 may be read as “circuit”, “process”, “procedure”, or “processing”.
The GUI test screen verification device 100 may be an electronic circuit such as a logic IC (Integrated Circuit), a GA (Gate Array), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array).
In this case, the GUI operation recording / playback processing unit 201 and the screen matching processing unit 206 are each realized as part of an electronic circuit.
The processor and the electronic circuit are also collectively referred to as a processing circuit.

  100 GUI test screen verification device, 101 processor, 102 memory, 103 secondary storage device, 104 display, 105 operation interface, 201 GUI operation recording / playback processing unit, 202 automatic verification processing unit, 203 feature quantity calculation processing unit, 204 classification Model generation processing unit, 205 pattern classification processing unit, 206 screen collation processing unit, 211 screen data recording unit, 212 classification model recording unit.

Claims (6)

The data input procedure in the test for the first test object is reproduced for the second test object related to the first test object, the second test object is tested, and the second test object is tested. A test reproduction unit that obtains a screen shot image of a display screen including a plurality of display components generated in the test of the test object of 2 as a second screen shot image;
The first screen shot image, which is a screen shot image of a display screen including the plurality of display components, generated by the test on the first test object is compared with the second screen shot image, and the second screen shot image is compared. Extracting a difference between the first screenshot image and the second screenshot image , calculating a feature amount of the extracted difference,
For each display component, by using definition information in which a plurality of types of changes that occur between the first screenshot image and the second screenshot image are defined in association with a feature amount of difference,
An information processing apparatus comprising : a comparison and identification unit that identifies a type of change between the first screenshot image and the second screenshot image in a display component having an extracted difference . The test reproduction unit is
In the test for the first test object, the data input procedure and the first screenshot image are stored in a storage device,
In the test for the second test object, the data input procedure is read from the storage device, and the data input procedure is reproduced for the second test object.
The comparison and identification unit includes:
The information processing apparatus according to claim 1, wherein the first screen shot image is read from the storage device, and the first screen shot image and the second screen shot image are compared. The comparison and identification unit includes:
The difference between the first screen shot image and the second screen shot image is extracted by comparing the first screen shot image and the second screen shot image in pixel units. Information processing device. The test reproduction unit is
The information processing apparatus according to claim 1, wherein a data input procedure in a test for the first test object that is software is reproduced for the second test object that is software. The computer reproduces the data input procedure in the test for the first test object with respect to the second test object related to the first test object, and tests the second test object. A test reproduction step of acquiring a screen shot image of a display screen including a plurality of display components generated in the test of the second test object as a second screen shot image;
The computer generates a first screen shot image and a second screen shot image, which are screen shot images of a display screen including the plurality of display components, generated by a test on the first test object. Comparing and extracting the difference between the first screenshot image and the second screenshot image , calculating the feature value of the extracted difference,
For each display component, by using definition information in which a plurality of types of changes that occur between the first screenshot image and the second screenshot image are defined in association with a feature amount of difference,
An information processing method comprising : a comparison and identification step for identifying a type of change between the first screenshot image and the second screenshot image in a display component having an extracted difference . The data input procedure in the test for the first test object is reproduced for the second test object related to the first test object, the second test object is tested, and the second test object is tested. A test reproduction step of acquiring a screen shot image of a display screen including a plurality of display components generated in the test of the test object of 2 as a second screen shot image;
The first screen shot image, which is a screen shot image of a display screen including the plurality of display components, generated by the test on the first test object is compared with the second screen shot image, and the second screen shot image is compared. Extracting a difference between the first screenshot image and the second screenshot image , calculating a feature amount of the extracted difference,
For each display component, by using definition information in which a plurality of types of changes that occur between the first screenshot image and the second screenshot image are defined in association with a feature amount of difference,
Information that causes the computer to perform a comparison and identification step that identifies which type the change between the first screenshot image and the second screenshot image in the display component in which the extracted difference exists matches. Processing program.

Images