JP6543649B2 - Screen difference detection device and program - Google Patents

Description

  The present invention relates to a screen difference detection apparatus and program.

  Correctly displayed screen (correct screen) for the execution result screen of the application for the purpose of detection of layout collapse of UI (screen) when operating the application in multiple environments, degradation detection in regression test, etc. There is a tool for automatically detecting an error location by comparing the image of and the image of the test target screen (comparison screen).

  In the pixel-by-pixel difference extraction tool (for example, non-patent document 4) or technique (for example, non-patent document 1), the difference between the correct screen and the comparison screen can be compared in pixel units and the difference can be displayed. Further, in the technique of Non-Patent Document 2, the difference between screens can be extracted and displayed in units of screen elements such as a text box and a button on the correct screen and the comparison screen.

  These tools can be used by preparing only the correct answer screen and the screen shot image of the comparison screen, so a specific implementation technology (eg: Web application, native application (Windows (registered trademark), Android (registered trademark), etc.) It can be widely applied without depending on iOS (registered trademark) and the like.

2014. Finding HTML presentation failures using image comparison technology. In Proceedings of the 29th ACM / IEEE international conference on Automated software engineering (ASE '14). ACM, New York, NY, USA, 91-96 (University of Southern California, USA) Osamu Tanno: "Proposal of UI Layout Failure Detection Support Method Using Image Processing", IPSJ SIG Technical Report, vol. 2016-SE-194, no. 9, pp. 1-8, November 2016 Hiroyuki Sakurai, Ryo Toyozawa, Masato Nagata, "Practical OpenCV 2.4 for Python: Image Processing & Analysis", Cut System, 2014 (ISBN = 9784877833466), P82-P83 DiffImage, [online], Internet <URL: https://ja.osdn.net/projects/sfnet_diffimg/>

  However, in applications that provide services for consumers, there are parts (hereinafter referred to as “dynamic change areas”) in which the contents change each time the screen is displayed, such as advertisements and news articles. In the pixel-by-pixel difference extraction tool and the techniques of Non-Patent Documents 1 and 2, all differences are detected. As a result, many differences are detected for one screen, which makes it difficult to detect a UI layout collapse that is originally intended to be detected and degradation in a regression test.

  In order to solve this problem, when comparing the difference between two screens in non-patent documents 1 and 2, as shown in non-patent document 3, an area group where it is not necessary to detect the difference is separately used as a mask image. It is also conceivable to prepare and use this mask image to exclude the dynamically changing area from the target of the difference comparison so that it is not detected as a difference.

  However, since each pixel constituting the mask area is designated by absolute coordinates, according to this method, there is a change in the screen configuration due to the case where the size of the dynamic change area changes every time or the software is modified. When the position of the change area is shifted, there arises a problem that the dynamic change area can not be appropriately excluded from comparison (outside detection of difference).

  FIG. 1 is a diagram for explaining the problems of the prior art. A correct answer screen g100 and a mask image g200 are shown in (1) of FIG. The mask image g200 is a mask image for excluding the area of “★ ad” in the correct answer screen g100 from the targets of the difference comparison.

  In (2), a comparison screen g110 in which the size of the area of "★ ad" is changed is shown. In this case, in the mask image g200, it is difficult to appropriately exclude the area of “advertisement” on the comparison screen g110.

  In (3), the area of the “link collection” is enlarged due to the software correction, and a comparison screen g120 in which the position of “★ ad” is shifted is shown. Also in this case, in the mask image g200, it is difficult to appropriately exclude the area of “advertisement” on the comparison screen g120.

  The present invention has been made in view of the above-described point, and an object thereof is to make it possible to appropriately remove a partial area of a screen from comparison targets.

  Then, in order to solve the above-mentioned subject, the screen difference detection device detects the difference between the first image showing the first display result of the screen and the second image showing the second display result of the screen. A detection device according to any one or more of a plurality of screen element rectangles included in the first image as information for specifying a first region which is a partial region in the first image. A reception unit that receives from the user a designation of a relative positional relationship based on the first screen element rectangle for each of the screen element rectangles, and matches each of the first screen element rectangles in the second image A second area specified based on the respective second screen element rectangles and the relative positional relationship specified for each of the first screen element rectangles, and the second area in the first image Target of detection or detection of the difference As outer elephant, and having a detection portion for detecting the difference.

  A partial area of the screen can be appropriately removed from the comparison target.

It is a figure for demonstrating the problem of a prior art. It is a figure which shows the example of hardware constitutions of the screen difference detection apparatus 10 in embodiment of this invention. It is a figure which shows the function structural example of the screen difference detection apparatus 10 in embodiment of this invention. It is a figure for demonstrating the direction of the designation | designated area | region. It is a flowchart for demonstrating an example of the process sequence which the screen difference detection apparatus 10 performs. It is a figure which shows an example of the data structure of a result report. It is a flow chart for explaining an example of a processing procedure of extraction processing of a landmark candidate. It is a flowchart for demonstrating an example of a process procedure of designation | designated process of a test object area | region. It is a figure which shows an example of the data structure of test object area | region information. It is a flowchart for demonstrating an example of a process procedure of a detection process of a difference in a test object field. It is a flowchart for demonstrating an example of a process procedure of the removal process of the screen element rectangle out of a test object area | region. It is a figure for demonstrating concretely an example of the effect of this Embodiment. It is a figure for demonstrating concretely an example of the effect of this Embodiment.

  Hereinafter, embodiments of the present invention will be described based on the drawings. FIG. 2 is a diagram showing an example of the hardware configuration of the screen difference detection device 10 according to the embodiment of the present invention. The screen difference detection device 10 of FIG. 2 has a drive device 100, an auxiliary storage device 102, a memory device 103, a CPU 104, an interface device 105, a display device 106, an input device 107, etc. .

  A program for realizing the processing in the screen difference detection apparatus 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive apparatus 100, the program is installed from the recording medium 101 to the auxiliary storage apparatus 102 via the drive apparatus 100. However, the installation of the program does not necessarily have to be performed from the recording medium 101, and may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program and also stores necessary files and data.

  The memory device 103 reads out the program from the auxiliary storage device 102 and stores it when there is an instruction to start the program. The CPU 104 implements the function of the screen difference detection apparatus 10 in accordance with a program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network. The display device 106 displays a graphical user interface (GUI) according to a program. The input device 107 includes a keyboard, a mouse, and the like, and is used to input various operation instructions.

  FIG. 3 is a diagram showing an example of a functional configuration of the screen difference detection device 10 according to the embodiment of the present invention. In FIG. 3, the screen difference detection apparatus 10 has an image input unit 11, a landmark candidate extraction unit 12, a test target area specification unit 13, a screen difference detection unit 14, a report output unit 15, and the like. These units are realized by processing that one or more programs installed in the screen difference detection device 10 cause the CPU 104 to execute.

  The image input unit 11 reads a pair of a correct answer screen and one or more comparison screens from the folder specified by the user for each screen to be tested. Multiple comparison screens may exist for one correct answer screen. The image input unit 11 also generates a pair of a correct answer screen and a comparison screen. One pair includes one correct answer screen and one comparison screen. When there are a plurality of comparison screens for the same correct answer screen, a pair with the same correct answer screen is generated for each comparison screen.

  The correct answer screen is, for example, image data (hereinafter referred to as a “screen image”) indicating the display result of the screen in an environment where the screen is displayed according to the intention of the creator (hereinafter referred to as “correct answer environment”). Say). The comparison screen is, for example, a screen image showing a display result of the screen in an environment (hereinafter, referred to as “comparison environment”) different from the correct answer environment. The environment is distinguished by, for example, a combination of a model of a terminal displaying a screen, an OS (Operating System), and a browser. Furthermore, environments may be distinguished according to browser display sizes. The environment can be said to be an example of the display condition of the screen. The correct answer screen and the comparison screen may be acquired by, for example, screen capture.

  Here, the screen refers to a screen that the program causes a computer to display. The screen may be a screen of an application program or the like, or may be a screen described in HTML (Hyper Text Markup Language) or the like.

  The landmark candidate extraction unit 12 extracts screen element rectangles that can be uniquely identified from the correct answer screen without duplication. The screen element rectangle refers to a rectangular area of a component of the screen. In the present embodiment, the screen element rectangle is a candidate for a landmark rectangle (hereinafter, referred to as “landmark candidate”). The landmark rectangle refers to a rectangular area serving as a mark or a reference for specifying a target area of a test (comparison) or a partial area to be excluded from the test (hereinafter referred to as “designated area”). The landmark rectangle is a rectangle (screen element) which exists at least on the correct answer screen and can be uniquely identified, and a plurality of landmark rectangles may exist. In the present embodiment, a designated area surrounded by a plurality of landmark rectangles selected by the user from the screen element rectangles extracted by the landmark candidate extraction unit 12 (the designated area is a correct screen and a comparison screen) Exists) is regarded as an exclusion zone. More specifically, in the present embodiment, it is excluded by a plurality of combinations of "landmark rectangle and relative direction (hereinafter referred to as" direction of designated area ") relative to the landmark rectangle. The target area is specified.

  FIG. 4 is a diagram for explaining the direction of the designated area. The upper part of FIG. 4 shows the definition of the direction of the designated area. The direction of the designated area is designated by one or more combinations of upper left, upper, upper right, right, lower right, lower, lower left, and left with reference to the landmark rectangle.

  In (1) of FIG. 4, the range of the designated area when the “upper right, right, lower right” of the landmark rectangle is designated as the direction of the designated area is indicated by hatching.

  In (2) of FIG. 4, the range of the designated area when the “upper right, right” of the landmark rectangle is designated as the direction of the designated area is indicated by hatching.

  In (3) of FIG. 4, the range of the designated area when the “right” of the landmark rectangle is designated as the direction of the designated area is indicated by hatching.

  In addition, in (1), (2) and (3), the rectangle of an outer periphery shows the range of a screen.

  An area surrounded by these landmark rectangles (or an area specified by these pairs) is specified by specifying a plurality of sets of such landmark rectangles and the direction of the designated area in each of the landmark rectangles. Is specified as a designated area.

  The test target area specification unit 13 receives from the user selection of a landmark candidate to be a landmark rectangle and also receives specification of the direction of the specified area from the user for each landmark rectangle.

  The screen difference detection unit 14 detects the difference between the correct answer screen and the comparison screen. At this time, the screen difference detection unit 14 regards one or more designated areas specified by each landmark rectangle and the direction of the designated area designated for each landmark rectangle as a test target area or an area other than the test target area. Detect the difference. The test target area refers to an area to be compared (that is, a difference detection target area).

  The report output unit 15 outputs the detection result of the difference between the correct answer screen and the comparison screen.

  The processing procedure executed by the screen difference detection device 10 will be described below. FIG. 5 is a flowchart for explaining an example of the processing procedure executed by the screen difference detection device 10.

  In step S110, the image input unit 11 reads the correct answer screen group and the comparison screen group stored for each screen to be tested under the folder specified by the user, and generates a pair of the correct answer screen and the comparison screen. Do.

  Subsequently, the image input unit 11 creates an empty result report (S120).

  FIG. 6 is a diagram showing an example of the data structure of the result report. In FIG. 6, an example (configuration example) of the data structure of the result report is described by BNF (Backus-Naur Form).

  As shown in FIG. 6, the result report is information including a correct answer screen and a comparison screen, and a screen difference list for each pair of the correct answer screen and the comparison screen. The screen difference list is a set of zero or more screen differences between the correct answer screen and the comparison screen. The screen difference is information including a difference type, a mutation value, a correct screen element rectangle, and a comparison screen element rectangle. The difference type is the type of difference, and the value is "no difference", "movement in the X-axis direction", "movement in the Y-axis direction", "scaling", "disappear", or "addition". The mutation value is a value indicating the magnitude of the difference in the difference type. The correct answer screen element rectangle is a screen element rectangle related to the difference in the correct answer screen. The comparison screen element rectangle is a screen element rectangle related to a difference in the comparison screen. That is, the difference is detected in units of screen element rectangles.

  In addition to the screen information of the correct answer screen, the landmark candidate group and the test target area information can be set in the correct answer screen. The landmark candidate group is one or more sets of screen element rectangles which become landmark candidates.

  Subsequently, steps S130 to S160 are executed for each pair of the correct answer screen and the comparison screen. Hereinafter, the pair to be processed is referred to as a “target pair”.

In step S130, the landmark candidate extraction unit 12 extracts a landmark candidate from the correct answer screen of the target pair (hereinafter referred to as "target correct screen"). Subsequently, the test target area designation unit 13 extracts the landmark candidate from the target correct screen. Specification of the test target area is received from the user based on the extracted landmark rectangle (S140). The test target area is an area to be compared in the target pair.

  Subsequently, the screen difference detection unit 14 detects (extracts) all the differences in the test target area for the correct screen and the comparison screen related to the target pair (S150). That is, a screen difference list regarding the target pair is generated, and the screen difference list is added to the result report.

  Subsequently, the screen difference detection unit 14 adds the correct answer screen and the comparison screen related to the target pair to the result report (S160).

  When steps S130 to S160 are executed for all the pairs, the report output unit 15 outputs the content of the result report (S170).

  Subsequently, the details of step S130 will be described. FIG. 7 is a flowchart for explaining an example of a processing procedure of landmark candidate extraction processing.

  In step S210, the landmark candidate extraction unit 12 determines whether test target area information has already been set in the target correct screen. If the test target area information has already been set for the target correct screen (Yes in S210), the processing procedure ends. For example, when there are three comparison screens for one correct answer screen, three pairs are generated, but it is not efficient to execute step S320 or later for the second and subsequent pairs. . The branch in step S310 is to avoid duplicate processing for the second and subsequent pairs having the same correct answer screen.

  When the test target area information is not set in the target correct answer screen (No in S210), the landmark candidate extraction unit 12 extracts all screen element rectangles from the target correct answer screen, and displays the screen element rectangle list. Obtained (S220). The extraction of screen element rectangles can be realized by combining known techniques. For example, screen element rectangles are extracted by executing a procedure such as grayscale processing, Canny processing, dilation processing, contour extraction, and rectangle conversion (extracting circumscribed rectangles of contours and extracting rectangles therefrom) on the image. May be The screen element rectangle is information represented by the upper left coordinates (integer two-dimensional coordinate points) of the rectangular area and the lower right coordinates (integer two-dimensional coordinate points) of the rectangular area.

  Subsequently, the landmark candidate extraction unit 12 removes screen element rectangles in which images overlap each other in the screen element rectangle list from the screen element rectangle list (S230). For example, in the screen element rectangle list, when the image of the screen element rectangle a and the screen element rectangle b overlap, both the screen element rectangle a and the screen element rectangle b are removed from the screen element rectangle list. This is because screen element rectangles whose images overlap with other screen element rectangles are not unique and can not be landmark rectangles. The image overlap means that the images completely match. For example, in the screen of a web page in which a plurality of the same icons are arranged, screen element rectangles related to the respective icons overlap.

  Note that the screen element rectangle that can be uniquely identified without duplication is extracted as a landmark candidate from the correct answer screen, so that the user can select a landmark candidate from among them and specify a test target area in step S140 of FIG. As a result, in step S150 of FIG. 5, it is easy to uniquely determine the test target area (or the area to be excluded) on the correct screen and the comparison screen.

  Subsequently, the landmark candidate extraction unit 12 sets the screen element rectangle list from which the duplication is removed as the landmark candidate group on the target correct screen (S240).

  Subsequently, the details of step S140 in FIG. 5 will be described. FIG. 8 is a flowchart for explaining an example of the processing procedure of the test target area designation process.

  In step S310, the test target area specification unit 13 determines whether test target area information has already been set on the target correct screen. The purport of the said determination is the same as step S210. When the test target area information is not set in the target correct answer screen (No in S310), the test target area designation unit 13 acquires a landmark candidate group set in the target correct answer screen (S320).

  Subsequently, the test target area designation unit 13 creates empty test target area information, and sets the test target area information on the target correct answer screen (S330).

  FIG. 9 is a diagram showing an example of a data structure of test target area information. In FIG. 9, an example (configuration example) of a data structure of test target area information is described by BNF.

  As shown in FIG. 9, the test target area information is information including an area exclusion type and a designated area information group. The area exclusion type is information that distinguishes whether the specified area specified by the specified area information group is an area excluded from the test object or an area to be tested, “specified area It can take the value of "excludes" or "excludes non-specified area".

  The designated area information group is 0 or more designated area information. One piece of designated area information is information indicating one designated area, and is represented by a set of one or more pieces of designated area specifying information. One piece of specified area specifying information is a set of one landmark rectangle and the direction of one specified area. That is, the designated area information is information for specifying one designated area by a combination of the landmark rectangle and the direction of the designated area. The direction of the designated area is one or more combinations of upper left, upper, upper right, right, lower right, lower, lower left, and left as described in FIG. 4.

  Subsequently, the test target area designation unit 13 receives an input of the area exclusion type from the user (S340). The test target area specification unit 13 sets the received area exclusion type in the test target area information. The area exclusion type may be preset and stored in the auxiliary storage device 102. By specifying the area exclusion type, it is possible to flexibly set the test target area. That is, the variation of the method of specifying the test target area can be increased.

  Subsequently, the test target area designation unit 13 presents the user with the landmark candidate group set on the target correct screen (S350). For example, in the target correct answer screen, an image in which the range of each landmark candidate is highlighted may be displayed on the display device 106.

  Subsequently, the loop process of steps S360 to S400 is performed for each area that the user intends to specify as the specified area. Therefore, the loop processing ends at the discretion of the user. Hereinafter, the designated area to be processed is referred to as a “target designated area”.

  In step S360, the test target area designation unit 13 creates designated area information for the target designation area.

  Subsequently, steps S370 to S390 are loop processing for each of the designated area specifying information that the user intends to designate in order to specify the target designating area. The loop process can also be said to be a loop process for each landmark candidate selected as a landmark rectangle by the user among the landmark candidates presented in step S350.

  In step S370, the test target area designation unit 13 receives, from the user, the selection of a landmark candidate to be a landmark rectangle from the landmark candidate group. In response to the selection, the test target area specification unit 13 creates new specified area specification information, and the selected landmark rectangle (hereinafter referred to as “target landmark rectangle”) is the specified area specification information (hereinafter referred to as Set as "target specified area specification information". The designated area specifying information is, as described above, a set of the landmark rectangle and the direction of the designated area. At this time, the direction of the designated area of the new designated area identification information is undefined.

  Subsequently, the test target area designation unit 13 receives, from the user, selection of the direction of the designated area with respect to the target landmark rectangle (S380). For example, as described in FIG. 4, the orientation of the designated area with respect to the target landmark is selected by the user. The test target area designation unit 13 sets the direction of the selected designated area in the target designated area specification information.

  Subsequently, the test target area specification unit 13 adds the target specification area specification information to the specification area information (the specification area information created in step S360) corresponding to the target specification area (S390).

  When the user instructs to end the input of the specified area specifying information for specifying the target specified area, the loop process of steps S370 to S390 ends with respect to the target specified area.

  Subsequently, the test target area designation unit 13 adds designated area information corresponding to the target designated area to the designated area information group of the test target designated area information (S400).

  Steps S360 to S400 are repeatedly executed until the end of the designation of the designated area is instructed by the user.

  Subsequently, details of step S150 in FIG. 5 will be described. FIG. 10 is a flowchart for explaining an example of the processing procedure of the process of detecting the difference in the test target area.

  In step S410, the screen difference detection unit 14 extracts screen element rectangles from the target correct screen. The extraction of the screen element rectangle may be performed based on the same method as step S220 of FIG. As a result, a screen element rectangle list on the correct screen is obtained.

  Subsequently, the screen difference detection unit 14 removes screen element rectangles outside the test target area from the screen element rectangle list (S420). This makes it possible to extract only the differences that need to be confirmed as test objects.

  Subsequently, the screen difference detection unit 14 executes steps S410 and S420 for the comparison screen related to the target pair (S430).

  Subsequently, the screen difference detection unit 14 detects a difference in only the test target area on the correct screen and the comparison screen related to the target pair (S440). Specifically, the screen difference detection unit 14 selects only the screen element rectangles included in the screen element rectangle list (the screen element rectangle list from which the screen element rectangle outside the test target area is excluded) of each of the correct screen and the comparison screen. By comparing as a target, the difference between the correct answer screen and the comparison screen is detected (extracted). For detection (extraction) of such a difference, for example, known techniques such as the technique disclosed in Non-Patent Document 2 can be used. A screen difference list, which is a list of detected differences, is set in the result report.

  Subsequently, details of step S420 in FIG. 10 will be described. FIG. 11 is a flowchart for explaining an example of the processing procedure of the process of removing screen element rectangles outside the test target area. The screen (correct screen or comparison screen) to be processed at the time when the processing procedure of FIG. 11 is called is referred to as a “target screen”.

  In steps S510 to S540, for each of the screen element rectangles included in the screen element rectangle list of the target screen, the designated area information group of the designated area information included in the test target area information set in the target correct screen is displayed. It is executed for each included specified area information. Hereinafter, the screen element rectangle to be processed is referred to as a “target rectangle”, and the designated area information to be processed is referred to as “target designated area information”.

  Step S510 is executed for each landmark rectangle of each of the designated area specifying information included in the target specifying area information. Hereinafter, the landmark rectangle to be processed in step S510 is referred to as a “target landmark rectangle”.

  In step S510, the screen difference detection unit 14 specifies a screen element rectangle that matches the target landmark rectangle on the target screen. The screen element rectangle that matches the target landmark rectangle can be obtained by searching, for example, by template matching. The rectangle specified here is referred to as "real landmark rectangle".

  When the actual landmark rectangles in the target screen are specified for all the landmark rectangles, the screen difference detection unit 14 branches the process according to the area exclusion type included in the test target area information (S520). If the area exclusion type is "excluded the specified area", the screen difference detection unit 14 removes the target rectangle from the screen element rectangle list of the target screen if the target rectangle is included in the specified area (S530). ). The designated area is an area identified by replacing the landmark rectangles of the designated area identification information of the target designating area information with the actual landmark rectangles identified in step S510 for the landmark rectangles. That is, in a state in which the landmark rectangles of each designation area specification information of the target designation area information are replaced by the corresponding real landmark rectangles, the direction of the designation area corresponding to the real landmark rectangle for each real landmark rectangle. The area pertaining to An area where the areas specified for all the specified area specifying information of the target specified area information overlap (overlaps) is the specified area. For example, if (1), (2), and (3) in the lower row of FIG. 4 are the respective designated area identification information included in the target designation area information, then (1), (2), and (3) The overlapping area of each shaded portion is a designated area.

  On the other hand, when the area exclusion type is “exclude the area not designated”, the screen difference detection unit 14 displays the target rectangle as the screen element rectangle list of the target screen if the target rectangle is not included in the specified area. It removes from (S540).

  In the above, an example in which non-overlapping screen element rectangles are set as landmark candidates (landmark rectangles) has been described, but screen element rectangles relating to the same image may be set as landmark rectangles. That is, step S230 of FIG. 7 may not be performed. In this case, each uniqueness may be ensured based on the relative positional relationship of the screen element rectangles. For example, in the case where three screen element rectangles related to the same image are extracted from the correct answer screen, if the upper and lower relationship is clear at these arrangement positions, upper, middle, and lower for three screen element rectangles (landmark rectangles) Add the following identification information. When extracting screen element rectangles that match the landmark rectangles from the comparison screen, screen element rectangles corresponding to each may be extracted based on the relative positional relationship between these three landmark rectangles.

  As described above, according to the present embodiment, the designated area which is the area to be excluded or the area to be tested is expressed by the relative positional relationship based on the landmark rectangle. Therefore, even if the size or position of the dynamic change area changes so that the contents change each time the screen is displayed, it moves appropriately if the relative positional relationship with the landmark rectangle is maintained. Dynamic change area can be identified. As a result, a partial area of the screen can be appropriately removed from the comparison target.

  12 and 13 are diagrams for specifically describing an example of the effect of the present embodiment. FIG. 12 illustrates an example in which the test target area information d1 is generated in order to set the area of “★ ad” as the specified area.

  In this case, as shown in FIG. 13 (1), even on the comparison screen g110 in which the size of the area of “★ ad” has changed, based on the test target area information d1, The area can be identified with high accuracy.

  Further, as shown in FIG. 13 (2), even in the comparison screen g 120 in which the position of “★ ad” is shifted, the area of “★ ad” which is a designated area is accurately determined based on the test target area information d 1 It can be identified.

  Therefore, in any of the comparison screen g110 and the comparison screen g120, for example, the area of "★ advertisement" can be easily excluded from the comparison targets.

  In the present embodiment, the test target area designation unit 13 is an example of a reception unit. The screen difference detection unit 14 is an example of a detection unit. The correct answer screen is an example of a first image showing the first display result. The comparison screen is an example of a second image showing a second display result. The designated area is an example of the first area and the second area. The landmark rectangle is an example of a first screen element rectangle and a second screen element rectangle.

  Although the embodiments of the present invention have been described above in detail, the present invention is not limited to such specific embodiments, and various modifications may be made within the scope of the present invention as set forth in the claims.・ Change is possible.

DESCRIPTION OF SYMBOLS 10 Screen difference detection apparatus 11 Image input part 12 Landmark candidate extraction part 13 Test object area designation part 14 Screen difference detection part 15 Report output part 100 Drive apparatus 101 Recording medium 102 Auxiliary storage apparatus 103 Memory apparatus 104 CPU
105 interface device 106 display device 107 input device B bus

Claims (4)

A screen difference detection device for detecting a difference between a first image indicating a first display result of a screen and a second image indicating a second display result of the screen,
As information for specifying a first area which is a partial area in the first image, each one or more first screen element rectangles of a plurality of screen element rectangles included in the first image A reception unit for receiving from the user a specification of a relative positional relationship based on the first screen element rectangle;
Identification based on each second screen element rectangle that matches the first screen element rectangle in the second image and the relative positional relationship specified for each first screen element rectangle A detection unit that detects the difference by setting the second region to be detected and the first region in the first image as a detection target or non-detection target of the difference;
A screen difference detection apparatus characterized by having: The receiving unit receives, from the user, designation of a direction in which the first area is located with respect to the first screen element rectangle for each of the one or more first screen element rectangles.
The screen difference detection apparatus according to claim 1, wherein The receiving unit receives specification of the first screen element rectangle from among screen element rectangles obtained by removing the same screen element rectangle as an image from a plurality of screen element rectangles included in the first image.
The screen difference detection apparatus according to claim 1 or 2, characterized in that: A screen difference detection device for detecting a difference between a first image showing a first display result of a screen and a second image showing a second display result of the screen,
As information for specifying a first area which is a partial area in the first image, each one or more first screen element rectangles of a plurality of screen element rectangles included in the first image A reception procedure for receiving from the user a specification of a relative positional relationship based on the first screen element rectangle;
Identification based on each second screen element rectangle that matches the first screen element rectangle in the second image and the relative positional relationship specified for each first screen element rectangle A detection step of detecting the difference by setting the second region to be detected and the first region in the first image as a detection target or non-detection target of the difference;
A program characterized by causing

Images