JP2018092361A - Test script correction apparatus and test script correction program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an apparatus which improves accuracy of correction on a test script to be used in a test which requires screen operation.SOLUTION: A test script correction program includes: acquiring a test script including an order of executing operation instructions on screen elements of an old application screen (old screen), and including identification information of the screen elements to be operated, for each operation instruction (S102); acquiring image data in a range of the screen elements to be operated, from the old screen, for each operation instruction included in the test script (S103); and storing the image data in a storage unit, identifying a screen element to be operated in a new screen, on the basis of a comparison between image data stored in the storage unit and image data of screen elements acquired from the new screen, for a first operation instruction of the operation instructions included in the test script, indicating unused identification information on a new application screen (new screen) as a screen element to be operated, and correcting the identification information for the first operation instruction to the identification information of the specified screen element (S105).SELECTED DRAWING: Figure 4

Description

  The present invention relates to a test script correction device and a test script correction program.

  Methods such as Capture & Replay and Programmable Testing are used as methods for automating tests involving screen operations. These tools are used to streamline regression testing. The regression test is a test for confirming that existing functions are not adversely affected when the application is modified. By automating the execution of the regression test, the test man-hours can be expected to be reduced compared to the case where the test is executed manually.

  However, when an application that outputs a screen (for example, a Web application) is modified, the test script may need to be modified accordingly. The test script is data in which screen operation instructions are described. In other words, the operation command described in the test script is automatically executed, thereby automating the test with screen operation. If man-hours are required to modify the test script, the expected man-hour reduction effect may not be obtained.

  It is said that 70% or more of the correction of the test script is correction of the locator (Non-patent Document 3). The locator is an identifier for uniquely specifying a screen element to be operated (one element in a screen such as a button or a form). For example, id, name, XPath, etc. can be used as the locator. When the screen element id, name, XPath, etc. are changed due to the modification of the application, it is necessary to modify the corresponding locator in the test script.

  As a method to reduce the trouble of correcting the locator, it may be possible to prevent the occurrence of locator inconsistency between revisions of the application by implementing the application in advance so as to use a highly robust locator. It is done. A highly robust locator is, for example, a locator that identifies screen elements based on relative information. UFT, Ranorex, etc. exist as tools adopting the first method. There are a plurality of papers proposing the first method (for example, Non-Patent Document 1).

  However, locators that are less likely to cause inconsistencies are less flexible because they vary depending on the type of application and the case. In addition, when a person sets a locator, it is necessary to consider in advance what the locator is. Furthermore, if an inconsistency occurs in the locator due to an unexpected modification of the application, the test script will eventually need to be modified.

  Therefore, it is conceivable to automatically correct the locator to a correct value when a locator mismatch occurs (for example, Non-Patent Document 1, Non-Patent Document 2, etc.). Locator inconsistency refers to a state where there is no screen element corresponding to the locator specified in the test script corresponding to the old revision application on the screen of the new revision application.

Maurizio Leotta, Andrea Stocco, Filippo Ricca, Paolo Tonella, Using Multi-Locators to Increase the Robustness of Web Test Cases, International Conference on Software Testing, Verification and Validation (ICST) 2015 Shauvik Roy Choudhary, Dan Zhao, Husayn Versee, Alessandro Orso, Water: Web Application TEst Repair, Proceedings of the First International Workshop on End-to-End Test Script Engineering (ESTE) 2011 Why do Record / Replay Tests of Web Applications Break ?, Mouna Hammoudi, Gregg Rothermel, Paolo Tonella, Software Testing, Verification and Validation (ICST) 2016

  However, the prior art has a problem that the accuracy of locator correction is low.

  In Non-Patent Document 1, the locator is corrected using five indices similar to XPath (FirePath Abs, FirePath Rel, Selenium IDE (rel xpath), Montoto, ROBULA +). By using a plurality of indexes, matching can be determined from various aspects, and flexibility is increased.

  However, these indexes are all based on the structure information of a DOM (Document Object Model) tree, and as a disadvantage of correction using the structure information, it is weak against structural changes.

  For example, if there is a modification such as a large change in the screen layout or a change in the position of two screen elements, matching between screen elements cannot be performed, and the locator modification fails. Since these five indicators are similar, all are considered to have the same disadvantages. Further, each index is evaluated by matching or non-matching, and since it is not possible to perform an evaluation that does not match but is similar, flexibility is low.

  In Non-Patent Document 2, the locator is corrected using 10 types of indices such as attribute, position, and Xpath. However, since the weighting of the indices is biased, it is substantially equivalent to using only XPath. XPath is evaluated using the Levenshtein distance, not coincidence or disagreement, and the closeness of the position can be taken into account.

  However, since it depends substantially on one kind of index (XPath), there is a drawback that multi-faceted evaluation cannot be performed. Moreover, since it depends only on XPath, the defect of the correction using the structure information appears more strongly, and it is difficult to cope with many corrections involving layout correction.

  The present invention has been made in view of the above points, and an object of the present invention is to improve the accuracy of correction of a test script used for a test involving screen operation.

  Therefore, in order to solve the above-described problem, the test script correction device is a test script in which the execution order of the operation instructions for the screen elements of the first screen is defined, and for each operation instruction, the screen element to be operated is identified. A test script acquisition unit that acquires a test script including information; and, for each operation instruction included in the test script, acquires image data in a range of screen elements to be operated from the first screen, and acquires the acquired image data Among the operation instructions included in the image acquisition unit stored in the storage unit and the test script, the first operation instruction in which identification information that is not used for the second screen is indicated as the screen element to be operated, Based on a comparison between each image data stored in the storage unit and image data of each screen element acquired from the second screen The first specifying unit that specifies the screen element to be operated on the second screen and the identification information for the first operation command are corrected to the identification information of the screen element specified by the first specifying unit. A first correction unit.

  It is possible to improve the accuracy of the correction of the test script used for the test involving the operation of the screen.

It is a figure for demonstrating the system configuration example in embodiment of this invention. It is a figure which shows the hardware structural example of the test execution apparatus 10 in embodiment of this invention. It is a figure which shows the function structural example of the test execution apparatus 10 in embodiment of this invention. 4 is a flowchart for explaining an example of a processing procedure executed by a test execution device 10. It is a flowchart for demonstrating an example of the process sequence of the acquisition process of the screen element information of an old screen. It is a figure for demonstrating an example of a test script. 3 is a diagram illustrating a configuration example of a screen element information storage unit 122. FIG. It is a flowchart for demonstrating an example of the process sequence of the test process of a new application. It is a flowchart for demonstrating an example of the process sequence of the search process of a corresponding element candidate. It is a flowchart for demonstrating an example of the process sequence of the search process based on image data. It is a flowchart for demonstrating an example of the process sequence of the search process based on an index value. It is a flowchart for demonstrating an example of the process sequence of the correction process of an undetermined locator. It is a flowchart for demonstrating an example of the process sequence of a calculation process of a weight.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram for explaining a system configuration example according to an embodiment of the present invention. In FIG. 1, the server device 20 and the test execution device 10 are connected via a network such as a LAN (Local Area Network) or the Internet.

  The server device 20 is a computer having an application that generates a screen. The server device 20 has an old revision and a new revision of the application. For example, a Web server may be used as the server device 20. Note that the screen configuration (that is, the configuration of screen elements included in the screen) does not necessarily match between revisions. In addition, the same screen element across revisions in data (for example, HTML (HyperText Markup Language) or XML (eXtensible Markup Language), etc., hereinafter referred to as “screen definition data”) in which screen elements included in the screen are defined. The identity of the locator assigned to (corresponding screen element) is not guaranteed. The same screen element (corresponding screen element) means, for example, a set of screen elements that have the same function and use and are recognized to be the same when viewed from a human.

  The screen element refers to an element such as a button, an input form, or a link that configures a screen (Web screen or the like). One screen element is defined by, for example, one HTML tag or XML tag, and can have a nested structure. The locator refers to information given as an identifier to each screen element in the screen definition data.

  The test execution device 10 is a computer that executes a regression test on the application of the server device 20. In the regression test, a test script created for a screen generated by an old revision application (hereinafter referred to as “old application”) is generated by a new revision application (hereinafter referred to as “new application”). It is executed efficiently by applying to the screen. The test script is data in which an operation procedure for the screen is defined, and is data including information indicating the locator of the screen element to be operated and the operation content for each operation command. A PC (Personal Computer), a smartphone, a tablet terminal, or the like may be used as the test execution device 10.

  FIG. 2 is a diagram illustrating a hardware configuration example of the test execution device 10 according to the embodiment of the present invention. The test execution device 10 in FIG. 2 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a CPU 104, an interface device 105, a display device 106, an input device 107, and the like that are mutually connected by a bus B.

  A program for realizing processing in the test execution device 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. However, the program need not be installed from the recording medium 101 and may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program and also stores necessary files and data.

  The memory device 103 reads the program from the auxiliary storage device 102 and stores it when there is an instruction to start the program. The CPU 104 realizes functions related to the test execution device 10 in accordance with a program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network. The display device 106 displays a GUI (Graphical User Interface) or the like by a program. The input device 107 includes a keyboard and a mouse, and is used for inputting various operation instructions.

  FIG. 3 is a diagram illustrating a functional configuration example of the test execution device 10 according to the embodiment of the present invention. In FIG. 3, the test execution device 10 includes a test script acquisition unit 11, a test execution unit 12, a screen display control unit 13, an index value acquisition unit 14, an image acquisition unit 15, a corresponding element candidate search unit 16, and a test script correction unit 17. A test script output unit 18 and a weight calculation unit 19. Each of these units is realized by processing that one or more programs installed in the test execution device 10 cause the CPU 104 to execute.

  The test execution apparatus 10 also uses a test script storage unit 121, a screen element information storage unit 122, a screen history storage unit 123, a weight storage unit 124, and the like. Each of these storage units can be realized using, for example, a storage device that can be connected to the auxiliary storage device 102 or the test execution device 10 via a network.

  The test script acquisition unit 11 acquires a test script corresponding to the old application from the test script storage unit 121.

  The screen display control unit 13 displays an old application screen (hereinafter referred to as “old screen”) and a new application screen (hereinafter referred to as “new screen”). For example, the screen display control unit 13 may be realized based on a Web browser.

  The test execution unit 12 uses the test script acquired by the test script acquisition unit 11 to execute a test for each of the old screen and the new screen.

  The index value acquisition unit 14 acquires a plurality of index values (index values) for evaluating the similarity between screen elements for each screen element of the old screen. The acquired index value group is stored in the screen element information storage unit 122.

  The image acquisition unit 15 acquires image data of the display range of the screen element for each screen element of the old screen. The acquired image data is stored in the screen element information storage unit 122.

  When the corresponding element candidate search unit 16 is executing a test for a new application (new screen), any of the locators specified for indicating the screen element to be operated in the test script is not used for the new screen. In addition, a screen element candidate corresponding to the locator is searched from the screen elements of the new screen.

  The test script correcting unit 17 corrects the test script to a state corresponding to the new application (new screen) based on the search result by the corresponding element candidate searching unit 16.

  The test script output unit 18 outputs the corrected test script.

  The weight calculation unit 19 stores the weight for each index, which is used when the corresponding element candidate search unit 16 calculates the similarity between each screen element of the old screen and each screen element of the new screen, in the screen history storage unit 123. Statistically calculated based on stored information. The calculation result (weight of each index) is stored in the weight storage unit 124.

  The screen history storage unit 123 stores image data and an index value group for each screen element of the screen of each revision of the application.

  The test execution device 10 may have the function of the server device 20. In this case, the server device 20 may not be arranged.

  Hereinafter, a processing procedure executed by the test execution device 10 will be described. FIG. 4 is a flowchart for explaining an example of a processing procedure executed by the test execution device 10.

  In step S <b> 101, the screen display control unit 13 displays the old application screen (old screen) of the server device 20 on the display device 106. Specifically, the screen display control unit 13 accesses the old application of the server device 20, downloads the screen definition data, and displays the old screen based on the screen definition data.

  Subsequently, the test script acquisition unit 11 acquires a test script corresponding to the old application from the test script storage unit 121 and stores it in the memory device 103 (S102). In the following, “test script” refers to a test script stored in the memory device 103.

  Subsequently, the test execution device 10 executes screen element information acquisition processing for the old screen (S103). In step S103, for each screen element of the old screen, the values (index values) of a plurality of indices for evaluating the similarity to other screen elements, the image data of the display range of the screen element, and the like are acquired. Is done. The acquired index value group and image data are stored in the screen element information storage unit 122.

  Subsequently, the test execution device 10 executes a test process for the new application (S104). In the test process, a test script corresponding to the old screen is applied to the new screen, and a regression test is executed for the new application. In the regression test process, when a screen element corresponding to the locator that is the operation target of the operation instruction in the test script does not exist on the new screen, the screen element corresponding to the locator is searched. For the search of the screen element, information stored in the screen element information storage unit 122 is used. When the screen element is specified, the locator in the test script is replaced (modified) by the locator of the screen element.

  Subsequently, the test script output unit 18 outputs the test script corrected in step S104 (S105). For example, the test script is stored in the test script storage unit 121 as a test script corresponding to the new revision.

  Next, details of step S103 will be described. FIG. 5 is a flowchart for explaining an example of the processing procedure of the acquisition process of the screen element information of the old screen.

  In step S201, the test execution unit 12 determines whether the next instruction c is present in the test script. The next instruction c is an instruction that is first in execution order among unexecuted instructions.

  FIG. 6 is a diagram for explaining an example of a test script. In FIG. 6, a screen g1 is an example of a displayed screen. The screen definition data h1 is screen definition data for a part of the screen g1 (screen element e1). The table T1 represents the test script related to the screen g1 in a table format.

  In the table T1, each row corresponds to one operation. That is, the test script includes an instruction, a locator, a value, and the like for each operation. The command is information indicating the type of operation. In FIG. 6, “type”, “click”, and the like are illustrated. “Type” means input of a value to the screen element to be operated. “Click” means a click on the screen element to be operated. The locator is a locator of the screen element to be operated. The value is a value applied to the screen element to be operated. As described above, the test script includes the operation content (command and value) and the operation target (locator) for each operation. The test script also defines the order of operations. For example, in the table T1, the process is executed from the top line first.

  When the next instruction c is present in the test script (Yes in S201), the test execution unit 12 executes the instruction c on the screen element corresponding to the instruction c on the old screen (S202).

  Subsequently, the index value acquisition unit 14 specifies a screen element operated by the instruction c (hereinafter referred to as “target element”) (S203). For example, the target element may be specified by a locator corresponding to the instruction c. Subsequently, the index value acquisition unit 14 acquires each index value related to the target element from the screen definition data of the old screen (S204). Here, index values of a plurality of indices are acquired.

  The index includes, for example, screen element attributes (id, type, class, name, tag name, etc.), screen element position information (XPath, x coordinate, y coordinate, horizontal width, vertical width), text associated with the screen element ( Link text, nearby text, URL, etc.). The link text is an index when the screen element is a link, and indicates a character string to which the link is assigned. The nearby text is text that is displayed around the display range of the screen element (for example, within a predetermined number of pixels such as up, down, left, and right). The display range of the screen element may be, for example, the minimum circumscribed rectangle of the screen element. An index value may be acquired for each of about 20 types of indices.

  Subsequently, the index value acquisition unit 14 stores the acquired index value group in the screen element information storage unit 122 in association with the target element (S205).

  Subsequently, the image acquisition unit 15 acquires a screen shot (image data) of the old screen (S206). Subsequently, the image acquisition unit 15 acquires (cuts out) image data of the display range of the target element from the screen shot (S207). The display range of the target element can be specified based on the position information of the target element included in the screen definition data of the old screen, for example. Subsequently, the image acquisition unit 15 stores the acquired image data in the screen element information storage unit 122 in association with the target element (S208).

  When step S202 and subsequent steps are executed for all instructions in the test script (No in S201), the processing procedure in FIG. 5 ends. As a result, the screen element information storage unit 122 stores an index value group and image data of each screen element operated by the test script among the screen elements of the old screen.

  FIG. 7 is a diagram illustrating a configuration example of the screen element information storage unit 122. As illustrated in FIG. 7, the screen element information storage unit 122 stores an index value group and image data for each screen element to be operated in the order of instructions executed in the test script.

  Next, details of step S104 in FIG. 4 will be described. FIG. 8 is a flowchart for explaining an example of a processing procedure of a test process for a new application.

  In step S <b> 301, the screen display control unit 13 displays a new application screen (new screen) of the server device 20 on the display device 106. Specifically, the screen display control unit 13 accesses a new application of the server device 20, downloads screen definition data, and displays a new screen based on the screen definition data.

  Subsequently, step S302 and subsequent steps are executed in order from the first instruction of the test script. The test script is a test script stored in the memory device 103 in step S102. That is, the test script is a test script corresponding to the old application.

  In step S302, the test execution unit 12 determines whether or not the next instruction c is present in the test script. When the test script includes the next command c (Yes in S302), the test execution unit 12 executes the command c on the screen element corresponding to the command c on the new screen (S303).

  Subsequently, the test execution unit 12 determines whether or not the instruction c is an assertion (S304). An assertion is a command for verifying that an application state (screen state) is as expected. For example, an assertion is included in the test script for each group of instructions. For example, in the table T1 of FIG. 6, the last line instruction is an assertion. If the instruction is an assertion, the value means the expected value. However, the test script may not include an assertion.

  When the instruction c is not an assertion (No in S304), the test execution unit 12 determines whether or not the screen element having the locator of the instruction c is included in the new screen (S305). This determination can be made by referring to the screen definition data of the new screen. When the screen element having the locator of the instruction c is included in the new screen (No in S305), Step S302 and subsequent steps are repeated.

  When the screen element having the locator of the instruction c is not included in the new screen (Yes in S305), the corresponding element candidate search unit 16 performs a corresponding element candidate search process for the locator (hereinafter referred to as “original locator”). Is executed (S306). In the corresponding element candidate search process, screen element candidates (hereinafter referred to as “corresponding element candidates”) corresponding to the locator of the instruction c are searched for in the screen elements of the new screen (hereinafter referred to as “corresponding element candidates”). Is done. In the search, a screen element that is recognized as a corresponding element candidate among screen elements of the new screen is similar to a screen element having the locator on the old screen (hereinafter referred to as “original element”). The degree is calculated. Details of the similarity will be described later.

  If the screen element having the locator of the instruction c is not included in the new screen, the execution of the instruction c fails in step S303. Therefore, step S305 may be executed when the execution of the instruction c fails.

  Subsequently, the test script correcting unit 17 determines whether or not there is a corresponding element candidate (S307). That is, in the corresponding element candidate search process, it is determined whether or not one or more corresponding element candidates have been specified (discovered). If there is no corresponding element candidate (No in SS307), the processing procedure in FIG. 5 ends.

  On the other hand, when one or more corresponding element candidates are identified (Yes in S308), the test script correcting unit 17 uses the locator of the corresponding element candidate having the maximum similarity among the corresponding element candidates to execute an instruction in the test script. The locator of c is corrected (S309). That is, the locator of the instruction c in the test script is replaced with the corresponding element candidate locator. Subsequently, the test execution unit 12 executes the instruction c (S309). Subsequent to step S309, steps after step S302 are repeated. As described above, in the present embodiment, when an instruction in which a screen element corresponding to a locator is not found is executed, the locator is corrected at that time and the test is continued.

  On the other hand, when the instruction c is an assertion (Yes in S304), the test script correcting unit 17 determines whether or not the execution result of the assertion is acceptable (S310). That is, it is determined whether or not the state of the new application (new screen) is as expected in the instruction c (assertion) executed in step S303. This determination can be made based on the execution result of the assertion.

  When the execution result of the assertion is acceptable (Yes in S310), Step S302 and subsequent steps are repeated. If the execution result of the assertion is not acceptable (No in S310), the test script correcting unit 17 executes an undetermined locator correcting process (S311). An indeterminate locator is a locator for each instruction included between the (N-1) th assertion and the Nth assertion, assuming that the assertion that did not pass is the Nth assertion. However, when N = 1, the locators of all instructions prior to the Nth assertion become undetermined locators.

  That is, if the assertion passes, the locator for the instruction prior to the assertion is determined. The fact that the assertion has passed means that the state of the new application (new screen) is as expected. Therefore, even if any of the locators executed so far has been corrected in step S308, it is determined that the correction was correct. On the other hand, if the assertion does not pass, there is a possibility that the correction in step S308 was an error. In this case, step S311 is executed.

  Subsequent to step S311, steps after step S301 are executed again. In other words, if the assertion does not pass, an indeterminate locator correction process is executed, a new screen is downloaded again, and the test script is executed from the beginning. If the assertion does not pass, the internal state of the new application may not be the original state when executed up to instruction c. Even if the test is continued as it is, there is a possibility that the correct test cannot be performed. It is.

  If it is determined in step S302 that all instructions in the test script have been executed (No in S302), the processing procedure in FIG. 8 ends.

  Next, details of step S306 in FIG. 8 will be described. FIG. 9 is a flowchart for explaining an example of the processing procedure of the corresponding element candidate search processing.

  In step S401, the corresponding element candidate search unit 16 acquires screen element information (index value group and image data) of the screen element (hereinafter referred to as “screen element x”) corresponding to the original locator, and the screen element information storage unit 122. Get from. Subsequently, the corresponding element candidate search unit 16 acquires each index value group and image data of all screen elements of the new screen from the screen definition data of the new screen (S402). The set of all screen elements is hereinafter referred to as “screen element set Y”. Further, the method for acquiring the index value group and image data of the screen element of the new screen (image data of the display range of the screen element) may be the same as the method of acquiring the screen element of the old screen.

  Subsequently, the corresponding element candidate search unit 16 executes a search process based on the image data (S403). The screen element yεY determined to be a corresponding element candidate in the search process is added to the screen element set Y ′. Note that the screen element set Y ′ is empty at the start of search processing based on image data.

  If the number of elements of the screen element set Y ′ is plural after execution of the search process based on the image data (Yes in S404), the corresponding element candidate search unit 16 executes the search process based on the index value (S405). On the other hand, when the number of elements of the screen element set Y ′ is 1 or less (No in S404), the corresponding element candidate search unit 16 substitutes the screen element set Y ′ for the screen element set Y ″ (S406). The screen element set Y ″ is a set of corresponding element candidates. That is, the screen element set Y ″ is referred to in steps S307 and S308 of FIG.

  Next, details of step S403 in FIG. 9 will be described. If the images (appearance) of two screen elements match, they are considered to have the same role with a high probability. Therefore, in the present embodiment, by using the image data of the screen elements, the corresponding element candidates are narrowed down to improve the accuracy and shorten the processing time.

  FIG. 10 is a flowchart for explaining an example of a processing procedure of search processing based on image data.

  First, steps S501 to S503 are executed for all screen elements yεY. In step S502, the corresponding element candidate search unit 16 compares the image data of the screen element x and the image data of the screen element y, and if they match, the screen element y is changed to the screen element set Y ′. to add. Note that the matching of the image data need not be limited to perfect matching (a state in which all pixels match). For example, the pattern matching may be performed by shifting the image of one image data by a predetermined number of pixels with respect to the image of the other image data to determine whether or not they match. That is, in the field of image processing technology, when it can be determined that two pieces of image data indicate the same object, it may be determined that the two match.

  When steps S501 to S503 are executed for all screen elements yεY, the corresponding element candidate search unit 16 determines whether or not one or more screen elements y are included in the screen element set Y ′ (S504). ). When the screen element set Y ′ includes one or more screen elements (Yes in S504), the processing procedure in FIG. 10 ends. When the surface element set Y ′ is empty (No in S504), the corresponding element candidate search unit 16 adds the screen element set Y to the screen element set Y ′ (S505).

  Next, details of step S405 in FIG. 9 will be described. For example, when screen element X (id = login, XPath = // A / B / X) is modified to become screen element X ′ (id = user_login, XPath = // C / D / X), non- It is difficult to cope with the methods disclosed in Patent Document 1 and Non-Patent Document 2 (determining that screen element X ′ is a corresponding element candidate for screen element X).

  In the method of Non-Patent Document 1, an index representing a relative structure (for example, a screen element having a tag X as a child of a screen element having id = foo) is also used. However, in this modification example, the parent of X before modification is different from A and B, and the parent of X after modification is different from C and D. Indicators do not match.

  In the method of Non-Patent Document 2, determination is made based on the Lavenstein distance of XPath. If there is another screen element Y having XPath such as // A / B / Y, the screen elements X and Y It is determined that the similarity is higher than the similarity between the screen element X and the screen element X ′.

  The above problem is caused by evaluating only matching or mismatching of indices or depending on one index.

  In the present embodiment, as described in FIG. 10, the corresponding element candidate is determined based on the image data of the screen element, or as described below with reference to FIG. Instead, the accuracy is calculated by calculating the similarity as a real value and integrating them into one value.

  FIG. 11 is a flowchart for explaining an example of the processing procedure of the search processing based on the index value. In FIG. 11, steps S601 to S613 are executed for all screen elements y included in the screen element set Y ′. In addition, steps S602 to S609 are executed for all indices iεI. Note that I is a set of all indexes in this embodiment (id, type, class, name, tag name, XPath, x coordinate, y coordinate, horizontal width, vertical width, link text, nearby text, URL, etc.) It is.

  In step S603, the corresponding element candidate search unit 16 determines whether or not the index value of the index i is selective. For example, the value of id can be arbitrarily set, but the value of type is selected from predetermined values (input, button, etc.). The tag name is the same. In step S <b> 603, it is determined whether or not the index i is an index value selected from options determined in advance.

When the index value of the index i is selective (Yes in S603), the corresponding element candidate search unit 16 determines whether the index values of the respective indices i are the same for the screen element x and the screen element y. Based on this, the similarity s _i of the index i between the screen element x and the screen element y is determined (S604). That is, if the index values of the respective indices i match, the similarity s _i of the index _i is set to 1, and if not, the similarity s _i of the index _i is set to 0. That is, the similarity s _i for the index i that is selective is 0 or 1. In the present embodiment, the similarity s is set to a value not less than 0 and not more than 1 for all indexes. However, the similarity s may be evaluated by other scales (for example, 0 to 100, etc.).

When the index value of the index i is not selective (No in S603), the corresponding element candidate search unit 16 determines whether or not the index value of the index i is an arbitrary character string (S605). For example, id, class, name, XPath, etc. are arbitrary character strings. When the index value of the index i is an arbitrary character string (Yes in S605), the corresponding element candidate search unit 16 for the screen element x and the screen element y is based on the Levenshtein distance of the index value of each index i. Then, the similarity s _i of the index i between the screen element x and the screen element y is calculated (S606). In particular,
Similarity s _i = 1−Levenstein distance / similarity s _i of index i between screen element x and screen element y is calculated based on the longer character string length. The length of the longer character string is the number of characters in the longer character string of the index value (character string) of the index i of the screen element x and the index value (character string) of the index i of the screen element y. Say.

When the index value of the index i is not an arbitrary character string (No in S605), the corresponding element candidate search unit 16 determines whether or not the index value of the index i is a numerical value (S607). For example, the x coordinate, the y coordinate, the horizontal width, the vertical width, etc. are numerical values. When the index value of the index i is a numerical value (Yes in S607), the corresponding element candidate search unit 16 selects the screen element for the screen element x and the screen element y based on the Euclidean distance of the index value of each index i. The similarity s _i of the index i between x and the screen element y is calculated (S608). In particular,
Similarity s _i = 1− | difference between two index values / maximum value |
Based on the above, the similarity s _i of the index i between the screen element x and the screen element y is calculated. Note that the difference between the two index values refers to the difference between the index value of the index i of the screen element x and the index value of the index i of the screen element y. The maximum value is the maximum value that can be taken for the index i. For example, if the index i is the x coordinate, the maximum value is the horizontal width of the screen.

  Note that it is known in the algorithm of the corresponding element candidate search unit 16 whether the index value of each index i corresponds selectively, any character string, or a numerical value.

For all indices i∈I, the step S602~S609 are performed, the corresponding element candidate search unit 16, the weight _{W i} for each index i, obtained from the weight storage unit 124 (S610). Incidentally, the weight W _i, for example, as described below, is pre-calculated by the weight calculator 19. However, the weight W _i may be set by any user.

Subsequently, the corresponding element candidate search unit 16 calculates the similarity S _total between the screen element x and the screen element y based on the following formula (S611).

S _total = Σ (s _i × W _i ) / ΣW _i
That is, the similarity S _total is calculated by calculating a weighted average of the index value of the index _i with the weight as _Wi .

Subsequently, the corresponding element candidate search unit 16 adds the screen element y to the screen element set Y ″ if the similarity S _total is larger than the preset threshold value α (S612).

  When steps S601 to S613 are executed for all the screen elements yεY ′, the processing procedure of FIG. 11 ends.

  Next, details of step S311 in FIG. 8 will be described. FIG. 12 is a flowchart for explaining an example of a processing procedure of correction processing for an undetermined locator.

  In step S701, the test script correcting unit 17 extracts an instruction group C including the locator corrected in step S308 from the instruction group corresponding to the undetermined locator. The instruction group corresponding to the undetermined locator means the locator of the instruction included between the (N-1) th assertion and the Nth assertion if the last executed assertion is the Nth assertion. . However, when N = 1, the locators of all instructions prior to the Nth assertion become undetermined locators.

Subsequently, the test script correcting unit 17 acquires the instruction c _last having the last execution order from the instruction group C (S702). Subsequently, the test script correcting unit 17 determines whether or not there is an unsuccessful corresponding element candidate for the instruction c _last (S703). For instructions c _last the corresponding component candidate untried, as unmodified locator original locator instruction c _last, among the searched corresponding component candidate in step S306, has not been used in the locator of the correction in step S308 corresponds An element candidate.

When there is no corresponding corresponding element candidate (No in S703), the test script correcting unit 17 deletes the instruction c _last from the instruction group C (S704), and repeats step S702 and subsequent steps. In step S704, the instruction c _last is deleted from the test script. This is because there is a high possibility that the instruction c _last is not an appropriate instruction for the new application.

On the other hand, when there is one or more corresponding element candidates (Yes in S703), the test script correcting unit 17 uses the corresponding element candidate locator having the maximum similarity S _total to the original element among the corresponding element candidates. The locator of the instruction c _last is replaced (S705).

As described above, in the present embodiment, when an instruction for a locator that is not used in the new screen is executed, the corresponding element candidates for the locator are tried in descending order of the similarity S _total . Corresponding element candidates that have been successfully tested (passed the assertion) are identified as correct corresponding elements.

  Subsequently, a processing procedure executed by the weight calculation unit 19 will be described. The processing procedure may be executed asynchronously with each processing procedure described above. For example, the processing procedure may be executed every time a new revision occurs for the test target application.

For example, when the weight of each index is set arbitrarily by the user, the following problems can be considered.
(A) Important indicators differ depending on the application.
(B) It is not easy to determine which index is important.
(C) As development progresses, important indicators change.

  Therefore, in this embodiment, appropriate weighting is performed for each application using a statistical method.

In the present embodiment, the determination (matching) of the correspondence between screen elements is performed based on the degree of matching (similarity s) of each index. Therefore, the following two ways of thinking are adopted as the weighting method.
(1) The weight of an index that is likely to change due to application modification is reduced, and the weight of an index that is difficult to change is increased. For example, since id is difficult to change, importance is placed on matching, but XPath is easy to change and is not important.
(2) The weight of an index having a high probability of having the same value between different screen elements is reduced, and the weight of an index having a low probability of having the same value is increased. For example, since id does not have the same value between screen elements, the weight is increased. However, since the value of class may be duplicated among a plurality of screen elements, the weight is decreased.

  In order to determine the weight of the index, it is necessary to evaluate the variability of the index between the same screen elements and the ease of matching of the indices between different screen elements. Hereinafter, an index for evaluating the variability of the index between the same screen elements is referred to as “change rate”. In addition, an index for evaluating the ease of matching of indexes between different screen elements is referred to as “match rate”. Note that “between the same screen elements” refers to a set of screen elements corresponding to the revisions (the same screen elements as viewed from the user). Between different screen elements means a set of screen elements that do not correspond between revisions (screen elements that differ from the user's perspective).

  Based on the above idea, the weight calculator 19 calculates the weight of each index by the processing procedure shown in FIG. FIG. 13 is a flowchart for explaining an example of a processing procedure of weight calculation processing.

  Steps S801 to S813 are executed for each revision from revision v to revision v + 1.

  Steps S802 to S812 are executed for each screen element aεA included in the screen element set A of revision k. Note that the initial value of revision k is v.

  Further, steps S803 to S810 are executed for each screen element a ′ included in the screen element set A ′ of revision k + 1.

  In step S804, the weight calculation unit 19 determines whether or not the image data of the screen element a matches the image data of the screen element a ′. The method for determining whether the image data matches or does not match may be the same as the method in step S502 of FIG. Note that image data of each screen element of each screen of each revision is stored in the screen history storage unit 123.

  When the compared image data matches (Yes in S804), the weight calculation unit 19 compares the index value of the screen element a and the index value of the screen element a ′ for each index (S805). The index value of each index of each screen element of each screen of each revision is stored in the screen history storage unit 123.

  If the index values match for all indices (No in S805), Step S804 and subsequent steps are executed for the next screen element a ′ of revision k + 1. On the other hand, when the index values are different for any one or more indices i (Yes in S805), the weight calculation unit 19 adds 1 to the counter c (i) corresponding to each index i having a different index value (S806). . Note that the counter c is an array of counters for each index, and is used to count the number of times that index values differ between the same screen elements for each index. At the start of the processing procedure of FIG. 13, the value of each element of the counter c is 0.

  Subsequently, the weight calculation unit 19 adds 1 to the counter d (S807). The counter d is a counter used for counting the number of times that the index value of any index is different between the same screen elements. At the start of the processing procedure of FIG. 13, the value of the counter d is 0.

  On the other hand, when the image data compared in step S804 does not match (No in S804), the weight calculation unit 19 determines that there is one or more indexes with the same index value between the screen element a and the screen element a ′. It is determined whether or not (S808). If there is no corresponding index i (No in S808), Step S804 and subsequent steps are executed for the next screen element a ′ of the revision k + 1. When there is a corresponding index i (Yes in S808), the weight calculation unit 19 adds 1 to the counter e (i) corresponding to the index i with the matching index value. The counter e is an array of counters for each index, and is used to count the number of times that index values match between different screen elements for each index value. At the start of the processing procedure of FIG. 13, the value of each element of the counter e is 0.

  When Steps S803 to S810 are executed for all screen elements a′εA ′ of the revision k + 1, the weight calculation unit 19 adds 1 to the counter f (S811). The counter f is a counter used for counting the total number of screen elements of all revisions (total number of screen elements). For example, for the revision v, when the execution of steps S802 to S812 is completed, the counter f stores the total number of screen elements of the revision v. Note that the value of the counter f is 0 at the start of the processing procedure of FIG.

When steps S801 to S813 are executed from revision v to revision v + n, the weight calculator 19 calculates the rate of change of each index (S814). The rate of change (i) of the index i is calculated based on the following equation.
Rate of change (i) = c (i) / d
That is, the rate of change (i) indicates the number of times that the index i has changed (a different number of times) in the comparison of each index between the same screen elements, and the number of times that any index has changed in the comparison of each index between the same screen elements ( It is calculated by dividing by (different number of times).

Subsequently, the weight calculation unit 19 calculates the matching rate of each index (S815). The coincidence rate (i) of the index i is calculated based on the following formula.
Match rate (i) = e (i) / f
That is, the coincidence rate (i) is calculated by dividing the number of times the index i matches in the comparison of each index between different screen elements by the total number (total number) of screen elements.

Subsequently, the weight calculator 19 calculates the weight of each index (S816). The weight W _{i of the} index i is calculated based on the following formula.
W _i = change rate (i) × (1−match rate (i)) × constant β
According to the processing procedure of FIG. 13, different weighting can be automatically performed for each application, and when an important index changes as development progresses by devising such as using the latest 10 revisions for weighting. Can also be supported.

  As described above, according to the present embodiment, the correspondence between screen elements is determined based on the image data of each screen element. Here, a screen element having the same image data (appearance) is highly likely to be a screen element having the same usage and function even if there is a difference between the locator and each index. Therefore, it is possible to improve the accuracy of determining the correspondence between screen elements. As a result, automatic correction of the locator in the test script can be performed with higher accuracy and efficiency. That is, it is possible to improve the accuracy of the correction of the test script used for the test involving the operation of the screen.

  Traditionally, automation of tests involving screen operations has often been avoided cost-effectively due to heavy maintenance, but according to the present embodiment, there is a barrier to test automation involving screen operations. This can make software development more efficient.

  Further, in this embodiment, in order to further increase the determination accuracy of the correspondence relationship between the screen elements, the similarity is calculated for each of the plurality of indices, and the similarity between the screen elements is calculated based on the similarity. . In other words, in the present embodiment, the possibility of matching (similarity) is determined, not just whether or not the screen elements match. By doing so, a plurality of corresponding candidates can be obtained from the screen elements of the new revision for a screen element of the old revision. As a result, it is possible to increase the possibility of correctly searching for the corresponding screen element.

  In addition, since it takes time to calculate the similarity for all screen elements in the new screen, in this embodiment, the calculation target of the similarity is narrowed (filtered) based on the comparison of image data. Shortening of processing time can be expected.

  Further, in the present embodiment, even when inconsistencies exist for a plurality of locators, the test can be continued while correcting the locators, so that the execution speed can be improved.

  That is, if a test is executed from the beginning every time one locator is corrected, it is necessary to execute the number of locator errors plus one test for one test script. Tests that involve screen operations require browser startup, screen loading, etc., so the overhead is larger than unit tests, and if the test is executed from the beginning each time the locator is modified, these overheads cause the test. Man-hours increase.

  In order to avoid such an increase in the number of test steps, a method of ignoring the locator error by dynamic program rewriting, etc., forcibly continuing the test execution while trying various inputs, and correcting the locator can be considered.

  However, in this case, there is a possibility that the internal state of the program should not be originally when any operation is performed on the screen, and even if all the locators can be corrected at once, the validity of the test execution can be reduced. There is a possibility that it cannot be kept. It may be possible to force all the inconsistent locators to be corrected and then re-run the test, but for applications with many dynamic elements, the possibility of correct correction is low and the number of trials may increase. is there.

  Therefore, in this embodiment, the program is not re-executed while simply correcting the locator, but the program is re-executed as necessary (when the assertion is not passed) while maintaining the consistency of the internal state of the program. Done.

  In the present embodiment, the test execution device 10 is an example of a test script correction device. The corresponding element candidate search unit 16 is an example of a first specifying unit and a second specifying unit. The test script correction unit 17 is an example of a first correction unit and a second correction unit. The weight calculation unit 19 is an example of a calculation unit. The screen element information storage unit is an example of a storage unit. A locator is an example of identification information. The old screen is an example of a first screen. The new screen is an example of a second screen. The value of the counter c is an example of the first number of times. The value of the counter d is an example of the second number of times. The value of the counter e is an example of the third number of times.

  As mentioned above, although the Example of this invention was explained in full detail, this invention is not limited to such specific embodiment, In the range of the summary of this invention described in the claim, various deformation | transformation・ Change is possible.

DESCRIPTION OF SYMBOLS 10 Test execution apparatus 11 Test script acquisition part 12 Test execution part 13 Screen display control part 14 Index value acquisition part 15 Image acquisition part 16 Corresponding element candidate search part 17 Test script correction part 18 Test script output part 19 Weight calculation part 20 Server apparatus 100 drive device 101 recording medium 102 auxiliary storage device 103 memory device 104 CPU
105 interface device 106 display device 107 input device 121 test script storage unit 122 screen element information storage unit 123 screen history storage unit 124 weight storage unit B bus

Claims (6)

A test script in which an execution order of operation instructions for the screen elements of the first screen is defined, and a test script acquisition unit that acquires a test script including identification information of a screen element to be operated for each operation instruction;
For each operation command included in the test script, an image acquisition unit that acquires image data in a range of screen elements to be operated from the first screen and stores the acquired image data in a storage unit;
Among the operation commands included in the test script, each image stored in the storage unit for the first operation command in which identification information that is not used for the second screen is indicated as a screen element to be operated. A first specifying unit for specifying a screen element to be operated on the second screen based on comparison of data and image data of each screen element acquired from the second screen;
A first correction unit that corrects the identification information for the first operation command to the identification information of the screen element specified by the first specification unit;
A test script correcting device comprising: For each operation command included in the test script, for each screen element to be operated, a plurality of index values used for calculating the similarity between the screen elements are acquired for the first screen, and each acquired value An index value acquisition unit for storing in the storage unit,
When a plurality of screen elements are specified by the first specifying unit, based on each value stored in the storage unit and a value of each index of each screen element acquired with respect to the second screen The similarity between each screen element of the first screen and each screen element of the second screen is calculated, and the screen element to be operated on the second screen is calculated based on the calculated similarity. A second identifying part to identify;
The test script correcting device according to claim 1, comprising: The second specifying unit calculates, for each index, a similarity between a value related to the first screen and a value related to the second screen, and the similarity for each index is used as a weight for each index. Calculating a weighted average based on the similarity between each screen element of the first screen and each screen element of the second screen;
The test script correcting device according to claim 2, wherein For a plurality of screens, a first number of times different in value between screen elements having the same image data is counted for each index, and a second number of times that the value of any of the indices is matched between the screen elements. The third number of times of counting and the number of times that the values match between screen elements that do not match the image data is counted for each index, and the value obtained by dividing the first number by the second number of times and the third number of times , Based on a value obtained by dividing by the total number of screen elements in the plurality of screens, a calculation unit that calculates a weight for each index,
4. The test script correcting device according to claim 3, further comprising: The test script includes assertions in any of the execution order;
When executing the assertion, if the second screen is not in a state expected by the assertion, a second one that corrects any identification information corrected by the first correction unit before the assertion. Having a correction part,
Re-execution from the first operation instruction of the test script when correction is performed by the second correction unit;
The test script correcting device according to claim 1, wherein the test script correcting device is a test script correcting device. A test script in which an execution order of operation instructions for the screen elements of the first screen is defined, and for each operation instruction, a test script acquisition procedure for acquiring a test script including identification information of an operation target screen element;
For each operation command included in the test script, an image acquisition procedure for acquiring image data in a range of screen elements to be operated from the first screen and storing the acquired image data in a storage unit;
Among the operation commands included in the test script, each image stored in the storage unit for the first operation command in which identification information that is not used for the second screen is indicated as a screen element to be operated. A first specifying procedure for specifying a screen element to be operated on the second screen based on a comparison between the data and image data of each screen element acquired from the second screen;
A first correction procedure for correcting the identification information for the first operation command into the identification information of the screen element specified by the first specification procedure;
A test script correction program characterized by causing a computer to execute.

Images