WO2022070353A1

WO2022070353A1 - Information processing system, information processing method, and program

Info

Publication number: WO2022070353A1
Application number: PCT/JP2020/037307
Authority: WO
Inventors: 良近澤; 颯太山下; 敬太守屋
Original assignee: オーティファイ株式会社; オーティファイ，インコーポレーテッド
Priority date: 2020-09-30
Filing date: 2020-09-30
Publication date: 2022-04-07

Abstract

The present invention comprises: a determination unit which, when testing a second system screen displayed on a display unit and changed from a first system screen according to a scenario generated on the basis of the first system screen displayed on the display unit, determines a second element changed from a first element composing the first system screen in the second system screen; a display processing unit which displays, on the display unit, information indicating the determination result from the determination unit; an acquisition unit which acquires, with respect to the determination result, information about an operation input of the user indicating that the second element corresponds to the first element; and a scenario correction unit which corrects the first element in the scenario to the second element on the basis of the information about the operation input of the user acquired by the acquisition unit.

Description

Information processing systems, information processing methods, and programs

The present invention relates to an information processing system, an information processing method, and a program.

A operation verification device that identifies and displays a discrepancy between the previous test result data executed by the test program and the current test result data is disclosed (for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2011-164786

The operation verification device described in Patent Document 1 acquires a log in which the identification information of the object included in the web page to be confirmed to be operated and the combination of the data input by the data input operation for the object are recorded. The operation verification device automatically executes a data input operation based on a log for a web page. As a result of the execution, the operation verification device compares the previous test result data with the current test result data, and outputs a comparison result image in which the inconsistent portion is identified and displayed. This makes it easy to compare and verify the output results in the operation verification test of the web page.

However, in the operation verification device described in Patent Document 1, if there is a changed object in the previous test result data and the current test result data, the user has to modify the test scenario (log) for the changed object.

Therefore, it is an object of the present invention to reflect the procedure including the changed element (object) of the system screen in the test scenario by a simple method in order to solve the above-mentioned problem.

The information processing system according to one aspect of the present invention is displayed on the display unit, which is changed from the first system screen according to the scenario generated based on the first system screen displayed on the display unit. When testing the system screen, a discriminating unit that discriminates the second element changed from the first element constituting the first system screen on the second system screen and information indicating the discriminating result in the discriminating unit are displayed. A display processing unit to be displayed on the unit, an acquisition unit for acquiring information on a user's operation input indicating that the second element is an element corresponding to the first element, and an acquisition unit for the determination result. A scenario correction unit for modifying the first element in the scenario to the second element is provided based on the information regarding the operation input of the user.

The information processing method according to one aspect of the present invention is displayed on the display unit modified from the first system screen according to the scenario generated by the computer based on the first system screen displayed on the display unit. When testing the second system screen, the second element changed from the first element constituting the first system screen on the second system screen is discriminated, and the second element is discriminated. Information indicating the result is displayed on the display unit, and information regarding the user's operation input indicating that the second element is an element corresponding to the first element is acquired for the determination result. Based on the information regarding the operation input of the user, the first element in the scenario is modified to the second element, and so on.

The program according to one aspect of the present invention is displayed on the display unit modified from the first system screen according to a scenario generated based on the first system screen displayed on the display unit on the computer. 2 When testing the system screen, the second element changed from the first element constituting the first system screen on the second system screen is discriminated, and the discriminant result in which the second element is discriminated is determined. Displaying the information to be shown on the display unit, acquiring information regarding the user's operation input indicating that the second element is an element corresponding to the first element, and the acquired information regarding the determination result. Based on the information regarding the operation input of the user, the modification of the first element in the scenario to the second element is executed.

According to the present invention, the procedure including the changed system screen element can be reflected in the test scenario by a simple method.

It is a figure which shows the structure of the test automation system which concerns on this embodiment. It is a figure which shows the outline of the process of the test automation system which concerns on this embodiment. It is a figure which shows an example of the functional structure of the test automation server which concerns on this embodiment. It is a figure which shows an example of the scenario information. It is a figure which shows an example of a feature information. It is a figure which shows an example of the test result information. It is a figure which shows an example of the 1st system screen. It is a figure which shows an example of the 2nd system screen including the 2nd element. It is a figure which shows an example of a scenario screen. It is a figure which shows an example of a test result screen. It is a figure which shows an example of the element comparison screen. It is a figure which shows an example of the functional structure of the user terminal which concerns on this embodiment. It is a flowchart which shows the procedure of the information processing of the test automatic system which concerns on this embodiment. It is a figure which shows an example of the comparison selection screen which selectively displays a plurality of elements discriminated by a discriminating part. It is a figure which shows the other example of the comparison selection screen which selectively displays a plurality of elements discriminated by the discriminant part. It is a figure which shows the other example of the comparison selection screen which selectively displays a plurality of elements discriminated by the discriminant part. It is a figure which shows an example of the hardware configuration of a test automation server. It is a figure which shows an example of the hardware composition of a user terminal.

Hereinafter, the test automation system 10 according to the embodiment of the present invention will be described in detail with reference to the drawings. However, the embodiments described below are merely examples, and there is no intention of excluding the application of various modifications and techniques not specified below. That is, the present invention can be variously modified or combined with each embodiment without departing from the spirit of the present invention. Further, in the description of the following drawings, the same or similar parts are represented by the same or similar reference numerals.
=== Configuration of test automation system 10 ===

First, with reference to FIG. 1, an outline of the configuration of the test automation system 10 in the present embodiment will be described. FIG. 1 is a diagram showing a configuration of a test automation system 10. As shown in FIG. 1, the test automation system 10 includes, for example, a test automation server 100 and a user terminal 200. In the test automation system 10, the test automation server 100 and the user terminal 200 are connected via the network N. The test automation system 10 provides a service for providing various information from the test automation server 100 to a plurality of user terminals 200 via the network N.

The test automation server 100 executes a system screen test according to a scenario (hereinafter referred to as a test scenario) for testing the function of the screen (hereinafter referred to as a system screen) displayed on the display unit 205a of the user terminal 200, for example. It is a device to do. When the test is executed, the test automation server 100 determines the changed element among the elements of the system screen to be tested. The test automation server 100 receives a simple operation input by the user via the user terminal 200, and reflects the procedure including the determined element in the test scenario. Here, the system screen includes, for example, a web page displayed on the display unit 205a, a screen by a mobile application, and the like. That is, the test automation server 100 determines that the elements have been changed, for example, with respect to a test scenario for testing the function of a web page and a test scenario for testing the function of a screen by a mobile application operating on a mobile terminal or the like. It may have a function to reflect the included procedure.

The test automation server 100 includes, for example, a server computer, a personal computer (eg, desktop, laptop, tablet, etc.), a media computer platform (eg, cable, satellite set-top box, digital video recorder), a handheld computer device (eg, PDA). , E-mail client, etc.), or other types of computers, or communication platforms.

The user terminal 200 is, for example, a device for displaying various information transmitted from the test automation server 100 on the display unit 205a. The user terminal 200 receives an operation input from the user and transmits various information to the test automation server 100.

The user terminal 200 may be any information processing device (computer) that can realize the functions in each embodiment. The user terminal 200 is, for example, a smartphone, a mobile phone (feature phone), a personal computer (for example, a desktop, a laptop, a tablet, etc.), a media computer platform (for example, a cable, a satellite set top box, a digital video recorder), a handheld computer. Includes devices (eg, PDA (Personal Digital Assistant), e-mail clients, etc.), wearable terminals (glass-type devices, watch-type devices, etc.), other types of computers, or communication platforms.

Next, the outline of the processing of the test automation system 10 will be described with reference to FIG. FIG. 2 is a diagram showing an outline of processing of the test automation system 10 according to the present embodiment. First, in steps S10 to S13, an outline of a process for creating a test scenario for testing the function of the system screen (hereinafter referred to as a scenario creation process) will be described. Next, in steps S20 to S24, an outline of a process of executing a system screen test according to the test scenario and modifying the test scenario based on the executed result (hereinafter referred to as a scenario modification process) will be described.

First, in step S10, the user terminal 200 sets test conditions on, for example, a setting screen (not shown) displayed on the display unit 205a. In the "test condition", for example, a system screen for creating a test scenario (hereinafter referred to as a first system screen) is displayed, for example, a browser type, an OS (Operating System) of a user terminal 200, and a user terminal 200. The device specifications, the size of the display screen of the user terminal 200, and the like may be included. Next, the user terminal 200 may accept a URL (Uniform Resouce Locator) corresponding to the first system screen by inputting a user's operation on the setting screen displayed on the display unit 205a, for example. Hereinafter, for convenience, the information regarding the test conditions will be referred to as "setting information".

Next, in step S11, the test automation server 100 generates information about the first system screen, for example, based on the setting information. The test automation server 100 transmits information about the first system screen to the user terminal 200.

Next, in step S12, the user terminal 200 causes, for example, the display unit 205a to display the first system screen. For example, when the user terminal 200 accepts an operation input by the user for the "record" button (see FIG. 7), the information regarding the operation input by the user after that (hereinafter referred to as operation information) is test-automated. Send to server 100.

Next, in step S13, the test automation server 100 creates a test scenario based on, for example, operation information. The test automation server 100 stores the created test scenario in the storage unit 110.

In this way, the test automation system 10 creates a test scenario by inputting an operation to the first system screen by the user. Therefore, even a user who does not know how to create a test scenario by code can easily create a test scenario. In addition, since the time and effort of the user who creates the test code can be reduced, the man-hours required for creating the test scenario can be reduced.

Next, in step S20, the user terminal 200 opens, for example, the system screen provided by the test automation server 100, and specifies a test scenario. The user terminal 200 transmits, for example, information indicating a designated test scenario (hereinafter referred to as scenario information) to the test automation server 100.

Next, in step S21, the test automation server 100 tests, for example, the system screen to be tested (hereinafter referred to as a second system screen) according to the test scenario. Here, the second system screen is, for example, a web page on the same website in which a specific element of the first system screen is changed.

Next, in step S22, in the test automation server 100, for example, among the elements of the second system screen, the element of the first system screen (hereinafter referred to as the first element) is changed (hereinafter referred to as the second element). ) Is determined. The test automation server 100 generates, for example, a screen that identifiable the second element. The test automation server 100 transmits, for example, information about the generated screen to the user terminal 200. Here, the "information about the screen" is, for example, a code for displaying an image on a browser.

Next, in step S23, the user terminal 200 displays, for example, a screen that identifiable the second element on the display unit 205a based on the information about the screen. For example, when the user determines that the second element is an element corresponding to the first element, the user terminal 200 includes a procedure including the first element in the test scenario and a procedure including the second element by the operation input of the user. A request for correction (hereinafter referred to as a correction request) is sent to the test automation server 100.

Next, in step S24, the test automation server 100 modifies the procedure including the first element in the test scenario to the procedure including the second element, for example, based on the modification request. The test automation server 100 executes the process as, for example, the second system screen when the test scenario is modified based on the modification request as the first system screen when the next test scenario is executed.

In this way, the test automation system 10 can identify the changed element among the elements constituting the system screen by a method that can be easily discriminated by the user. Further, since the test automation system 10 can reflect the procedure including the changed element in the test scenario without inputting the code, the man-hours required for testing the system screen can be reduced.

Hereinafter, the functional configuration of each device constituting the test automation system 10 will be described.

First, the functional configuration of the test automation server 100 will be described with reference to FIG. FIG. 3 is a diagram showing an example of the functional configuration of the test automation server 100 according to the present embodiment. The test automation server 100 tests the second system screen according to, for example, a test scenario. Then, the test automation server 100 presents the test result to the user, for example, and automatically corrects the test scenario based on the user's operation input. The "test result" is, for example, information indicating whether or not the test was successful as a result of executing the test scenario.

The test automation server 100 is, for example, a storage unit 110, an acquisition unit 120, a scenario creation unit 130, a test execution unit 140, a discrimination unit 150, a scenario correction unit 160, a display processing unit 170, and a transmission unit 180. Including the functional part of.

The storage unit 110 includes, for example, scenario information D111, feature information D112, and test result information D113.

FIG. 4 is a diagram showing an example of scenario information D111. As shown in FIG. 4, the scenario information D111 may include, for example, the items of [user ID], [scenario ID], and [scenario content]. [User ID] is, for example, an item including a user ID which is a user identification code. The user ID is, for example, a code assigned to each user and is assigned to a user newly registered in the test automation server 100. The [scenario ID] is, for example, an item associated with the item of the [user ID] and including the scenario ID which is the identification code of the test scenario registered by the user. The scenario ID is assigned to the created test scenario, for example, when the scenario creation process is executed by the test automation server 100. [Scenario content] is, for example, an item associated with the item of [Scenario ID] and including the content of the scenario registered by the user. The test scenario is stored in the scenario information D111, for example, when the scenario creation process is executed by the test automation server 100.

FIG. 5 is a diagram showing an example of the feature information D112. As shown in FIG. 5, the feature information D112 may include, for example, [scenario ID], [step ID], [element ID], [class name], [position], and [text] items. .. Since the [scenario ID] is the same as the [scenario ID] of the scenario information D111, the description thereof will be omitted. The [step ID] is, for example, an item associated with the item of the [scenario ID] and including the step ID which is an identification code of the operation input by the user. The step ID is assigned to the operation input when the user's operation input is received, for example. [Element ID] is, for example, an item associated with the item of [Scenario ID] and including an element ID which is an identification code of an element constituting the first system screen. The element ID is assigned to each element, for example, when the scenario creation process is executed by the test automation server 100. The [class name] is, for example, an item associated with the item of [element ID] and including the class name of the element constituting the first system screen. The class name is specified, for example, when the scenario creation process is executed on the test automation server 100. [Position] is, for example, an item associated with the item of [Element ID] and including the position of the element constituting the first system screen on the first system screen. The location is specified, for example, when the scenario creation process is executed on the test automation server 100. The [text] is, for example, an item associated with the item of the [element ID] and including the text indicated by the element constituting the first system screen. The text is specified, for example, when the scenario creation process is executed on the test automation server 100.

FIG. 6 is a diagram showing an example of test result information D113. As shown in FIG. 6, the test result information D113 includes, for example, items of [user ID], [scenario ID], [date and time], [scenario content], [test result], and [correction history]. Since the items of [User ID], [Scenario ID], and [Scenario content] are the same as [User ID], [Scenario ID], and [Scenario content] of the scenario information D111, the description thereof will be omitted. However, in the item of [scenario content], for example, the content of the test scenario modified by the scenario modification unit 160 described later is input. [Date and time] is, for example, an item associated with the item of [Scenario ID] and including the date and time when the test scenario is executed. The date and time are entered, for example, when the test scenario is executed. [Test result] is, for example, an item associated with the item of [Scenario ID] and including a test result indicating the result of executing the test scenario. The test result is input, for example, as a result of executing the test scenario. The [correction history] is, for example, an item associated with the item of [scenario ID] and including the content corrected based on the operation input of the user as a result of executing the test scenario. The modification history is input, for example, when the test scenario is modified.

The scenario information D111, the feature information D112, and the test result information D113 stored in the storage unit 110 do not have to be stored in the test automation server 100, and are stored in, for example, a device different from the test automation server 100. You may be. In this case, the test automation server 100 may acquire the scenario information D111, the feature information D112, and the test result information D113 from the apparatus.

The acquisition unit 120 acquires various information from other devices such as the user terminal 200.

The scenario creation unit 130 creates a test scenario for testing the function of the system screen, for example, based on the operation information related to the operation input by the user. An example of a test scenario created for the first system screen will be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of the first system screen. As shown in FIG. 7, the first system screen is, for example, an inquiry form including elements E10 to E17. The elements E10 to E14 are formed by, for example, a text box, and text is input by accepting a user's operation input. The element E15 is, for example, a button for registering the inquiry contents by the information input to the elements E10 to E14. The element E16 is, for example, an element provided in the footer of the first system screen and linked to, for example, a predetermined URL. The element E17 is a button for starting the creation of the test scenario when the operation input by the user is received. That is, for example, when the scenario creation unit 130 receives an operation input by the user for the element E17, the text input to the element and the operation input by the user for the element after that are stored together with the order. Store in unit 110. Hereinafter, it is assumed that the operation input by the user is executed in the order of the elements E10 to E16 on the first system screen, and the test scenario is created in that order.

The test execution unit 140 tests the second system screen according to the test scenario generated based on the first system screen, for example. As a result of testing the second system screen, for example, the test execution unit 140 specifies the degree of agreement regarding the operation input for the elements constituting the second system screen. In other words, the test execution unit 140 calculates, for example, the degree of agreement regarding the procedure of the test schedule including the elements constituting the second system screen. Here, the degree of coincidence is, for example, an index indicating the validity of the behavior regarding the element (or procedure) when there is an operation input for the element constituting the second system screen according to the test scenario. Specifically, the test execution unit 140 is, for example, when an element arranged at a predetermined position on the first system screen is arranged at a position different from the predetermined position on the second system screen, at that position. Calculate the degree of matching according to the degree of deviation. Further, for example, when a different element is arranged in place of the element arranged at a predetermined position on the first system screen, the test execution unit 140 has a behavior different from the behavior due to the operation input to the element on the first system screen. The degree of agreement may be calculated according to the degree of difference from the behavior due to the operation input to.

The discrimination unit 150 discriminates, for example, the second element in which the first element constituting the first system screen is changed in the second system screen tested according to the test scenario generated based on the first system screen.

First, the outline of the second system screen and the second element will be described with reference to FIG. FIG. 8 is a diagram showing an example of a second system screen including the second element. As shown in FIG. 8, the second system screen is composed of, for example, elements E20 to E26. The required E20 to E25 are the same as those of E10 to E15 on the first system screen. The element E26 is provided in the footer divided into columns of the second system screen, and is linked to a predetermined URL, for example. That is, in the present embodiment, the second system screen is, for example, a system screen in which the footer of the first system screen is modified. In the following, a case where the test is executed on the second system screen in which the position of the footer “career” (E26) is different from that in the first system screen in the discrimination unit 150 will be described.

Next, an example of a process (hereinafter referred to as a discrimination process) in which the second element of the second system screen is discriminated by the discrimination unit 150 will be described. The determination unit 150 identifies, for example, a procedure of a test scenario showing a degree of agreement that does not satisfy a predetermined threshold value. The discriminating unit 150 identifies, for example, the first element of the first system screen included in the identified procedure. The discriminating unit 150, for example, refers to the feature information D112 and extracts a feature related to the first element (hereinafter, referred to as a first element feature). The discriminating unit 150, for example, refers to the feature information D112 and extracts features related to the elements of the second system screen. The "feature" may include, for example, at least one of an identification code, a class name, a position on the system screen, and text in the elements constituting the system screen. The discriminating unit 150 discriminates the second element of the second system screen corresponding to the first element, for example, based on the first element feature. Specifically, the discriminating unit 150 is, for example, a feature (hereinafter referred to as a second element feature) that satisfies a predetermined condition (hereinafter referred to as a first condition) with respect to the first element feature among the elements of the second system screen. ) Is determined as the second element. The "first condition" is, for example, a condition that the index indicating identity with respect to the first element feature is the highest. The discrimination unit 150 may be configured by, for example, a neural network. The discrimination unit 150 generates information (hereinafter, referred to as discrimination result information) indicating the result of discriminating the second element. The discrimination unit 150 stores, for example, the discrimination result information in the test result information D113.

The scenario correction unit 160 is, for example, a first test scenario based on information regarding a user's operation input (hereinafter, referred to as operation input information) indicating that the determined second element is an element corresponding to the first element. Modify the procedure including the element to the procedure including the second element. That is, for example, the scenario correction unit 160 rewrites the procedure including the first element to the procedure including the second element with respect to the content of the item of [scenario content] in the scenario information D111. Further, the scenario correction unit 160 stores, for example, the rewritten test scenario in the test result information D113. This eliminates the need for the user to manually rewrite the test scenario, so that the test automation system 10 can enhance the convenience of the user.

The display processing unit 170 generates, for example, various screens. The display processing unit 170 executes, for example, a process for displaying the generated screen on the display unit 205a of the user terminal 200 (hereinafter, referred to as a display process). The display processing unit 170 includes, for example, a scenario screen T1 in which an image including an element of each procedure of a test scenario is arranged according to the procedure, an image including an element showing a test result, and a text indicating an operation input related to the image. A test result screen T2 and an element comparison screen T3 in which the first element and the second element are displayed in a comparable manner are generated.

FIG. 9 is a diagram showing an example of the scenario screen T1. The display processing unit 170 generates the scenario screen T1 with reference to, for example, the scenario information D111. As shown in FIG. 9, the scenario screen T1 includes, for example, a scenario element image T11 corresponding to each procedure and a scenario text data T12 showing test contents corresponding to the scenario element image T11. In the test automation system 10, by displaying the scenario screen T1 on the display unit 205a of the user terminal 200, the user can easily visually confirm the test scenario, so that the convenience of the user can be improved.

The scenario element image T11 is, for example, an image including the first element for each procedure obtained by extracting a part of the first system screen. Specifically, for example, in the scenario element image T11b, the element E10 (first element) in which "Yamada" is input is included. By displaying the scenario element image T11, the user can visually confirm the procedure of the scenario, so that the convenience of the user can be improved.

The scenario text data T12 is, for example, data for displaying the user's operation input for the first element in text. For example, when the user's operation input is an operation input for inputting text in the first element, scenario text data T12 indicating that the text has been input is generated (for example, scenario text data T12b). Further, for example, when the operation input of the user is an operation input for selecting the text displayed in the first element by pull-down or the like, the scenario text data T12 indicating that the text is selected is generated. Further, when the operation input of the user is an operation input for clicking the first element, scenario text data T12 indicating that the first element is clicked is generated (for example, scenario text data T12g). The scenario text data T12 is arranged so as to correspond to, for example, the scenario element image T11. Specifically, for example, the scenario text data T12b is arranged so that the text "Enter Yamada" is attached to the lower side of the scenario element image T11b on the paper. By displaying the scenario text data T12, the user can visually confirm the procedure of the test scenario, so that the convenience of the user can be improved.

FIG. 10 is a diagram showing an example of the test result screen T2. The display processing unit 170 generates the test result screen T2 with reference to, for example, the test result information D113. As shown in FIG. 10, the test result screen T2 is, for example, a test element image T21 corresponding to each procedure of the test scenario of the executed test and a test showing the contents executed in the test corresponding to the test element image T12. The text data T22 and the confirmation request content T23 for identifiablely displaying the test element image T21 including the second element determined by the determination unit 150 to be changed are included. In the test automation system 10, by displaying the test result screen T2 on the display unit 205a of the user terminal 200, the user can easily visually confirm the test result for the second system screen, so that the convenience of the user can be improved. ..

The test element screen T21 is, for example, a screen including a second element for each procedure obtained by extracting a part of the second system screen. Specifically, for example, in the test element image T21b, the element E20 (second element) in which "Yamada" is input is included. By generating the test element image T21, the user can visually confirm the result of the test, so that the convenience of the user can be improved.

The test text data T22 is, for example, data in which the operation input of the test program for the second element is displayed in text. For example, when the operation input of the test program is an operation input for inputting text in the second element, test text data T22 indicating that the text has been input is generated (for example, text data T22b). Similarly, the test text data T22 is generated for the operation input for clicking the second element (for example, the test text data T22g), the operation input selected by the pull-down, and the like. The test text data T22 is arranged so as to correspond to, for example, the test element image T21. Specifically, for example, in the text data T22b, the text of "input Yamada" is arranged so as to be attached to the scenario element image T11b. By displaying the test text data T22, the user can visually confirm the test result according to the procedure of the test scenario, so that the convenience of the user can be improved.

The confirmation request content T23 is, for example, a test including a first object T23a including a text indicating that the second element determined by the discrimination unit 150 does not match the first element, and a second element determined by the discrimination unit 150. It includes a second object T23b that highlights the element image T21. The first object T23a is composed of a text prompting the user to confirm, for example, "confirmation required". The second object T23b may be, for example, an object that surrounds the test element image T21 including the second element determined by the discrimination unit 150 with a predetermined color. By displaying the confirmation request content T23, the user can visually easily recognize the procedure including the second element determined by the determination unit 150, so that the convenience of the user can be improved.

FIG. 11 is a diagram showing an example of the element comparison screen T3. The display processing unit 170 generates an element comparison screen T3 with reference to, for example, the test result information D113. As shown in FIG. 11, the element comparison screen T3 includes, for example, a captured image T31 showing past test results, a captured image T32 showing the current test results, a first button T33, and a second button T34. .. The element comparison screen T3 may include a first system screen instead of the captured image T31 showing the past test results, for example.

The captured image T31 is, for example, an image showing a successful system screen as a result of being tested in the past according to a test scenario. Further, the captured image T31 may be, for example, an image showing a successful system screen as a result of retesting after modifying the test scenario in the past. The captured image T31 may be generated, for example, by attaching an object having a predetermined color (for example, the object T31a) to the element corresponding to the second element determined by the determination unit 150 in this test. .. The "past" is, for example, the latest successful test.

The captured image T32 is, for example, an image showing a state in which an operation input according to a test scenario is executed for each element of the second system screen in this test. The captured image T32 may be generated, for example, by attaching an object having a predetermined color (for example, the object T32a) to the second element determined by the determination unit 150 in this test.

In this way, by displaying the captured image T31 and the captured image T32 on the display unit 205a of the user terminal 200 so as to be comparable, the user visually easily recognizes the second element determined by the discriminating unit 150. Therefore, the convenience of the user can be improved.

The first button T33 is, for example, a button for modifying a procedure including the first element in a test scenario to a procedure including the second element. In FIG. 11, the first button T33 is displayed with the text "successful". Specifically, for example, when the user terminal 200 receives an operation input by the user for the first button T33, the user terminal 200 performs a process of modifying the procedure including the first element in the test scenario to the procedure including the second element. The requested modification request is sent to the test automation server 100. As a result, the user can modify the test scenario simply by inputting an operation to the first button T33, which can improve the convenience of the user.

The second button T34 is, for example, a button for modifying a procedure including the first element in a test scenario to a procedure different from the procedure including the second element. In other words, the second button T34 is, for example, a button that does not cause the procedure including the first element in the test scenario to be modified by the procedure including the second element. In FIG. 11, the second button T34 is displayed with the text "Failure". Specifically, for example, when the user terminal 200 receives an operation input by the user for the second button T34, the user terminal 200 requests a process of replacing the procedure including the first element in the test scenario with another procedure. The request is sent to the test automation server 100.

The transmission unit 180 transmits, for example, information about various image screens to the user terminal 200.

Next, the functional configuration of the user terminal 200 will be described with reference to FIG. FIG. 12 is a diagram showing an example of the functional configuration of the user terminal 200 according to the present embodiment. The user terminal 200 causes the display unit 205a to display various screens based on the information acquired from the test automation server 100, for example. The user terminal 200 includes, for example, a storage unit 210, an acquisition unit 220, a display processing unit 230, and a functional unit of a transmission unit 240. The storage unit 210 stores, for example, various types of information. The acquisition unit 220 acquires, for example, information about various screens from the test automation server 100. The display processing unit 230 causes the display unit 205a to display, for example, information on various acquired screens. The transmission unit 240 transmits, for example, various information to the test automation server 100.

=== Detailed procedure ===
Next, the procedure of information processing according to the present embodiment will be described with reference to FIG. 13 as appropriate. FIG. 13 is a flowchart showing a procedure of information processing of the test automation system 10 according to the present embodiment. Here, in FIG. 13, as an example, not limited to, an information processing procedure for testing the second system screen shown in FIG. 8 according to the test scenario after the test scenario is created by the first system screen shown in FIG. Will be explained.

First, in step S101, when the user terminal 200 receives a user's operation input for a button displayed on the display unit 205a for requesting test execution, the test automation server 100 tests the second system screen. Generate an execution request to execute. The user terminal 200 sends, for example, an execution request to the test automation server 100.

Next, in step S102, the test automation server 100 specifies a test scenario based on, for example, an execution request.

Next, in step S103, the test automation server 100 executes a test on the second system screen according to a test scenario, for example.

Next, in step S104, the test automation server 100 executes the above-mentioned determination process. For example, in the discrimination process, the test automation server 100 discriminates the second element of the second system screen changed from the first element of the first system screen.

Next, in step S105, the test automation server 100 generates information about the test result screen T2 with reference to, for example, the test result information D113. The test automation server 100 may acquire information about the test result screen T2 from a database (not shown) that stores information about the test result screen T2 in advance, for example. The test automation server 100 transmits, for example, information about the test result screen T2 to the user terminal 200.

Next, in step S106, the user terminal 200 causes the display unit 205a to display the test result screen T2. For example, when the user terminal 200 receives an operation input by the user for the test element image T21 to which the confirmation request content T23 is attached, the user terminal 200 requests a confirmation request for an element comparison screen T3 for confirming the second element. Is sent to the test automation server 100.

Next, in step S107, the test automation server 100 generates information about the element comparison screen T3 based on the confirmation request. The test automation server 100 may acquire information about the element comparison screen T3 from a database (not shown) that stores the element comparison screen T3 in advance, for example. The test automation server 100 transmits, for example, information about the element comparison screen T3 to the user terminal 200.

Next, in step S108, the user terminal 200 causes the display unit 205a to display the element comparison screen T3. For example, when the user terminal 200 receives an operation input by the user for the first button T33 of the element comparison screen T3, the user terminal 200 performs a process of modifying the procedure including the first element in the test scenario to the procedure including the second element. The requested modification request is sent to the test automation server 100.

Next, in step S109, the test automation server 100 corrects the contents of the [scenario content] item in the scenario information D111 based on the correction request. As a result, when the second system screen is tested again in the same test scenario, the confirmation request content T23 is not displayed. That is, the test of the second system screen is completed normally.

In this way, if the elements that make up the system screen when the test scenario is created are changed on the system screen when the test is executed, the user identifies the cause of the test error and the user manually tests the test scenario. Since the work of modifying the system screen can be reduced, the test man-hours related to the system screen can be significantly reduced.

<< First variant >>
A first modification will be described with reference to FIG. FIG. 14 is a diagram showing an example of a comparison selection screen T4 that selectively displays a plurality of elements determined by the discrimination unit 150.

The test automation server 100 according to the first modification determines, for example, a plurality of elements presumed to have been changed from the first element in the determination unit 150. The test automation server 100 generates, for example, a comparison selection screen T4 including a plurality of determined elements. Hereinafter, the discrimination process in the discrimination unit 150 will be described first, and then the comparison selection screen T4 will be described. Hereinafter, since the discrimination process of the first modification is the same as the process of extracting the "feature" in the discrimination process of the present embodiment, the description thereof will be omitted.

The discrimination unit 150 discriminates a plurality of elements of the second system screen presumed to correspond to the first element, for example, based on the first element feature. Specifically, the discriminating unit 150 discriminates, for example, a second element having a second element feature satisfying the first condition with respect to the first element feature among the elements of the second system screen. Further, the discriminating unit 150 discriminates, for example, an element (hereinafter referred to as a third element) having a feature satisfying the second condition (hereinafter referred to as a third element feature) among the elements of the second system screen. The "second condition" is, for example, a condition that the index showing the identity with respect to the first element feature is higher than the index shown by the second element feature. The discrimination unit 150 generates discrimination result information indicating the result of discriminating the second element and the third element. The discrimination unit 150 stores, for example, the discrimination result information in the test result information D113. In the above, the discrimination unit 150 has been described so as to discriminate between the two elements of the second element and the third element, but the present invention is not limited to this. The discriminating unit 150 may discriminate three or more elements.

The display processing unit 170 generates, for example, the comparison selection screen T4 with reference to the test result information D113. Hereinafter, the comparison selection screen T4 will be described with reference to FIG. FIG. 14 is a diagram showing an example of the comparison selection screen T4 capable of comparing the past test result and the current test result. As shown in FIG. 14, the comparison selection screen T4 has, for example, a captured image T41 showing the past test results, captured images T42 and T43 showing the current test results, a first button T44, and a second button T45. including. Since the captured image T41, the first button T44, and the second button T45 are the same as the captured image T31, the first button T33, and the second button T34 shown in FIG. 11, the description thereof will be omitted.

The captured image T42 includes, for example, a highlighting object T42a for highlighting the second element (here, “career”) determined in this test. The emphasis object T42a may be, for example, an object that surrounds the second element with a predetermined color. The captured image T43 includes, for example, a highlighting object T43a for highlighting the third element (here, “about us”) determined in this test. The emphasis object T43a may be, for example, an object that surrounds the third element with a predetermined color. As a result, the user can confirm a plurality of candidate elements corresponding to the first element, so that the convenience of the user can be improved. Note that, in FIG. 14, the captured images T42 and T43 are shown to display the entire system screen, but the present invention is not limited to this. For example, in the comparison selection image T4, a part of the image of the system screen including the second element and a part of the image of the system screen including the third element may be displayed. Further, for example, in the comparative selection image T4, the captured image T42 including the second element and the image including the element constituting the second system screen are displayed in descending order of the index indicating the identity of the element with respect to the first element feature. You may let me. Specifically, as shown in FIG. 15, in the comparative selection image T4, the captured image T42 in which the second element (here, “career”) is highlighted and the captured image T42 near the captured image T42, on the left side of the paper. The images T431 and T432 may be displayed in order from the first element, which includes elements having a high index of identity with respect to the feature (here, “about us” and “Company” in order from the left). Further, for example, in the comparative selection image T4, the captured image T42 includes the emphasis object T42b for emphasizing the second element, and the emphasis object T42c for emphasizing the element having a lower index of identity than the second element. May include. The emphasis object T42c is set to have a lower degree of emphasis than, for example, the emphasis object T42b. Specifically, as shown in FIG. 16, in the captured image T42 of the comparative selection image T4, for example, an element in which the second element is highlighted and the index indicating identity is lower than that of the second element is It may be highlighted with a darker highlight than the highlight of the second element. Further, for example, an element having a higher index indicating identity may be highlighted by a highlighted object having a higher degree of highlighting. Specifically, in the elements constituting the captured image T42, the index indicating the identity may be highlighted with bright highlights in descending order. Further, the elements constituting the captured image T42 may be assigned a reference numeral (number or the like) indicating the height of the index indicating the identity.

Next, the procedure of information processing according to the first modification will be described. In the information processing flowchart according to the first modification, the element comparison screen T3 is replaced with the selection comparison screen T4, and the following step S108A is added between S107 and S108 with respect to the flowchart shown in FIG. It is a thing.

In step S108A, the user terminal 200 causes, for example, the display unit 205a to display the comparison selection screen T4. For example, when the user terminal 200 receives an operation input by the user for either the captured image T42 or the second captured image T43 of the comparison selection screen T4 shown in FIG. 14, the image that has received the operation input is selected. Recognize that. In the comparison selection screen T4 shown in FIG. 15, when the user terminal 200 accepts an operation input by the user for any of the captured image T42, the captured image T431, and the captured image T432, for example, the user terminal 200 receives the operation input. Recognize that the accepted image has been selected. Further, in the comparison selection screen T4 shown in FIG. 16, when the user terminal 200 accepts an operation input by the user for the element highlighted by highlighting, for example, the element that has received the operation input is selected. Recognize that it was done. In step S108, for example, when the user terminal 200 receives an operation input by the user for the first button T44, the procedure including the first element in the test scenario is highlighted in the selected image (elements). A modification request for modifying the procedure including the second element or the third element) is sent to the test automation server 100.

In this way, by presenting a plurality of elements presumed to correspond to the first element to the user and letting the user select them, the selected result is fed back to the test automation system 10, and the test automation system 10 The estimation accuracy of can be improved.

<< Second variant >>
The test automation server 100 according to the second modification confirms, for example, whether or not the determined element is correct to the user each time the determination unit 150 determines the changed element from the first element. .. Specifically, the discriminating unit 150 determines whether or not the specified degree of agreement satisfies a predetermined threshold value for each procedure of the test scenario, for example. For example, when it is determined that the degree of agreement of a predetermined procedure does not satisfy a predetermined threshold value, the discrimination unit 150 discriminates the second element by a discrimination process. For example, each time the discrimination unit 150 determines the second element, the display processing unit 170 causes the display unit 205a to display a test result screen T2 showing the determination results up to the procedure determined by the discrimination unit 150, for example. As a result, the user can confirm the second element in the middle of the test scenario and modify the test scenario, so that the convenience of the user can be improved.

<< Third variant >>
When the test automation server 100 according to the third modification receives an operation input by the user for the second button T34 of the element comparison screen T3 shown in FIG. 11 in the user terminal 200, for example, the first element is used. Do not replace the including procedure with the procedure including the second element. In this case, the test automation server 100 causes, for example, the display unit 205a to display the second system screen shown in FIG. Then, the test execution unit 140 automatically executes, for example, the procedure up to the procedure including the second element among the procedures of the test scenario. As a result, the operation procedure related to the user's test can be reduced, so that the user's convenience can be improved.

=== Hardware configuration of test automation system 10 ===
The HW configuration of each device included in the test automation system 10 will be described with reference to FIGS. 17 and 18. FIG. 17 is a diagram showing an example of the hardware configuration of the test automation server 100. FIG. 18 is a diagram showing an example of the hardware configuration of the user terminal 200.

As shown in FIG. 17, the test automation server 100 includes a control unit (CPU) 101, a storage device 102, a communication I / F (interface) 103, an input device 104, and an output device 105. The components of the HW of the test automation server 100 are connected to each other via, for example, the bus B1. It is not essential that the HW of the test automation server 100 includes all the components as the configuration of the HW of the test automation server 100. For example, the HW of the test automation server 100 may or may not have a configuration in which the output device 105 can be removed.

The control unit 101 has a circuit physically structured to execute a function realized by a code or an instruction contained in the program P1, and is realized by, for example, a data processing device built in hardware. To. The control unit 101 is typically a central processing unit (CPU), and may or may not be a microprocessor, a processor core, a multiprocessor, an ASIC, or an FPGA. In the present disclosure, the control unit 101 is not limited to these.

The storage device 102 has a function of storing various programs and various data required for the test automation server 100 to operate. The storage device 102 is realized by various storage media such as HDD, SSD, and flash memory. However, in the present disclosure, the storage device 102 is not limited to these. Further, the storage device 102 may or may not be expressed as a memory.

The communication I / F 103 transmits and receives various data via the network N. The communication by the communication I / F 103 may be executed by either wired or wireless, and any communication protocol may be used as long as the mutual communication can be executed. The communication I / F 103 has a function of executing communication with another device such as a user terminal 200 via the network N. The communication I / F 103 transmits various data to other devices according to instructions from the control unit 101. Further, the communication I / F 103 receives various data transmitted from other devices and transmits them to the control unit 101. Further, the communication I / F 103 may be simply expressed as a communication unit. Further, when the communication I / F 103 is composed of a physically structured circuit, it may be expressed as a communication circuit.

The input device 104 is realized by a device that inputs various operations to the test automation server 100. The input device 104 is realized by any one or a combination of all kinds of devices capable of receiving an input from a user and transmitting information related to the received input to the control unit 101. The input device 104 is typically realized by a hardware key typified by a keyboard or the like, or a pointing device such as a mouse. The input device 104 may or may not include, for example, a touch panel, a camera (operation input via a moving image), and a microphone (operation input by voice). However, in the present disclosure, the input device 104 is not limited to these.

The output device 105 includes, for example, a display unit. The display unit is typically realized by a monitor (for example, a liquid crystal display or OELD). The display unit may or may not be a head-mounted display (HDM) or the like. It should be noted that these display units 13a may or may not be able to display display data in 3D. However, in the present disclosure, the display unit is not limited to these.

The test automation server 100 stores the program P1 and various databases in the storage device 102. In the test automation server 100, the control unit 101 executes this program P1 to execute the processing as each unit included in the control unit 101. That is, the program 15a stored in the storage device 102 causes the test automation server 100 to realize each function executed by the control unit 101. This program P1 may or may not be expressed as a program module.

As shown in FIG. 18, the user terminal 200 includes a control unit (CPU: Central Processing Unit) 201, a storage device 202, a communication I / F (interface) 203, an input device 204, and an output device 205. .. The components of the HW of the user terminal 200 are connected to each other via, for example, the bus B2. It is not essential that the HW configuration of the user terminal 200 includes all the components. For example, the user terminal 200 may or may not have a configuration in which individual components such as an input device 204 and an output device 205 can be removed, or a plurality of components can be removed.

The control unit 201 has a circuit physically structured to execute a function realized by a code or an instruction contained in the program, and is realized by, for example, a data processing device built in hardware. .. Therefore, the control unit 201 may or may not be expressed as a control circuit. The control unit 201 is, for example, a central processing unit (CPU), a microprocessor (microprocessor), a processor core (processor core), a multiprocessor (multiprocessor), an ASIC (Application-Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array). including.

The storage device 202 has a function of storing various programs and various data required for the user terminal 200 to operate. The storage device 202 includes various storage media such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, a RAM (Random Access Memory), and a ROM (Read Only Memory). Further, the storage device 202 may or may not be expressed as a memory.

Communication I / F203 sends and receives various data via network N. Communication by communication I / F 203 may be executed by either wired or wireless, and any communication protocol may be used as long as mutual communication can be executed. The communication I / F 203 has a function of executing communication with the test automation server 100 via the network N. The communication I / F 203 transmits various data to the test automation server 100 according to the instruction from the control unit 201. Further, the communication I / F 203 receives various data transmitted from the test automation server 100 and transmits them to the control unit 201. Further, the communication I / F 203 may be simply expressed as a communication unit. Further, when the communication I / F 203 is composed of a physically structured circuit, it may be expressed as a communication circuit.

The input device 204 includes a device for inputting various operations to the user terminal 200. The output device 205 includes a device that outputs a processing result processed by the user terminal 200. The input device 204 and the output device 205 may or may not be specified as input / output units. When specified as an input / output unit, the input / output unit may be integrated with the input device 204 and the output device 205, or may be separated into the input device 204 and the output device 205.

The input device 204 is realized by any one or a combination of all kinds of devices that can receive the input from the user and transmit the information related to the received input to the control unit 201. The input device 204 includes, for example, an operation unit, a microphone, and a camera. The operation unit includes, for example, a hardware key such as a touch panel, a touch display, and a keyboard, and a pointing device such as a mouse. The microphone includes, for example, a device for inputting voice. The microphone is used for inputting voice data. The camera includes, for example, a device for inputting an image. The camera is used to acquire image data.

The output device 205 is realized by any one or a combination of all kinds of devices capable of outputting the processing result processed by the control unit 201. The output device 205 includes, for example, a display unit 205a and a speaker (audio output) 205b. The output device 205 further includes, for example, a touch panel, a touch display, a lens (for example, 3D (three dimensions) output, a hologram output), a printer, and the like.

The display unit 205a is realized by any or a combination of all kinds of devices capable of displaying a screen according to the display data written in the frame buffer. The display unit 205a may be, for example, a touch panel, a touch display, a monitor (for example, a liquid crystal display or an OLELD (Organic Electroluminescence Display)), a head mounted display (HDM: Head Mounted Display), a projection mapping, a hologram, in the air, or the like (vacuum). It may or may not be) include devices capable of displaying screens, text information, etc. Note that these display units 205a may or may not be capable of displaying display data in 3D. It is not necessary. The speaker 205b is used for output of audio data.

The user terminal 200 stores the program P2 and the database in the storage device 202. In the user terminal 200, the control unit 201 executes the program P2 to execute the processing as each unit included in the control unit 201. That is, the program P2 stored in the storage device 202 causes the user terminal 200 to realize each function executed by the control unit 201. Further, this program P2 may or may not be expressed as a program module.

Each embodiment of the present disclosure will be described as being realized by the CPU of the user terminal 200 and / or the test automation server 100 executing the program.

Further, the program of each embodiment of the present disclosure (for example, a software program, a computer program, or a program module) may or may not be provided stored in a computer-readable storage medium. .. The storage medium can store the program of each embodiment in a "non-temporary tangible medium". Further, the program of each embodiment may or may not be for realizing a part of the function of each embodiment of the present disclosure. Further, it may or may not be a so-called difference file (difference program) that can realize the functions of each embodiment of the present disclosure in combination with a program already recorded in the storage medium.

The test automation server 100 and / or the user terminal 200 can read the program stored in the storage medium and execute the read program to realize the functions of the plurality of functional units shown in each embodiment.

Further, the program of the present disclosure may or may not be provided to the test automation server 100 and / or the user terminal 200 via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. May be good. The test automation server 100 and / or the user terminal 200 realizes the functions of the plurality of functional units shown in each embodiment by executing the program P downloaded via the Internet or the like, for example.

In addition, each embodiment of the present disclosure can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission. At least a portion of the processing in the test automation server 100 and / or the user terminal 200 may or may not be implemented by one or more computers (eg, cloud computing composed of one or more computers). May be good.

At least a part of the processing in the user terminal 200 may or may not be performed by the test automation server 100. In this case, at least a part of the processing of each functional unit of the control unit 201 of the user terminal 200 may or may not be performed by the test automation server 100.

At least a part of the processing in the test automation server 100 may or may not be performed by the user terminal 200. In this case, at least a part of the processing of each functional unit of the control unit 101 of the test automation server 100 is performed by the user terminal 200.

=== Summary ===
The test automation system 10 according to the present embodiment has a first system screen (first user interface) according to a test scenario (scenario) generated based on the first system screen (first user interface displayed on the display unit). ), When testing the second system screen (second user interface displayed on the display unit), the first system screen (first user interface) is displayed on the second system screen (second user interface). The discriminant unit 150 that discriminates the second element changed from the constituent first element, the display processing unit 170 that displays the information indicating the discriminant result in the discriminant unit 150 on the display unit 205a, and the second element regarding the discriminant result. The first element in the scenario is the first element based on the acquisition unit 120 that acquires information on the user's operation input indicating that the element corresponds to the first element and the information on the user's operation input acquired by the acquisition unit 120. It includes a scenario correction unit 160 that corrects to two elements. This makes it possible to identify the changed elements among the elements that make up the system screen by a method that can be easily identified by the user, and the procedure including the changed elements can be reflected in the test scenario without entering code. , The man-hours required for testing the system screen can be reduced.

Further, the display processing unit 170 of the test automation system 10 according to the present embodiment compares the captured image T31 (first image) including the first element with the captured image T32 (second image) including the second element. It is possible to display it on the display unit 205a. As a result, the user can visually easily recognize the second element determined by the discrimination unit 150, which can improve the convenience of the user.

Further, the display processing unit 170 of the test automation system 10 according to the present embodiment displays the second element in the captured image T32 (second image) in an identifiable manner. As a result, the user can visually easily recognize the second element determined by the discrimination unit 150, which can improve the convenience of the user.

Further, the display processing unit 170 of the test automation system 10 according to the present embodiment does not match the test element image T21 (image) including the second element determined by the discrimination unit 150 with the second element. Confirmation request content T23 (text) indicating that is displayed. As a result, the user can visually easily recognize the procedure including the second element determined by the determination unit 150, so that the convenience of the user can be improved.

Further, the display processing unit 170 of the test automation system 10 according to the present embodiment causes the display unit 205a to display the first button T33 that accepts the user's operation input, and the acquisition unit 120 displays the user's first button T33 with respect to the first button T33. The scenario correction unit 160 acquires the information indicating that the operation input has been accepted, and corrects the scenario based on the information indicating that the operation input of the user has been accepted. As a result, the user can modify the test scenario simply by inputting an operation to the first button T33, which can improve the convenience of the user.

Further, the determination unit 150 of the test automation system 10 according to the present embodiment is a second element that satisfies the first condition with respect to the feature relating to the first element among the elements constituting the second system screen (second user interface). The element having the characteristic is discriminated as the second element. By using the second element feature of each element as the basis for discrimination, the changed element can be appropriately discriminated.

Further, the determination unit 150 of the test automation system 10 according to the present embodiment is a second element that satisfies the first condition with respect to the feature relating to the first element among the elements constituting the second system screen (second user interface). The display processing unit 170 discriminates between an element having a feature and an element having a third element feature satisfying the second condition with respect to the first element feature relating to the first element, and the display processing unit 170 determines the second element feature satisfying the first condition. The captured image T42 (third image) including the element having the second element and the captured image T43 (fourth image) including the element having the third element feature satisfying the second condition are displayed on the display unit 205a. As a result, the user can easily confirm a plurality of candidate elements corresponding to the first element, so that the convenience of the user can be improved.

Further, the acquisition unit 120 of the test automation system 10 according to the present embodiment has accepted the user's operation input for either the captured image T42 (third image) or the captured image T43 (fourth image). After acquiring the information to be shown, the discriminating unit 150 obtains the information indicating that the user's operation input has been accepted for either the captured image T42 (third image) or the captured image T43 (fourth image), based on the information. The element included in the image that has received the user's operation input is determined as the second element. By presenting a plurality of elements presumed to correspond to the first element to the user and letting the user select them, the selected result is fed back to the test automation system 10 to improve the estimation accuracy of the test automation system 10. Can be improved.

Further, the determination unit 150 of the test automation system 10 according to the present embodiment has a feature that satisfies the first condition with respect to the feature relating to the first element among the elements constituting the second system screen (second user interface). The element and the element having the feature satisfying the second condition with respect to the feature relating to the first element are discriminated, and the display processing unit 170 determines the element having the feature satisfying the first condition and the feature satisfying the second condition. The captured image T42 (fifth image) including the element to be identified is displayed. As a result, the user can easily confirm a plurality of candidate elements corresponding to the first element, so that the convenience of the user can be improved.

Further, the acquisition unit 120 of the test automation system 10 according to the present embodiment acquires information indicating that the user's operation input has been accepted for the captured image T42 (fifth image), and the determination unit 150 captures. Based on the information indicating that the user's operation input has been accepted for the image T42 (fifth image), the element that has received the user's operation input is determined as the second element. By presenting a plurality of elements presumed to correspond to the first element to the user and letting the user select them, the selected result is fed back to the test automation system 10 to improve the estimation accuracy of the test automation system 10. Can be improved.

Further, the "feature" in the test automation system 10 according to the present embodiment includes at least one of the identification code, the class name, the position, and the text in the elements constituting the system screen (user interface). As a result, the elements constituting the system screen can be appropriately determined.

Further, the display processing unit 170 of the test automation system 10 according to the present embodiment causes the display unit 205a to display information indicating the discrimination result each time the second element is discriminated by the discriminating unit 150. As a result, the user can confirm the second element in the middle of the test scenario and modify the test scenario, so that the convenience of the user can be improved.

Further, the display processing unit 170 of the test automation system 10 according to the present embodiment causes the display unit 205a to display the second button T34 that accepts the user's operation input, and the acquisition unit 120 displays the user's second button T34 with respect to the second button T34. The scenario correction unit 160 does not modify the test scenario based on the information indicating that the operation input of the user has been accepted for the second button T34 after acquiring the information indicating that the operation input has been accepted. As a result, it is possible to confirm with the user whether or not to modify the test scenario, and it is possible to appropriately modify the test scenario.

Further, the test automation system 10 according to the present embodiment further includes a test execution unit 140 that executes a test on the second system screen, and the test execution unit 140 is a user in the acquisition unit 120 with respect to the second button T34. When the information indicating that the operation input of is accepted is acquired, the procedure up to the procedure including the second element among the procedures of the test scenario is executed. As a result, the operation procedure related to the user's test can be reduced, so that the user's convenience can be improved.

B1, B2, B3 ... Bus, N ... Network, 10 ... Test automation system, 100 ... Test automation server, 200 ... User terminal, 101, 201 ... Control unit, 102, 202 ... Storage device.

Claims

The second system when testing the second system screen displayed on the display, modified from the first system screen, according to a scenario generated based on the first system screen displayed on the display. A discriminating unit that discriminates the second element changed from the first element constituting the first system screen on the screen, and
A display processing unit that displays information indicating the discrimination result in the discrimination unit on the display unit,
With respect to the determination result, an acquisition unit for acquiring information regarding a user's operation input indicating that the second element is an element corresponding to the first element, and
A scenario correction unit that modifies the first element in the scenario to the second element based on the information regarding the operation input of the user acquired by the acquisition unit.
Information processing system equipped with.
The display processing unit causes the display unit to display the first image including the first element and the second image including the second element in a comparable manner.
The information processing system according to claim 1.
The display processing unit displays the second element in the second image so that it can be identified.
The information processing system according to claim 2.
The display processing unit displays an image including the second element determined by the discrimination unit and a text indicating that the second element does not match the first element.
The information processing system according to claim 2 or 3.
The display processing unit causes the display unit to display a first button for receiving an operation input of the user.
The acquisition unit acquires information indicating that the user's operation input has been accepted for the first button, and obtains information.
The scenario correction unit corrects the scenario based on the information indicating that the operation input of the user has been accepted.
The information processing system according to any one of claims 2 to 4.
Among the elements constituting the second system screen, the discriminating unit discriminates an element having a feature satisfying the first condition with respect to the feature related to the first element as the second element.
The information processing system according to any one of claims 1 to 5.
Among the elements constituting the second system screen, the discriminating unit sets an element having a feature that satisfies the first condition with respect to the feature relating to the first element and a second condition for the feature relating to the first element. Distinguish between elements that have the characteristics to be satisfied,
The display processing unit causes the display unit to display a third image including an element having a feature satisfying the first condition and a fourth image including an element having a feature satisfying the second condition.
The information processing system according to any one of claims 1 to 5.
The acquisition unit acquires information indicating that the operation input of the user has been accepted for either the third image or the fourth image.
The discriminating unit is an element included in an image that has received the user's operation input based on information indicating that the user's operation input has been accepted for either the third image or the fourth image. Is determined as the second element.
The information processing system according to claim 7.
Among the elements constituting the second system screen, the discriminating unit sets an element having a feature that satisfies the first condition with respect to the feature relating to the first element and a second condition for the feature relating to the first element. Distinguish between elements that have the characteristics to be satisfied,
The display processing unit displays a fifth image that identifiablely includes an element having a feature satisfying the first condition and an element having a feature satisfying the second condition.
The information processing system according to any one of claims 1 to 5.
The acquisition unit acquires information indicating that the operation input of the user has been received for the fifth image, and obtains information indicating that the operation input of the user has been accepted.
The discriminating unit discriminates the element that has received the user's operation input as the second element based on the information indicating that the user's operation input has been received for the fifth image.
The information processing system according to claim 9.
The feature includes at least one of an identification code, a class name, a position, and a text in the elements constituting the system screen.
The information processing system according to any one of claims 6 to 10.
The display processing unit causes the display unit to display information indicating the discrimination result each time the second element is discriminated by the discrimination unit.
The information processing system according to any one of claims 1 to 11.
The display processing unit causes the display unit to display a second button for receiving an operation input of the user.
The acquisition unit acquires information indicating that the user's operation input has been accepted for the second button, and obtains information.
The scenario correction unit does not correct the scenario based on the information indicating that the operation input of the user has been accepted for the second button.
The information processing system according to any one of claims 1 to 12.
Further, a test execution unit for executing the test on the second system screen is provided.
The test execution unit uses the second element of the procedure of the scenario when the acquisition unit acquires information indicating that the user's operation input has been accepted for the second button. Perform the steps up to the including steps,
The information processing system according to claim 13.
The computer
The second system when testing the second system screen displayed on the display, modified from the first system screen, according to a scenario generated based on the first system screen displayed on the display. Distinguishing the second element changed from the first element constituting the first system screen on the screen,
Information indicating the discrimination result for which the second element is discriminated is displayed on the display unit, and
Regarding the determination result, acquisition of information regarding a user's operation input indicating that the second element is an element corresponding to the first element, and
Modifying the first element in the scenario to the second element based on the acquired information regarding the operation input of the user.
Information processing method to execute.
On the computer
The second system when testing the second system screen displayed on the display, modified from the first system screen, according to a scenario generated based on the first system screen displayed on the display. Distinguishing the second element changed from the first element constituting the first system screen on the screen,
Information indicating the discrimination result for which the second element is discriminated is displayed on the display unit, and
Regarding the determination result, acquisition of information regarding a user's operation input indicating that the second element is an element corresponding to the first element, and
Modifying the first element in the scenario to the second element based on the acquired information regarding the operation input of the user.
A program to execute.