JP6723976B2 - Test execution device and program - Google Patents

Description

The present invention relates to a test execution device and a program.

In recent years, in order to provide services that meet needs quickly, the development style of providing services in a short cycle is increasing, and the frequency of software testing is increasing accordingly. Under such circumstances, it is a problem that the screen transition test in the function test is activated in the Web application. According to JSTQB (Non-Patent Document 1), a functional test is defined as "a test performed based on an analysis of a functional specification of a component (a minimum unit of software that can be independently tested) or a system (a set of components)." There are tests that check the functionality of the level to tests that check the function of the system level.

The screen transition test in the functional test is, among the functional tests, "a test for the function that allows you to confirm whether the screen is displayed on the client side according to the interaction from the client side, according to the specifications". Define here.

The reason why the screen transition test in the functional test is activated is that human interaction such as clicking a link or button on the screen or inputting an appropriate value in the form is required as the interaction from the client side. .. As a method of automating such a screen transition test and aiming to reduce the operation, a method of scripting a test case and automatically executing it is widely known.

Since it takes time to manually script a test case, the following two test script automatic generation technologies exist.
・Automatic generation of test script using Capture and Replay technology (Non-Patent Document 2)
・Automatic generation of test script by model-based test (Non-Patent Document 3)

"JSTQB", [online], Internet <URL: http://jstqb.jp/syllabus.html> "Selenium IDE", [online], Internet <URL: https://addons.mozilla.org/en/firefox/addon/selenium-ide/> "Proposal and evaluation of automatic test item generation method in combination test", Harumon Tanno, Akiaki Zhang, Takashi Hoshino, IEICE technical report, Oct 2010

With the Capture and Replay technology, a user can manually execute a test case created from the specifications to record the operation on the user's screen and output it as a test script. However, when the Capture and Replay technology is used, additional operations such as tool operations for recording screen operations occur, so in general, if the equivalent tests are not performed three times or more, the operation cannot be recovered. There is a drawback.

In model-based testing, by writing specifications in a machine-readable model, test cases can be automatically generated from the read specifications and output as test scripts. However, it has the disadvantage of requiring additional work to describe the model.

From the above, it can be said that the existing test script automatic generation technology has a problem that additional operation for generating the test script occurs.

The present invention has been made in view of the above points, and an object of the present invention is to improve the efficiency of execution of a test related to a Web application.

Therefore, in order to solve the above-mentioned problem, the test execution apparatus extracts the information of the first screen transition defined in the source code from the source code of the Web application, and the screen displayed for the Web application. A first generation unit that generates an input value, an operation unit that inputs the input value generated by the first generation unit into the screen and automatically operates the screen, and is realized by an automatic operation. A second generation unit that generates information on a second screen transition, and each input to the transition source screen of the insufficient screen transition when the second screen transition is insufficient for the first screen transition The operation unit has a reception unit that receives a value input to the region from the user, and the operation unit inputs the value received by the reception unit to the transition source screen.

It is possible to efficiently execute the test related to the Web application.

It is a figure which shows the example of the screen which it wants to test as a different screen even with the same URL. It is a figure which shows the example of a form in which it is difficult to generate an appropriate input value. It is a figure which shows an example of Tree Edit Distance. It is a figure which shows the system structural example in embodiment of this invention. It is a figure which shows the hardware structural example of the client apparatus 10 in embodiment of this invention. It is a figure which shows the function structural example of the client apparatus 10 in embodiment of this invention. 6 is a flowchart for explaining an example of a processing procedure executed by the client device 10. It is a figure which shows an example of a static analysis. It is a figure which shows an example of a screen transition diagram. 9 is a flowchart illustrating an example of a processing procedure of screen transition information extraction processing. 6 is a flowchart for explaining an example of a processing procedure of test case generation and execution processing. It is a figure which shows an example of a form. 9 is a flowchart illustrating an example of a processing procedure of a test input value generation processing. It is a figure which shows the definition example of HTML of a certain form. It is a figure which shows an example of a dictionary. 7 is a flowchart for explaining an example of a processing procedure of determination processing of a transition destination screen. It is a figure which shows an example of a DOM tree. It is a flow chart for explaining an example of a processing procedure of generation processing of a screen transition diagram. It is a flow chart for explaining an example of a processing procedure of auxiliary processing of an input value to a test case. It is a figure which shows the example of a display of an input information auxiliary file. 9 is a flowchart illustrating an example of a processing procedure of test script output processing.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this embodiment, an approach is proposed in which a specification is reversed from a test target, an exhaustive test case is automatically generated based on the reversed specification, and is output as a test script. This approach allows us to automatically generate exhaustive test scripts without the need for additional work.

In this embodiment, since the screen transition test in the functional test is targeted, the process is executed in the following steps.
(1) The test target is analyzed, and the screens constituting the test target as specifications and the connections between the screens and the means for reaching each screen are extracted.
(2) A screen transition diagram suitable for the screen transition test in the functional test is generated from the specifications acquired in (1).
(3) Output the test script corresponding to each transition of the screen transition diagram generated in (2) according to the script format to be output based on the information of the means for reaching the screen extracted in (1) ..

In (1), a screen constituting the test target and a means for reaching the screen are extracted. In a Web application, a screen is generally generated dynamically from a file that is a source of a screen (HTML) such as a jsp file in response to a request from the client side. Therefore, the information about what kind of screen the test target is composed of and how they are connected is not directly described in the source code, and thus an appropriate analysis is required (Technical Problem 1: Test Examination of the method to reverse the specification from the target).

Further, also in the extraction of means for reaching the screen, in the Web application, there is a screen transition that cannot be transited without giving a specific input value. For example, in a system for registering a telephone number, there may be a case where the registration completion screen cannot be displayed unless it is an 11-digit half-width number. It is necessary to generate an appropriate input value for such a screen transition (Technical Problem 2: Examination of an appropriate input value generation method for realizing the screen transition).

In (2), the screen transition diagram suitable for the screen transition test in the functional test is generated by using the screen information constituting the test target acquired in (1). In the screen transition test in the functional test, it is necessary to identify the screen by the granularity of the function. Generally, screens are identified by a URL, but screens with the same URL may be identified as different screens by a functional test. For example, on the search site, the screen when there is no search result and the screen when there is a search result may be the same when viewed from the URL, but they are regarded as separate screens from the viewpoint of functional testing. .. Therefore, it is necessary to perform appropriate screen identification in accordance with the viewpoint of the functional test (Technical Problem 3: Examination of an appropriate screen identification method in the functional test).

(3) is a step of generating a test script according to the script format to be output. Describe the means for reaching the screen extracted in (1) (clicking links and buttons, inputting in the form (input area), etc.) in the script format you want to output. Regarding this step, it is considered that there is no technical problem because only the method corresponding to the extracted means is described.

From the above, it is considered that this approach has the following three technical issues.
Technical Problem 1: Examination of the method of reversing the specifications from the test target.
Technical problem 2: Examination of an appropriate input value generation method for realizing screen transition.
Technical Issue 3: Examination of an appropriate screen identification method in a functional test.

[Technical subject 1: Examination of the method to reverse the specifications from the test target]
<Prior art and problems>
There is a technique that uses static analysis as a technique for reversing specifications from an application (for example, "Reconstructing Software Architecture for J2EE Web Applications" Minmin Han, Christine Hofmeister, Robert L. Nord, Proceedings of the 10th Working Conference on Reverse Engineering 2003"). However, like HTML dynamically generated from a jsp file, information that cannot be generated without execution cannot be obtained, and therefore the specification cannot be completely reversed. There is also a technology that combines static analysis and dynamic analysis to compensate for such drawbacks (for example, "Silva CE, Campos JC. 2013. "Combining Static and Dynamic Analysis for the Reverse Engineering of Web Applications". Proceedings of the 5th Symposium on Engineering Interactive Computing Systems-EICS.: 107-112."). The present technology aims to cover the behavior of the Web application by identifying paths in the program by static analysis and generating test cases that cover all of these paths by dynamic analysis. However, in order to get the path passed by the dynamic analysis, it is necessary to insert the code that can get the log in all the places where the input value can pass, and the user can use the system from client side to server side. Since the environment for accurately grasping the configuration and outputting the log indicating the place where the program has passed must be prepared, the purpose of this embodiment is to reduce the additional operation for introducing the tool. Does not match. In addition, the behavior may be changed by modifying the test target, and when the test case generated from such test target is used for the actual test, the test contents different from the expected test scenario are tested. There is a risk that

<Proposal in this Embodiment: Reverse Technology of Specifications Using Static Analysis and Dynamic Analysis Focusing on Controller>
In the present embodiment, attention is paid to a controller that determines the process to be executed in response to a request from the client side in the configuration of MVC (Model View Controller) that is a design pattern that is generally used in Web applications. For example, a servlet is an example of a controller. By grasping all the transitions allocated by the controller by static analysis, and covering all those transitions by subsequent dynamic analysis, the screens that make up the test target are connected to each other, and the means to reach each screen is omitted. Can be reversed without. As a difference from the conventional method, since only the controller is focused in this embodiment, it is possible to infer to what extent the transition destination indicated by the controller can be covered from the information on the client side, and the environment for acquiring the log can be set. The advantage is that the operation to be prepared is unnecessary or can be reduced. With the conventional method of acquiring the execution path of the program, it is impossible to infer detailed information of the execution path from the client side.

[Technical subject 2: Examination of an appropriate input value generation method for realizing screen transition]
<Prior art and problems>
There is a method using Search Based Software Testing (hereinafter referred to as "SBST") as a method for generating an appropriate input value for realizing screen transition (N. Alshahwan and M. Harman, "Automated web. application testing using search based software engineering," 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011), Lawrence, KS, 2011, pp. 3-12.). In the input value generation using SBST, the heuristic search algorithm is used to generate the test input value that satisfies the requirement to be achieved, based on the evaluation function designed to quantitatively evaluate the degree of achievement for the requirement to be achieved. To do. For example, when there is a screen transition that transitions only when “a>10” for the input value a, the evaluation function is set as “a-10”. At this time, when the value of the evaluation function is larger than 0, a desired screen transition can be realized. First, when the input value a=0, the value of the evaluation function becomes −10. Next, when the input value a=2, the value of the evaluation function becomes −8, which approaches the condition that the value of the evaluation function is larger than 0, which is a condition for realizing a desired screen transition. Therefore, it is determined that the larger the input value a is, it is possible to generate a value for realizing a desired screen transition such as a=12 by gradually increasing the value of the input value a. it can.

The input value generation using the SBST is simple in the case where the condition for realizing the desired screen transition can be placed as an evaluation function capable of quantitative evaluation, as in the case of the above numerical type. It is valid. However, if it is necessary to judge what input value is required for the input form and give an appropriate input value based on the judgment result, design an evaluation function that enables quantitative evaluation. Not so effective because it is difficult. Specifically, when inputting a character string type such as name and email address, and when performing a type check and transitioning to a normal screen if there is no problem, the type check condition (at the It is difficult to put (such as whether or not it is included) as an evaluation function that can be evaluated quantitatively, so it is difficult to generate an appropriate input value. In addition, if an appropriate input value cannot be generated, the user must perform all the tests on the screen on which the input form of the input value exists, and it is necessary to perform the screen operation until the screen transitions. If you include it, it will take a lot of work. Specifically, the input value of the input form on the screen g1 shown in FIG. 2 should be a mail address, but it is difficult to generate an appropriate input value for the input form. Therefore, the transition to the screen g2 is easy, but the transition to the screen g3 requires a lot of operations.

<Proposal in the present embodiment: test input value generation using peripheral information of input form and user assistive technology>
This embodiment is composed of two steps. In the first step, what kind of input value is obtained by using the peripheral information of the input form such as the label (that is, the information extracted from the description portion of the screen description information corresponding to the input form). Whether a specific word corresponding to the requested content is obtained from a dictionary prepared in advance and the word is given as an input value, an appropriate input value suitable for each input form is given. You can

In the second step, the minimum required operation is presented in which only the input value is given by presenting the user with the information on the input form for which an appropriate input value could not be generated in the first step and requesting the user to assist the input value. With, it is possible to supplement the parts that could not be automatically generated.

[Technical subject 3: Examination of appropriate screen identification method in functional test]
<Prior art and problems>
In the present embodiment, the screen transition test in the functional test is described in the functional test as follows: "For the interaction from the client side, it is possible to confirm whether the screen is displayed according to the specifications on the screen displayed on the client side. "Functional test". Generally, screens are identified by URL, but in the screen transition test in this definition, it is important to test even screens with the same URL as different screens depending on the screen. For example, in the search site, the screen when there is no search result and the screen when there is a search result as shown in FIG. 1 may be the same when viewed by URL, but they are different from the viewpoint of functional test. Regarded as the screen. In such cases, it should be identified from the displayed screen rather than the URL. However, when the displayed screens are simply compared as images, if the functional parts to be confirmed are the same but the other parts are different, they are determined to be different screens. Therefore, it is necessary to compare some of the elements that make up the screen rather than the entire screen. The screen is mainly composed of elements such as HTML, DOM (Document Object Model) structure, text, image and CSS (Cascading Style Sheets).

Although the points to be noted differ depending on the test viewpoint, the present embodiment is directed to the case where the difference in function appears in the HTML DOM structure. As an existing method for comparing the DOM structure of HTML, there is a method using Tree Edit Distance (hereinafter referred to as “TED”) (for example, “A survey on tree edit distance and related problems” Philip Bille. 2004. Theoretical Computer Science. Volume 337, Issues 1-3, 9 June 2005, Pages 217-239.").

The TED is the number of deletions/insertions when deleting/inserting a node until both DOM trees match. For example, in order to match DOM tree 1 and DOM tree 2 in FIG. 3, it is necessary to delete three nodes <p>, <strong>, and <U> of DOM tree 2 and insert an <img> node. Therefore, the value of TED is 4. When the threshold is 3 or less, it can be determined that the structures of the DOM tree 1 and the DOM tree 2 do not match, and when the threshold is 4 or more, it can be determined that the structures of the DOM tree 1 and the DOM tree 2 match.

One of the drawbacks of the screen structure comparison using TED is that it is difficult to set a threshold value because an appropriate threshold value is different for each screen. There are two possible causes for requiring the threshold value (allowable range for TED).
Cause 1: Presence of node that does not affect screen identification in functional test Cause 2: Presence of parallelized node Regarding cause 1, bold tag <b>, etc. does not affect screen identification in functional test. It can be mentioned that there is a node. The cause 2 may be a heading tag or the like. For example, it can be said that the function of displaying news is the same even when the number of headlines is different on the news site, and therefore the number of headlines does not affect the function. In this way, there are nodes that do not affect the identification of the screen, and these nodes increase the TED value, so it is necessary to set a threshold value.

<Proposal in this Embodiment: Screen Identification Technology Using Similarity of HTML DOM Structure>
In the present embodiment, the nodes related to cause 1 are deleted and the nodes related to cause 2 are aggregated to abstract the HTML, and the abstracted HTMLs are compared to each other, thereby enabling the function without setting a threshold value. Allows proper screen identification in testing.

Hereinafter, a specific example for realizing the above will be described. FIG. 4 is a diagram showing a system configuration example in the embodiment of the present invention. In FIG. 4, the server device 20 and the client device 10 are connected via a network such as a LAN (Local Area Network) or the Internet, for example.

The server device 20 is one or more computers including a system to be tested including a web application.

The client device 10 is a device that supports a test (verification) of a Web application included in the server device 20.

FIG. 5 is a diagram showing a hardware configuration example of the client device 10 according to the embodiment of the present invention. The client device 10 of FIG. 5 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a CPU 104, an interface device 105, a display device 106, an input device 107, and the like, which are connected to each other by a bus B.

A program that implements the processing in the client device 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed in the auxiliary storage device 102 from the recording medium 101 via the drive device 100. However, the program does not necessarily have to be installed from the recording medium 101, and may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program and also stores necessary files and data.

The memory device 103 reads the program from the auxiliary storage device 102 and stores the program when an instruction to activate the program is given. The CPU 104 realizes a function related to the client device 10 according to a program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network. The display device 106 displays a GUI (Graphical User Interface) or the like according to a program. The input device 107 includes a keyboard, a mouse, and the like, and is used to input various operation instructions.

FIG. 6 is a diagram showing a functional configuration example of the client device 10 according to the embodiment of the present invention. 6, the client device 10 includes a screen transition information extraction unit 11, a test scenario generation unit 12, a screen automatic operation unit 13, a screen evaluation unit 14, a test material generation unit 15, and the like. Each of these units is realized by a process that causes the CPU 104 to execute one or more programs installed in the client device 10.

The screen transition information extraction unit 11 analyzes the controller using a static analysis technique based on the source code of the controller of the Web application, and acquires the number of transition destinations for each request.

The test scenario generator 12 generates a test scenario and a test input value for each screen displayed in the test. The test input value for each form of a certain screen is generated using the peripheral information of the form in the HTML DOM tree of the screen. The peripheral information of the form is an example of information defined for the form in HTML.

The screen automatic operation unit 13 realizes dynamic analysis by automatically executing screen operations using the generated test scenario and test input value as a test case.

The screen evaluation unit 14 determines whether or not the screen transitioned by executing the screen operation has already been transited using information such as URL and HTML. The screen evaluation unit 14 performs edit processing for abstracting each of the HTML data of the screen that has transitioned due to the execution of the screen operation and the HTML data of the screen that has already transitioned, and the edited HTML data. To compare.

The test material generation unit 15 generates a screen transition diagram based on the information obtained by the static analysis and the dynamic analysis. The test material generation unit 15 also outputs the test scenario and the test input value generated by the test scenario generation unit 12 in the form of a test script.

The processing procedure executed by the client device 10 will be described below. FIG. 7 is a flowchart for explaining an example of a processing procedure executed by the client device 10.

In step S101, the screen transition information extraction unit 11 analyzes the source code of the controller using a static analysis technique, and acquires the number of transition destinations for each request from the information of the controller that determines the transition destination of the screen. The information indicating the number of transition destinations for one request is called "screen transition information", and the list of screen transition information is called "screen transition information list".

FIG. 8 is a diagram showing an example of static analysis. FIG. 8 shows a part of the source code of the controller. In FIG. 8, description d1 indicates a request. The description d2 and the description d3 indicate the transition destination corresponding to the request. In step S101, it is extracted from the descriptions d1, d2, and d3 that there are two transition destinations for the request "owners/new". Such extraction is performed for all requests defined in the source code.

Subsequently, the screen automatic operation unit 13 accesses the initial URL and acquires HTML data or the like corresponding to the initial URL (S102). As a result, a screen based on the HTML data or the like is displayed on the display device 106. The initial URL may be input by the user or may be prestored in the auxiliary storage device 102, for example.

In the subsequent steps S103 to S108, dynamic analysis or the like is performed on the HTML data corresponding to the initial URL and all the HTML data that can be transitioned from the screen related to the HTML data as a starting point. Specifically, to cover all transition destinations for each request acquired by static analysis, as a dynamic analysis, the screen operation of the test target is automatically performed starting from the screen corresponding to the initial URL. To be done. In dynamic analysis, all links on each screen are clicked, and if there is an input form and the corresponding submit button, static analysis is performed for the request when the submit button is pressed. Test input values are generated and screen operations are repeated until the number of acquired transition destinations of the request is satisfied. Regarding whether or not the number of transition destinations of the request is satisfied, the transition destination screen is compared with the already transited screen, and it is determined whether or not the screen is a newly transited screen, and the number of reached screens is counted. By the above dynamic analysis, the information (HTML, URL, etc.) of the transition destination screen and the connection between the screens, and the means for reaching the screen are acquired. If a screen transition that does not pass through the controller is found by dynamic analysis, information on the screen transition is also added. The screen transition that does not pass through the controller is, for example, a screen transition that occurs when a link in the a tag is clicked.

Hereinafter, the HTML data targeted for processing between steps S103 and S108 is referred to as "target HTML".

In step S104, the test scenario generation unit 12 generates a test scenario for the screen related to the target HTML (hereinafter referred to as “target screen”). That is, a test scenario including a set of an operation and a locator is generated for each screen element (link, submit button, etc.) that causes a screen transition in the target screen. The operation is an operation such as clicking. The locator is identification information of the screen element to be operated.

The following steps S105 to S107 are executed for each test scenario generated in step S104. The test scenario targeted for processing in S105 to S107 is hereinafter referred to as "target test scenario".

In step S106, the test scenario generation unit 12 generates a test case based on the target test scenario, and the screen automatic operation unit 13 executes the test case on the target screen. As a result, screen transition occurs. When the target test scenario requires an input value, the test input value is generated in step S106.

When steps S103 to S108 are executed for all the transition destination screens (HTML), the test material generation unit 15 generates a screen transition diagram (S109). Specifically, the test material generation unit 15 compares the generated screen transition information list with the screen transition actually occurred in steps S103 to S108, and if there is a screen transition that has not occurred in the test. The screen transition is extracted (detected), and a screen transition diagram in which information of screen transitions that can be transited and screen transitions that cannot be transited is combined is generated. For screen transitions that could not be transited, a test script could not be generated, so support such as manual creation may be performed.

FIG. 9 is a diagram showing an example of a screen transition diagram. FIG. 9 shows that the screen A transits to the screen B and the screen C, and the screen C transits to the screen B. Also, the broken-line rectangle indicates a screen in which the transition is scheduled in the specifications, but the transition does not occur in the test. That is, the source code defines that there is a screen that transitions from the screen C, but the dynamic analysis indicates that the transition to the screen did not occur.

Subsequently, the test material generation unit 15 determines whether or not there is a portion that could not be transitioned in the screen transition diagram (S110). That is, in FIG. 9, it is determined whether or not there is a portion corresponding to the dashed rectangle.

When there is no corresponding part (No in S110), the test material generation unit 15 outputs the test scenario and the test input value when each screen transition occurs in the form of a test script (S112).

On the other hand, when there is a corresponding portion (Yes in S110), the test material generation unit 15 executes a process for receiving the input value assistance for the test case from the user (S111). Subsequent to step S111, steps S101 and thereafter are re-executed.

Next, details of step S101 in FIG. 7 will be described. FIG. 10 is a flowchart for explaining an example of the processing procedure of the screen transition information extraction processing.

In step S301, the screen transition information extraction unit 11 reads the source code of the controller. The source code may be stored in the auxiliary storage device 102 in advance.

Subsequently, the screen transition information extraction unit 11 executes steps S302 to 304 for each request detected from the source code. In step S303, the screen transition information extraction unit 11 identifies the number of transition destinations for the request to be processed by referring to the source code, and adds the set of the request and the number to the screen transition information list. Therefore, information on all screen transitions scheduled in the source code (specification) is registered in the screen transition information list.

Next, details of step S106 in FIG. 7 will be described. FIG. 11 is a flowchart for explaining an example of a processing procedure of test case generation and execution processing.

In step S401, the test scenario generation unit 12 determines whether the operation of the target test scenario includes a form input operation. When the operation of the target test scenario does not include the input operation to the form (No in S401), the test scenario generation unit 12 sets the target test scenario as the test case as it is (S402). Subsequently, the screen automatic operation unit 13 executes the test case on the target screen (S403). Screen transition occurs due to the execution of the test case. Subsequently, the screen evaluation unit 14 stores the request generated by clicking the link or pressing the button by executing the test case as "request A" (S405).

Subsequently, the screen evaluation unit 14 determines whether or not the screen transitioned in step S403 has already been transited by using information such as URL and HTML (S405). At this time, when the screen that has transited in step S403 is a screen that has not yet transited, information regarding the transition is added as an element of the transition list. The transition list is a list whose elements are information about transitions between screens that have occurred in the dynamic analysis. The element includes the URL of the pre-transition screen, the URL of the post-transition screen, the request A, the HTML of the post-transition screen, the test case, and the like.

On the other hand, when the operation of the target test scenario includes the input operation to the form (Yes in S401), the test scenario generation unit 12 generates the input value (test input value) to the form (S406). For example, if the target screen is as shown in FIG. 12, in the target test scenario, the input operation to the form corresponding to the surname, the input operation to the form corresponding to the first name, and the input to the form corresponding to the email address This includes operations and pressing of registration buttons. Therefore, in this case, a test input value is generated for each form. Multiple test input values may be generated for one form. Therefore, the following steps S407 to S412 are executed for each combination of test input values for each form. The combination may be all combinations, or a predetermined number of combinations may be extracted from all combinations. For example, in the case where N test input values are generated for a family name, M test input values are generated for a first name, and L test input values are generated for an email address, N* is a combination of all combinations. It means M×L combinations. Hereinafter, a combination of processing targets is referred to as a “target combination”.

In step S408, the test scenario generation unit 12 generates a test case in which each test input value belonging to the target combination is an input target for the corresponding form based on the target test scenario. Subsequently, the screen automatic operation unit 13 executes the test case on the target screen (S409). That is, the test input values belonging to the target combination are input to each form that is the input destination of the value in the target test scenario, and the submit button that is the pressing target in the target test scenario is pressed. As a result, screen transition occurs.

Subsequently, the screen evaluation unit 14 stores the request generated by pressing the button as "request A" (S410). Then, the screen evaluation part 14 performs the same process as step S405 (S411).

When steps S407 to S412 are executed for all the combinations, the screen evaluation unit 14 determines the number of URLs of the screen after transition for the request A in the transition list (that is, the number of elements including the request A, hereinafter “dynamic transition number”). Is equal to the number of transition destination URLs corresponding to the request A in the screen transition information list (hereinafter, referred to as “static transition number”) (S413). That is, in step S413, it is determined whether or not all the transitions defined in the source code for request A have been detected.

When the number of dynamic transitions is equal to the number of static transitions (Yes in S413), the process of FIG. 11 ends for the target test scenario. When the number of dynamic transitions does not reach the number of static transitions, that is, when the number of dynamic transitions is insufficient for static transitions (No in S413), the screen evaluation unit 14 performs a test on each form of the target screen. It is determined whether or not all the input values that can be generated by the test scenario generation unit 12 have been generated as the input values (S414). If all the input values that can be generated by the test scenario generation unit 12 have not been generated, that is, if there is room for generation of different input values by the test scenario generation unit 12 (No in S414), Step S406 and subsequent steps are repeated. Be done. In this case, for the target test scenario, a combination of different test input values is generated and the test case is executed.

On the other hand, when all the input values that can be generated by the test scenario generation unit 12 have been generated as the test input values for each form of the target screen (Yes in S414), the test material generation unit 15 sets the target information in the input information auxiliary file. The URL and title of the screen and the peripheral information (label, id and name) of each form on the target screen are added (recorded) (S415). That is, in the input information auxiliary file, the URL, the title, and the peripheral information of each form are recorded for each screen for which an input value that covers all screen transitions could not be generated. Here, the input of the input value to the form registered in the input information auxiliary file is assisted by the user in step S111 of FIG.

Although the format of the input information auxiliary file is not limited to a predetermined format, for example, a tabular file is suitable. The input information auxiliary file is stored in the auxiliary storage device 102, for example.

The determination in step S414 may be performed based on whether or not the number of repetitions after step S406 has reached the upper limit.

Next, details of step S406 will be described. FIG. 13 is a flowchart for explaining an example of the processing procedure of the test input value generation processing.

Steps S501 to S508 are executed for each input target form (form of target screen) in the target test scenario. Hereinafter, the form to be processed will be referred to as the “target form”.

In step S502, the test scenario generation unit 12 acquires the peripheral information (or attribute information) of the target form from the target HTML. In the present embodiment, the label (label), id, name and the like of the target form are acquired as the peripheral information. For example, if the target form is the third form in FIG. 12 (e-mail address input form) and the HTML descriptive information regarding the form is as shown in FIG. 14, the label (label) is “mail”. Please enter the address", "ma" as the id, and "mail" as the name.

Subsequently, the test scenario generation unit 12 determines whether the combination of the peripheral information and the URL and title of the target screen is registered in the input information auxiliary file (S503). That is, it is determined whether or not the target form is registered in the input information auxiliary file.

When the target form is not registered in the input information auxiliary file (No in S503), the test scenario generation unit 12 divides each character string as the peripheral information acquired in step S502 into words using a morphological analysis technique. (S504). According to the above example, a word group such as ["mail address", "input", "mail", "ma"] is obtained.

In subsequent steps S505 to S507, step S506 is executed for each word included in the word group. Hereinafter, the word to be processed will be referred to as a “target word”. In step S506, the test scenario generation unit 12 retrieves a value corresponding to the target word from a dictionary prepared in advance and sets the retrieved value as an input value for the target form.

FIG. 15 is a diagram showing an example of the dictionary. As shown in FIG. 15, the dictionary is information including an item name for each item and a specific value regarding the item, and is stored in, for example, the auxiliary storage device 102 or the like. Alternatively, a DB that stores attribute information of users of some system may be used as a dictionary.

In step S506, the test scenario generation unit 12 searches the dictionary for an item name that matches the target word. However, an item name having a meaning similar to the target word may be searched using the synonym dictionary. When the corresponding item name is searched, the test scenario generation unit 12 acquires a specific value registered in the item related to the item name as an input value. If the corresponding item name is not searched, the input value will not be obtained.

For example, in [“email address”, “input”, “mail”, “ma”], when “email address” is the target word, for example, “xxx@xxx.xx” is acquired as the input value. To be done. In addition, in the example of ["email address", "input", "mail", "ma"], only one input value is acquired, but a plurality of words that match different item names are obtained from the peripheral information. For the extracted form, multiple input values may be generated.

On the other hand, when the target form is registered in the input information auxiliary file (Yes in S503), the test scenario generation unit 12 sets the value recorded in the input information auxiliary file in association with the peripheral information of the target form. When it is acquired as an input value (S509), in this case, the process regarding the target form ends. If step S111 of FIG. 19 is executed for the target form, the input value for the target form is recorded in the input information auxiliary file.

Next, details of steps S405 and S411 in FIG. 11 will be described. FIG. 16 is a flowchart for explaining an example of the processing procedure of the transition destination screen determination processing.

In step S601, the screen evaluation unit 14 includes the URL of the current transition destination screen (that is, the target screen) as the URL of the post-transition screen in any element including the request A in the transition list. It is determined whether or not there is. When the URL of the current transition destination screen is not included in the transition list as the URL of the post-transition screen of request A (No in S601), the screen evaluation unit 14 requests the request A, the URL of the pre-transition screen, and the current transition destination. An element that sets the screen URL, the HTML of the current transition destination screen (target HTML), and the test case related to the target test scenario is added to the transition list (S602).

On the other hand, when the URL of the current transition destination screen is included in the transition list as the URL of the post-transition screen of the request A (Yes in S601), the screen evaluation unit 14 includes the request A in the transition list, and Steps S603 to S611 are executed for each HTML data of the post-transition screen of the element that includes the URL of the post-transition screen as the URL of the post-transition screen. Hereinafter, the HTML to be processed is referred to as "target transition completed HTML". Further, the HTML of the current transition destination screen is referred to as "target HTML" as defined above.

Subsequently, the screen evaluation unit 14 executes steps S604 to S608 for each of the target transition completed HTML and the target HTML. Of the target transition completed HTML and the target HTML, the HTML targeted for processing is referred to as “target HTML”.

In step S605, the screen evaluation unit 14 creates a DOM tree of the HTML of interest. FIG. 17 shows an example of the DOM tree. As shown in FIG. 17, the DOM tree shows a hierarchical structure of each node (each screen element).

Subsequently, the screen evaluation unit 14 deletes the node that does not affect the role of the screen (that is, does not affect the identification of the screen) from the DOM tree (S606). There are three types of nodes that do not affect the role of the screen, such as attribute nodes, text nodes, and nodes that represent the size, font, and position of characters (Center tag, U tag, etc.). Therefore, the node indicated by the dashed rectangle in FIG. 17 is deleted.

Subsequently, the screen evaluation unit 14 collects nodes (parallel nodes) having a parallel relationship in the DOM tree into one node (S607). The parallel node refers to a plurality of nodes having the same tag name and belonging to the same hierarchy in the hierarchical structure of the DOM tree. In FIG. 17, the nodes corresponding to the two <h1> tags surrounded by the one-dot chain line correspond to parallel nodes. Aggregation of nodes may be achieved by leaving one of the parallel nodes and deleting the other, or by replacing the parallel nodes by one node that abstracts each node. Good.

By performing steps S604 to S608 for each of the target transition completed HTML and the target HTML, an abstracted DOM tree is generated for each.

Subsequently, the screen evaluation unit 14 converts each of the two abstracted DOM trees into HTML, compares the conversion results, and outputs the comparison result (S609). That is, the identity between the screen related to the target transition completed HTML and the screen related to the target HTML is determined. When the compared HTMLs match each other, “match” is output as the comparison result. Otherwise, "mismatch" is output as the comparison result. It should be noted that complete matching may not be required for matching the compared HTMLs. For example, the appearance order of the same tag may be different.

When the comparison result is “match” (Yes in S610), the process regarding the target transition completed HTML ends. When the comparison result is “mismatch” (No in S610), the screen evaluation unit 14 executes the same process as step S602 (S612), and ends the process in FIG.

Next, details of step S109 in FIG. 7 will be described. FIG. 18 is a flowchart for explaining an example of a processing procedure of screen transition diagram generation processing.

Steps S701 to S703 are executed for each element (URL of screen before transition, URL of screen after transition, request, HTML of screen after transition, and set of test cases) included in the transition list. Hereinafter, the element to be processed is referred to as "target element".

In step S702, the test material generation unit 15 adds an element including the request of the target element, the URL of the screen before the transition of the target element, and the URL of the screen after the transition of the target element to the screen transition diagram list. That is, the screen transition diagram list is a list having a set of the request, the URL of the screen before transition, and the URL of the screen after transition as one element.

Steps S704 to S709 are executed for each element of the screen transition diagram list. Hereinafter, the element to be processed is referred to as "target element".

In step S705, the test material generation unit 15 counts the number of elements in the screen transition diagram list in which the URL of the request and the screen before transition is common to the target element. The number of elements is the number of screens transitioned from the same screen (pre-transition screen) based on the same request in the dynamic analysis. Hereinafter, the number of elements is referred to as “dynamic transition number n”.

Subsequently, the test material generation unit 15 acquires the number of transition destinations corresponding to the request of the target element from the screen transition information list (S706). The number of transition destinations is the number of transition destinations of the request extracted from the source code in the static analysis. Hereinafter, the number of transition destinations is referred to as “static transition number m”.

Subsequently, the test material generation unit 15 determines whether or not the number of dynamic transitions n is equal to or more than the number of static transitions m (S707). When the dynamic transition number n is equal to or greater than the static transition number m (Yes in S707), the process for the target element ends. When the number n of dynamic transitions is less than the number m of static transitions (No in S707), the test material generation unit 15 detects a difference between the dynamic transition and the static transition. That is, the presence of a screen that has not changed in the dynamic analysis is detected. Therefore, the test material generation unit 15 adds the elements including the request of the target element and the URL of the pre-transition screen of the target element to the screen transition diagram list by the number of mn (S708). That is, the element related to the transition corresponding to the rectangle of the broken line in FIG. 9 is added.

When steps S704 to S709 are executed for all the elements of the screen transition diagram list, the test material generation unit 15 generates a screen transition diagram based on the screen transition diagram list (S710). At this time, an element that does not have the URL of the post-transition screen is represented as a screen that cannot be transited, such as a dashed rectangle. The generated screen transition diagram may be displayed on the display device 106, or data indicating the screen transition diagram may be stored in the auxiliary storage device 102.

Next, details of step S111 in FIG. 7 will be described. FIG. 19 is a flowchart for explaining an example of a processing procedure of auxiliary processing of an input value for a test case.

In step S801, the test material generation unit 15 displays the input information auxiliary file on the display device 106.

FIG. 20 is a diagram showing a display example of the input information auxiliary file. As shown in FIG. 20, the input information auxiliary file includes “URL” and “title” for each screen that could not achieve all screen transitions, and “label” and “label” for each form of the screen. It has a format capable of storing “id”, “name”, “value” and the like. Among these, the values of “URL”, “title”, “label”, “id”, and “name” are registered in step S415 of FIG.

Subsequently, the test material generation unit 15 receives the input of the “value” of each form from the user via the input information auxiliary file (S802). That is, for each form registered in the input information auxiliary file, the user uses the “label”, “id”, “name”, and the like as clues to infer an appropriate value for the form, and Fill in the "Value" of the form.

Subsequently, the test material generation unit 15 saves the input information auxiliary file in the state where the “value” is input (S803).

When step S101 is executed after the value is input for the “value” of the input information auxiliary file, it is determined Yes in step S503 of FIG. 13 for each form registered in the input information auxiliary file. Becomes Therefore, in step S509, an input value is acquired from the input information auxiliary file for each form. As a result, there is an increased possibility that screen transitions that could not be achieved last time can be realized.

Next, details of step S112 in FIG. 7 will be described. FIG. 21 is a flowchart for explaining an example of a processing procedure of test script output processing.

In steps S901 to S903, the test material generation unit 15 outputs the test script by outputting the test case included in each element of the transition list in the script format (S902).

As described above, according to the present embodiment, the screen transition information (static screen transition information) scheduled as the Web application specifications is extracted from the source code, and the operation for the Web application is automatically executed. The screen transition information (dynamic screen transition information) generated by repeating the screen transition is automatically extracted, and the difference between the static screen transition information and the dynamic screen transition information is detected. can do. As a result, screen transitions that are defined in the source code but cannot be generated by the test script can be detected. At this time, in the present embodiment, peripheral information regarding each input area (each form) of the screen of the transition source of the screen transition that could not be generated by the test script is recorded in the input information auxiliary file. After that, the input information auxiliary file is displayed, and the input value for each input area is input by the user.

Therefore, it is possible to reduce the input load of the input value by the user as much as possible, and it is possible to increase the possibility that the screen transition that cannot be generated by the test script is realized. As a result, it is possible to improve the efficiency of executing the test related to the Web application.

Further, when the input value is received from the user, the peripheral information of each input form is displayed. Therefore, the user can infer a value suitable for each input form by using the peripheral information as a clue.

Further, according to the present embodiment, it is possible to reverse the specification from the test target, automatically generate a test case based on the reversed specification, and output it as a test script. This makes it possible to automatically generate a test script for a comprehensive screen transition test in a Web application without requiring additional operation. As a result, for example, the cost of software testing can be reduced.

Further, according to the present embodiment, with respect to the technical problem 1, it is possible to automatically reverse the means for reaching each screen automatically from the test target, the screens constituting the test target and the screens. In addition, since information on the number of screen transitions with respect to the request can be obtained in advance by the static analysis, it becomes easy to make a decision of search termination during the dynamic analysis, and the processing can be performed within a realistic time. In the present embodiment, server-side information is not required, so that the user does not need to prepare an environment for obtaining server-side information, and it is possible to suppress additional operation for introducing a tool.

Further, according to the present embodiment, regarding the technical problem 2, it is possible to generate an appropriate test input value for realizing the screen transition by using the client side information (HTML). Unlike the conventional technology, since the server side information is not required, the user does not need to prepare an environment that allows the server side information to be acquired, and it is possible to suppress the additional operation for introducing the tool. Further, even when the appropriate test input value cannot be generated, the user can assist the user as described above, and thus the possibility of covering the screen transition can be increased.

Further, according to the present embodiment, with respect to the technical problem 3, it is possible to appropriately identify the screen in the functional test without the need to set the threshold value, and improve the quality of the verification pattern.

Note that in the present embodiment, the client device 10 is an example of a test execution device. The screen transition information extraction unit 11 is an example of an extraction unit. The test scenario generation unit 12 is an example of a first generation unit. The screen automatic operation unit 13 is an example of an operation unit. The screen evaluation unit 14 is an example of a second generation unit. The test material generation unit 15 is an example of a reception unit and a recording unit. HTML data is an example of screen description information. The screen transition information list is an example of information on the first screen transition. The transition list is an example of information on the second screen transition. The input information auxiliary file (auxiliary storage device 102) is an example of a storage unit.

Although the examples of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims. -Can be changed.

10 Client Device 11 Screen Transition Information Extraction Unit 12 Test Scenario Generation Unit 13 Screen Automatic Operation Unit 14 Screen Evaluation Unit 15 Test Material Generation Unit 20 Server Device 100 Drive Device 101 Recording Medium 102 Auxiliary Storage Device 103 Memory Device 104 CPU
105 interface device 106 display device 107 input device B bus

Claims (4)

An extraction unit for extracting information on the first screen transition defined in the source code from the source code of the Web application;
A first generation unit that generates an input value for a screen displayed regarding the Web application;
An operation unit for inputting an input value generated by the first generation unit to the screen and automatically operating the screen;
A second generation unit that generates information on a second screen transition realized by automatic operation;
When the second screen transition is insufficient for the first screen transition, a reception unit that receives a value input from the user for each input area for the transition source screen of the insufficient screen transition,
The operation unit inputs the value received by the reception unit to the transition source screen,
A test execution device characterized by the above. A recording unit that records information about each input area of the transition source screen of the insufficient screen transition in the storage unit,
The reception unit displays the information recorded in the storage unit for each input area related to the information, and receives a value input from the user from the input area.
The test execution device according to claim 1, wherein The information regarding the input area is information extracted from the description portion regarding the input area in the description information of the screen including the input area.
The test execution device according to claim 2, wherein An extraction procedure for extracting information on the first screen transition defined in the source code from the source code of the Web application;
A first generation procedure for generating an input value for a screen displayed for the Web application,
An operation procedure of inputting an input value generated by the first generation procedure to the screen and automatically operating the screen,
A second generation procedure for generating information on the second screen transition realized by automatic operation,
When the second screen transition is insufficient for the first screen transition, the computer is made to execute an acceptance procedure for accepting a value input from the user for each input area for the transition source screen of the insufficient screen transition. ,
The operation procedure inputs the value accepted by the acceptance procedure to the transition source screen,
A program characterized by that.

Images