WO2012042560A1 - Image acquisition device, image acquisition method, and computer program - Google Patents

Abstract

An operation verification device (100) is provided with: a window display position acquisition unit (602) which acquires the on-screen display position of a browser window containing a client region where a web page is displayed and an additional region different from the client region; a client region determination unit (606) which determines the on-screen display position of the client region according to the on-screen display position of the browser window, the size of the additional region in the browser window, and the size of the status bar in the browser window; and an image acquisition unit (625) which captures a display image in the client region according to the determined on-screen display position.

Description

Image acquisition apparatus, image acquisition method, and computer program

This invention relates to a technique for acquiring an image of web content displayed on a screen.

∙ Operational testing before product shipment is indispensable to guarantee product quality. In general, a test engineer often performs an operation test by manually operating a product and confirming the operation result. However, manual operation tests are limited in the case of multi-function products with abundant operation variations.

Therefore, in many cases, the operation test is automatically executed by the test program to save labor of the operation test. For example, an automatic function test tool “QTP (Quick Test Professional)” provided by HP (Hewlett-Packard) automates an operation test by executing a test program created by a user.

JP 2004-227396 A

In an operation test of a web server that provides web content to an external device or a web application that runs on the web server, an image of the web content displayed on the external device may be captured. Then, the normality of the operation may be confirmed by a test engineer confirming the captured image.

By the way, when performing an operation test of a product using various web browsers, the display format of the window (hereinafter also referred to as “browser window”) in which the web browser is started differs depending on the web browser. For this reason, when the same web content is displayed on different web browsers, the present inventor has thought that the appearance of the entire browser window may be different, which may reduce the efficiency of the operation test.

The present invention has been completed based on the above-mentioned problem recognition of the present inventor, and its main purpose is to provide a technique for acquiring a display image of web content that is highly convenient in an operation test. .

In order to solve the above problems, an image acquisition device according to an aspect of the present invention provides a browser window display on a screen for a browser window including a client area in which a web page is displayed and an additional area different from the client area. Specifies the display position of the client area on the screen according to the window position acquisition unit that acquires the position, the display position of the browser window on the screen, the size of the additional area of the browser window, and the size of the status bar of the browser window And an image acquisition unit that captures a display image of the client area in accordance with the display position on the screen specified by the image position specification unit.

Another aspect of the present invention is an image acquisition method. This method is a method executed by the image acquisition device, and acquires the display position of the browser window on the screen for a browser window including a client area in which a web page is displayed and an additional area different from the client area. And specifying the display position of the client area on the screen according to the step of displaying, the display position of the browser window on the screen, the size of the additional area of the browser window, and the size of the status bar of the browser window Capturing a display image of the client area according to the display position on the screen specified in the step.

It should be noted that any combination of the above-described constituent elements and the expression of the present invention converted between the apparatus, method, system, program, recording medium storing the program, etc. are also effective as an aspect of the present invention.

According to the present invention, it is possible to obtain a web content display image that is highly convenient in an operation test.

It is a screen figure of a portal site 1st screen. It is a screen figure of a portal site 2nd screen. It is a figure which shows the log when a dictionary test is performed. It is a figure which shows the program for performing a dictionary test. It is a functional block diagram of an operation verification device. It is an outline figure until it acquires a log and performs an operation test. It is a screen figure of a log screen. It is a screen figure of a function setting screen. It is a screen figure of a program screen. It is a screen figure of an input setting screen. It is a screen figure of a case setting screen. It is a figure which shows the outline | summary of a structure in each of the operation verification apparatus of the reference technique 1, and the operation verification apparatus of the reference technique 2. It is a figure which shows the outline | summary of operation | movement in each of the operation verification apparatus of the reference technique 1, and the operation verification apparatus of the reference technique 2. It is a block diagram which shows the function structure of the operation verification apparatus of the reference technique 2. FIG. In the reference technique 2, it is a schematic diagram until it detects a user's data input operation and performs an operation test. It is a screen figure of the function setting screen in the reference technique 2. It is a screen figure of the function setting screen in the reference technique 2. It is a screen figure of the input setting screen in the reference technique 2. It is a screen figure of the case setting screen in the reference technique 2. It is a figure which shows the characteristic of two operation methods with respect to a web browser. It is a figure which shows typically the calling relationship between functions about two operation methods with respect to a web browser. It is a block diagram which shows the function structure of the operation verification apparatus of the reference technique 3. It is a block diagram which shows the detail of the program execution part of FIG. 10 is a flowchart showing the operation of the operation verification device of Reference Technique 3. It is a figure which shows typically the calling relationship between the functions in the operation | movement test of a web page. It is a figure which shows the example of data stored in a selenium window manager. It is a figure which shows the mechanism of operation with respect to FF dialog of FIG. It is a figure which shows the mechanism of operation with respect to the Safari dialog of FIG. It is a block diagram which shows the function structure of the operation verification apparatus of the reference technique 4. It is a block diagram which shows the detail of the program execution part of FIG. 10 is a flowchart showing the operation of the operation verification device of Reference Technique 4. FIG. 31 is a flowchart showing in detail a test program reading process in S402 of FIG. 30. FIG. It is a flowchart which shows the web browser starting process of S419 of FIG. 30 in detail. It is a flowchart which shows the web page operation process of S426 of FIG. 30 in detail. It is a flowchart which shows the operation process for Mac of S452 of FIG. 33 in detail. FIG. 34 is a flowchart showing in detail an IE operation process in S458 of FIG. 33. FIG. It is a flowchart which shows in detail the operation process for FF of S462 of FIG. It is a figure which shows the example of a display of the web content in a browser window. It is a figure which shows the captured image of the browser window of FIG. It is a block diagram which shows the function structure regarding the screen capture process in 1st Embodiment. It is a figure which shows the function which a proxy process part provides to a proof object setting part. It is a flowchart which shows operation | movement of the image capture process in an operation verification apparatus. It is a flowchart which shows the frame display position specific process of S606 of FIG. 41 in detail. It is a figure which shows the example of a display of the web content in a browser window.

Before describing the operation verification apparatus according to the present embodiment, the presupposed configuration will be described with reference technology 1, reference technology 2, reference technology 3, and reference technology 4.

(Reference technology 1)
FIG. 1A is a screen view of the portal site first screen 300.
In Reference Technique 1, an operation test for the portal site “XYZ!” Will be described. The portal site first screen 300 includes an edit box 302 and a search link 304. The search link 304 includes four links of “web”, “blog”, “image”, and “dictionary”. When the user inputs a character string into the edit box 302 and clicks one of the search links 304 with the mouse, the search for the input character string is started. For example, when the character string “liquid crystal” is input to the edit box 302 and the link “web” is clicked with the mouse, a website including the character string “liquid crystal” is searched. When the character string “LCD” is entered in the edit box 302 and the link “blog” is clicked with the mouse, the blog including the character string “LCD” among the blogs (Weblogs) established on this portal site. The page becomes the search target.
Here, it is assumed that the character string “test” is input to the edit box 302 and the link “dictionary” is clicked with the mouse.

FIG. 1B is a screen view of the portal site second screen 310.
When “test” is input on the portal site first screen 300 and “dictionary” is clicked with the mouse, a portal site second screen 310 shown in FIG. 1B is displayed. In the search result column 306, items related to the character string “test” are listed by the “dictionary” service provided by the portal site.
Here, it is assumed that the link “test” at the top of the search result column 306 is clicked with the mouse.

In the above process, the following three operations are performed.
A1. An operation of inputting the character string “test” in the edit box 302 of the portal site first screen 300.
A2. An operation of clicking the link “dictionary” in the search link 304 of the portal site first screen 300 with a mouse.
A3. An operation of clicking the link “test” in the search result column 306 of the portal site second screen 310 with a mouse.
Hereinafter, the operation process realized by the operations A1 to A3 is referred to as “dictionary test”.

FIG. 2 is a diagram showing the log 320 when the dictionary test is executed.
When a client terminal in which HP QTP is installed is operated to access the portal site and the operations A1 to A3 are executed, QTP generates a log 320 shown in FIG. In the log 320, operations A1 to A3 are recorded as code sentences in a predetermined format.
For example, the operation A1 is
Browser ("XYZ!"). Page ("XYZ!"). WebEdit ("p"). Set "Test"
It is expressed by the code sentence. This corresponds to a web site “XYZ!”, A web page named “XYZ!”, And an edit box named p (corresponding to the edit box 302 shown in FIGS. 1A and 1B). On the other hand, the character string “test” is input.
Similarly, the operation A2 is
Browser ("XYZ!"). Page ("XYZ!"). Link ("Dictionary"). Click
Operation A3 is
Browser ("XYZ!"). Page ("XYZ! -All dictionaries-Test"). Link ("Test"). Click
It is displayed with the code sentence. Reference technology 1 will be described based on the grammar of QTP, but the format of the code sentence is not limited to this.

QTP can reproduce and execute the operation expressed by the code sentence while interpreting the code sentence of the log 320. For this reason, once the log 320 is generated, the dictionary test with the same contents can be repeatedly executed. Further, if a part of the log 320 is rewritten, the dictionary test can be executed with contents different from those at the time of manual operation. For example,
Browser ("XYZ!"). Page ("XYZ!"). WebEdit ("p"). Set "Test"
Change the string "test" contained in to another string "taste"
Browser ("XYZ!"). Page ("XYZ!"). WebEdit ("p"). Set "Taste"
And re-input to QTP, it is possible to automatically execute a dictionary test of the same content in which only the input character string in the edit box 302 is changed.
However, in order to execute the operation test while rewriting the log 320, a certain degree of knowledge about the grammar of the log 320 is required. Further, when the log 320 is rewritten, a human error may occur.

FIG. 3 is a diagram showing a program 330 for executing the dictionary test.
Instead of executing the operation test using the log 320, a test program may be created by describing the content equivalent to the log 320 in a VB (Visual Basic) script or the like. A program 330 shown in FIG. 3 is a test program created with reference to the log 320 of FIG.
For example, the operation A1 is
WebEditSet (Browser ("XYZ!"). Page ("XYZ!"). WebEdit ("p"), DataTable ("p1, dtGlobalSheet")
It is expressed by the VB function. The specific logic of the WebEditSet function is as described in the middle of the figure.

The program 330 is different from the log 320 in that input data such as the character string “test” is not included in the program 330. Instead, the WebEditSet function receives input data from a file expressed as dtGlobalSheet (hereinafter referred to as “input data file”). That is, logic and input data are separated. By rewriting or replacing input data files, dictionary tests with the same logic can be executed based on various input data. Compared to the operation test by manual operation and the operation test by log 320, the operation test by program 330 has an advantage that it is easy to increase variations of input data.

However, the operation test by the program 330 has the following problems.
d1. There is a new effort to create a program.
d2. The content of the program changes depending on the program creator. For example, the WebEditSet function created by the programmer P1 corresponding to the operation A1 and the WebEditSet function created by the programmer P2 corresponding to the operation A1 do not necessarily have the same logic even though the function names are the same. In addition, bugs may be mixed in the program itself. For this reason, it is necessary to make efforts to ensure the reliability of the program 330 itself. The program 330 also requires logic that is not directly related to the operation test, such as exception processing, but not all programmers implement logic that considers even exception processing. For example, in the case of the WebEditSet function shown in FIG. 3, the logic is set up so that it can cope with the case where the input data is an empty character. Might do.
As a result, programmer skills are likely to affect the content and results of behavior tests.

FIG. 4 is a functional block diagram of the operation verification apparatus 100.
Each functional block shown in the block diagram of the present specification can be realized in hardware by an element or a mechanical device such as a CPU of a computer, and in software by a computer program or the like. Here, The functional block realized by those cooperation is drawn. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by a combination of hardware and software. For example, each functional block may be stored in a recording medium as a computer program, installed in a hard disk of an information processing device, read into a main memory as appropriate, and executed by a processor.
The operation verification apparatus 100 may be formed as dedicated hardware, or may be formed as a software module that cooperates with a user interface such as a web browser.
The operation verification apparatus 100 of the reference technique 1 will be described as a software module formed as an add-on of QTP.

The operation verification apparatus 100 includes an IF (interface) unit 110, a data processing unit 130, and a data holding unit 140.
The IF unit 110 is in charge of an interface with a user or QTP. The data processing unit 130 executes various data processing based on data acquired from the IF unit 110 and the data holding unit 140. The data processing unit 130 also serves as an interface between the IF unit 110 and the data holding unit 140. The data holding unit 140 is a storage area for holding various data.

IF unit 110:
The IF unit 110 includes an input unit 112 and an output unit 118. The input unit 112 is in charge of input processing from the user and QTP, and the output unit 118 is in charge of output processing for the user and QTP. The input unit 112 includes a log acquisition unit 114 and an input data acquisition unit 116. The log acquisition unit 114 acquires a log from QTP. The input data acquisition unit 116 acquires an input data file and acquires input data to be provided to the program from the input data file. Details of the input data file will be described in detail with reference to FIGS.

The output unit 118 includes a function setting screen display unit 120, an input setting screen display unit 122, a program screen display unit 124, and a case setting screen display unit 126. The function setting screen display unit 120 displays a function setting screen 210 shown in FIG. The input setting screen display unit 122 displays an input setting screen 230 shown in FIG. The program screen display unit 124 displays a program screen 220 shown in FIG. The case setting screen display unit 126 displays a case setting screen 240 shown in FIG.

Data processing unit 130:
The data processing unit 130 includes a program generation unit 132, a program execution unit 134, and a test case registration unit 136. The program generation unit 132 automatically generates an operation test program. The program execution unit 134 executes the generated program. In the reference technique 1, QTP is caused to execute a program. The test case registration unit 136 registers the execution set in the case file. The execution set and the case file will be described in detail with reference to FIG.

Data holding unit 140:
The data holding unit 140 includes a log holding unit 142, a program holding unit 144, an input data file holding unit 146, a case file holding unit 148, a function holding unit 150, and a result holding unit 152. The log holding unit 142 holds a log. The program holding unit 144 holds a program. The input data file holding unit 146 holds an input data file. The case file holding unit 148 holds a case file. The function holding unit 150 holds a function library for the functions registered in the operation test program. The result holding unit 152 holds the result of the operation test.

FIG. 5 is a schematic diagram from obtaining a log to executing an operation test in Reference Technology 1.
First, manually operate the device to be tested. In Reference Technology 1, the portal site “XYZ!” Is accessed by operating the web browser of the client terminal. QTP generates a log, and the log acquisition unit 114 acquires a log from QTP. A function is associated with each code sentence in advance. The function setting screen display unit 120 reads the function library of the function holding unit 150 and displays a list of functions corresponding to each code sentence included in the log (S1). In the function setting screen display unit 120, the user can add, change, and delete functions as appropriate. In this way, functions to be included in the program are specified. S1 will be described in detail later with reference to FIG.

Next, input data to be supplied to this program is set (S2). The setting contents are registered as an input data file. S2 will be described in detail later with reference to FIG. A program is automatically generated from the function group set in S1 (S3). S3 will be described later in detail with reference to FIG.

Next, a combination of a program and an input data file is registered as an “execution set” from a plurality of types of programs and a plurality of types of input data files. A plurality of execution sets can be registered (S4). The setting contents are registered as a case file. For example, when the program P1 is executed based on the input data file F2 and then the program P2 is executed based on the input data file F6,
Execution set 1: Program P1 + Input data file F2
Execution set 2: Program P2 + input data file F6
Is registered in the case file. S4 will be described in detail with reference to FIG.
Finally, the program is executed according to the case file (S5). In the case of the above example, the program execution unit 134 continuously executes the execution set 1 and the execution set 2. The program execution unit 134 registers the execution result of the program, in other words, the test result in the result holding unit 152.

FIG. 6 is a screen diagram of the log screen 200.
When the log acquisition unit 114 acquires the log, the output unit 118 displays the log on the log display area 202 in the log screen 200. The user may copy and paste the QTP log into the log display area 202. Alternatively, the output unit 118 may acquire a log (file) generated by QTP when the log screen 200 is displayed and display the log in the log display area 202. When the user clicks the execution button 204 with the mouse, a function setting screen 210 shown in FIG. 7 is displayed.

FIG. 7 is a screen diagram of the function setting screen 210.
The program generation unit 132 specifies a function for the code sentence included in the log, and displays the specified function in a list in the function column 214 of the function setting screen 210. The number column 212 indicates the execution order, the description column 216 indicates the outline of the function, the object column 218 indicates the operation target object, the parameter column 219 indicates the parameter name, and the input value column 217 indicates the input data.
A sign statement indicating the above-mentioned operation A1,
Browser ("XYZ!"). Page ("XYZ!"). WebEdit ("p"). Set "Test"
The WebEditSet function is associated with the function library in advance. When the program generating unit 132 reads the first code sentence of the log 320, the program generating unit 132 refers to the function library of the function holding unit 150 and identifies the WebEditSet function. Since the object to be operated by the operation A1 is the page “Browser (“ XYZ! ”). Page (“ XYZ! ”)”, The object column 218 displays “Browser (“ XYZ! ”). Page ( "XYZ!") "Is set. The WebEditSet function is a function that takes input data set in the edit box p (edit box 302) as an argument. Although it is possible to set input data to be set in the edit box p in the input value column 217, it is assumed that no input data is registered in the function setting screen 210. Input data to be set in the edit box p is set on the input setting screen 230 described in detail with reference to FIG.

Since the log 320 includes three code sentences corresponding to the operations A1 to A3, the program generation unit 132 specifies three functions. The user may add, delete, or change a function on the function setting screen 210. The fourth TextCheck function and the fifth WebScreenCapture function in FIG. 7 are functions additionally selected by the user.

The TextCheck function is a function that determines whether or not the character string to be assigned to the variable q is included in the displayed web page. The WebScreenCapture function is a function that captures a screen of the web page being displayed. The WebScreenCapture function is a convenient function for trail management of the results of operation tests. As described above, the function library includes not only functions corresponding to user operations such as the WebEditSet function but also functions for assisting and supporting the operation test such as the TestCheck function. The user can set logic to be included in the operation test simply by selecting a desired function from the function library.

When the test case setting button 215 is clicked, a case setting screen 240 of FIG. 10 is displayed. When the input data file creation button 213 is clicked, the input setting screen 230 of FIG. 9 is displayed. When the program creation button 211 is clicked, the program screen 220 of FIG. 8 is displayed.

The operation verification apparatus 100 of the reference technique 1 has the following merits.
m1. The logic to be included in the program can be selected simply by selecting a function based on GUI (Graphical User Interface). There is little effort to create a program.
m2. Since the functions included in the program are functions registered as a function library, the quality of the program hardly varies depending on the creator. Necessary logic such as exception handling is also incorporated in each function in advance.

FIG. 8 is a screen diagram of the program screen 220.
When the program creation button 211 is clicked with the mouse on the function setting screen 210, the program generation unit 132 generates a program, and the program screen display unit 124 displays the source code in the program display area 222 in the program screen 220. The program is held in the program holding unit 144.

FIG. 9 is a screen diagram of the input setting screen 230.
When the input data file creation button 213 is clicked on the function setting screen 210, the input setting screen display unit 122 displays the input setting screen 230. In the function setting screen 210, parameters p and q are defined. The number column 232 indicates the input order, the variable column 234 indicates the input data to the parameter p, and the variable column 236 indicates the input data to the parameter q. On the input setting screen 230, input data to be substituted for the parameters p and q is set. Each input data is held in the input data file holding unit 146 as an input data file.

For example, when an input data file in the format shown in FIG.
1. 1. A test for inputting a character string “test” into the edit box 302 to search the dictionary and determining whether the character string “test” is included in the screen showing the execution result. 2. A test for inputting a character string “test” into the edit box 302 and searching the dictionary to determine whether the character string “test” is included in the screen showing the execution result. Enter the character string “test case” in the edit box 302, search the dictionary, and determine whether the character string “test” is included in the screen showing the execution result. Multiple dictionary tests based on can be executed continuously.

FIG. 10 is a screen diagram of the case setting screen 240.
When the test case setting button 215 is clicked with the mouse on the function setting screen 210, the case setting screen display unit 126 displays the case setting screen 240. The number column 244 indicates the execution order, the execution set column 245 indicates the outline of the execution set, the program column 246 indicates the program to be executed, and the input data file column 247 indicates the input data file from which the input data is to be extracted.
Here, the website of the Internet securities will be described as an operation test target.

In the figure, 1. 1. Login 2. News check, 3. Stock sales; Bond sales, 5. Address change, 6. Six execution sets called logout are registered. First, QTP generates a “login” project from the program “login.vbs”. Here, the “project” is a directory including a group of QTP execution format files. QTP executes the process defined in login.vbs and login.xls with the “login” project as input. When the user clicks the execute button 248 with a mouse, the “login” project of the execution set “login” is first executed. When the first execution set “login” is completed, the “ncheck” project is executed as the execution set “news check”.

In this way, when the execution button 248 is clicked with the mouse, 1. Login 2. News check, 3. Stock sales; Bond sales, 5. Address change, 6. Six execution sets called logout are executed continuously. As a result, it is possible to automate an operation test based on a test scenario of “login, check news, instruct to sell hand shares and bonds, change address, and log out”.

The user can freely create a test scenario on the case setting screen 240. For example, instead of the execution set “Stock Sell”, you can register the execution set “Stock Purchase”, or register another input data file instead of “stock1.xls” as the input data file for the execution set “Stock Sell”. May be. The test scenario registered on the case setting screen 240 is recorded as a case file and held in the case file holding unit 148.
By changing the test scenario configuration and the input data file little by little, it becomes easier to expand the variation of the operation test.

The operation verification device 100 of the reference technique 1 has been described above.
According to the operation verification apparatus 100, a program and input data, that is, a process and a parameter can be separated. And the program for it can be automatically generated based on the setting of the GUI base. This makes it possible to easily generate a stable quality program while ensuring the diversity of logic included in the program. Moreover, since the contents of the operation test can be changed simply by changing the contents of the input data file, it is easy to expand the variations of the operation test.

In addition, by registering execution sets in case files and creating test scenarios, operation tests in various situations can be realized while combining existing programs. In addition to functions for operating the test target device such as the WebEditSet function, functions such as the WebScreenCapture function for supporting the operation test are prepared to further improve the efficiency and reliability of the operation test. It becomes easy to improve.

(Reference technology 2)
The reference technique 2 proposes an operation verification apparatus 100 obtained by improving the reference technique 1.
First, an outline of the operation verification apparatus 100 of the reference technique 2 will be described while comparing the operation verification apparatus 100 of the reference technique 1 and the operation verification apparatus 100 of the reference technique 2.

FIG. 11 shows an outline of the configuration of each of the operation verification apparatus 100 of the reference technique 1 and the operation verification apparatus 100 of the reference technique 2. As shown in FIG. 11A, the operation verification apparatus 100 of the reference technique 1 is used to improve the efficiency of the operation test of the QTP module 102 which is an execution engine of the operation test and the web server 400 using the QTP module 102. And a wrapper module 104. The wrapper module 104 includes the functional blocks shown in FIG.

In FIG. 11A, the user operates the web server 400, which is a target device for operation confirmation, via the QTP module 102 (S100). The wrapper module 104 obtains a log in which the operation content is recorded from the QTP module 102 and provides the function setting screen 210 to the user (S102). When performing the operation test, the user causes the wrapper module 104 to generate an operation test program that can be read by the QTP module, and causes the QTP module 102 to execute the program (S104).

On the other hand, as illustrated in FIG. 11B, the operation verification apparatus 100 of the reference technique 2 includes an integrated verification module 106 in which the function of the operation test execution engine and the function of the wrapper module 104 are integrated. When the user operates the web server 400 via the integrated verification module 106 (S200), the integrated verification module 106 sequentially detects the operation contents, sets the data of the function setting screen 210, and provides the function setting screen 210 to the user. (S202). When executing the operation test, the user causes the integrated verification module 106 to generate and execute an operation test program (S204).

FIG. 12 shows an outline of operations in each of the operation verification apparatus 100 of the reference technique 1 and the operation verification apparatus 100 of the reference technique 2. FIG. 12A shows an outline of the operation in the operation verification apparatus 100 of the reference technique 1. The user executes a data input operation on the web server 400 (S110), and the QTP module 102 outputs a log in which the operation content is recorded (S112). The user copies the log data to a predetermined input area of the QTP module 102 (S114). The wrapper module 104 sets and displays a function setting screen based on the log data set in the predetermined input area (S116), and the user adds the function / input data to be added / changed via the function setting screen. Set (S118).

The wrapper module 104 generates a QTP program for an operation test based on the data on the function setting screen (S120). The user copies the QTP program to a predetermined input area of the QTP module 102 (S122). On the case setting screen, the user sets an operation test scenario that specifies the QTP program and the input data file as a set (S124). Based on the scenario, the wrapper module 104 passes a set of the QTP program and the input data file to the QTP module 102, and executes an operation test for each set (S126).

FIG. 12B shows an outline of the operation in the operation verification apparatus 100 of the reference technique 2. The user executes a data input operation on the web server 400 (S210). The integrated verification module 106 sequentially detects the data input operation, records the function corresponding to the data input operation in the operation content file, and displays the recorded data of the operation content file on the function setting screen (S212). The user sets the function / input data to be added / changed in the operation content file via the function setting screen (S214). Then, on the case setting screen, the user sets an operation test scenario specifying the operation content file and the input data file as a set (S216). Based on the scenario, the integrated verification module 106 generates and executes an operation test program in which one or more sets in the scenario are aggregated (S218).

As described above, in the operation verification apparatus 100 of the reference technique 1, the QTP module 102 and the wrapper module 104 are separated. Therefore, in order to link the two modules, the user himself / herself has to execute a mediation operation. Further, the wrapper module 104 needs to call the QTP module 102 for each set in the scenario and execute the test program for each set.

On the other hand, in the operation verification apparatus 100 of the reference technique 2, the execution engine for the operation test and the wrapper for realizing its efficient use are seamlessly integrated without being separated. Therefore, compared with the operation verification apparatus 100 of the reference technique 1, the user's work required for linking the separated modules becomes unnecessary, and the user's work amount for the operation test is reduced. In addition, since an operation test program in which a plurality of sets in a scenario are aggregated is generated and a series of scenarios is executed by executing the program, a quick operation test is realized.

FIG. 13 is a block diagram showing a functional configuration of the operation verification apparatus 100 of the reference technique 2. The functional blocks shown in the figure are included in the integrated verification module 106 shown in FIG. Among the functional blocks of the reference technique 2, the functional blocks corresponding to the functional blocks of the reference technique 1 are denoted by the same reference numerals. Functional blocks with the same reference numerals as those of the reference technique 1 are described when the functions are different from the contents described in the reference technique 1, and description of similar functions is omitted.

The data holding unit 140 includes an operation content file holding unit 143, a program holding unit 144, an input data file holding unit 146, a case file holding unit 148, and a function holding unit 150. The operation content file holding unit 143 stores an operation content file in which a function to be set in the operation test program is recorded. The program holding unit 144 stores an operation test program generated according to the operation content file.

The input unit 112 of the IF unit 110 includes an operation detection unit 115 and a user setting reception unit 117. The operation detection unit 115 sequentially detects user data input operations on the web server 400. The user setting reception unit 117 detects setting information by the user for the function setting screen 210, the input setting screen 230, and the case setting screen 240.

The output unit 118 of the IF unit 110 includes a function setting screen display unit 120, an input setting screen display unit 122, and a case setting screen display unit 126. The function setting screen display unit 120 displays a function setting screen 210 that displays the contents of the operation content file recorded in the operation content file holding unit 143 on a predetermined display device.

The data processing unit 130 includes a program generation unit 132, a program execution unit 134, a test case registration unit 136, an operation recording unit 137, and an input data recording unit 138. The operation recording unit 137 refers to the function holding unit 150, specifies a function corresponding to the data input operation detected by the operation detection unit 115, and records the function in the operation content file. The operation recording unit 137 records user setting information for the function setting screen 210 in an operation content file. The input data recording unit 138 records input data input by the user via the input setting screen 230 in an input data file. The program generation unit 132 generates an operation test program according to the function recorded in the operation content file. The program execution unit 134 executes an operation test program as an operation test execution engine, and records the result in the result holding unit 152.

FIG. 14 is a schematic diagram from when the reference data 2 is detected until a user data input operation is detected and an operation test is executed. First, the user operates the web browser of the client terminal to access the web server 400, and executes various operations on the web server 400, typically data input operations on web pages provided by the web server 400. The operation detection unit 115 sequentially detects data input operations by the user, and the operation recording unit 137 reads the function library of the function holding unit 150 and sequentially records the functions corresponding to the data input operations in the operation content file (S220).

Next, the function setting screen display unit 120 reads the operation content file in the operation content file holding unit 143 and displays a function setting screen that displays a list of functions corresponding to the operation by the user. On the function setting screen, the user can add, change, and delete functions as appropriate, and can set input data to be statically set in the operation test program. In this way, a function to be included in the operation test program is specified. The function setting screen will be described later with reference to FIGS. 15 and 16.

Next, the input setting screen display unit 122 displays the input setting screen 230. The user sets input data to be dynamically supplied to the operation test program when the operation test is executed via the input setting screen 230. The input data set by the user is recorded in the input data file (S222). The input setting screen 230 will be described later with reference to FIG.

Next, the case setting screen display unit 126 displays the case setting screen 240. The user registers a combination of an operation content file and an input data file as an execution set from a plurality of types of operation content files and a plurality of types of input data files. The test case registration unit 136 records one or more execution sets in the case file (S224). For example, in the test case 1, after the setting content of the operation content file F01 is executed based on the input data file F12, the setting content of the operation content file F02 is executed based on the input data file F16.
Test case 1:
Execution set 1: operation content file F01 + input data file F12
Execution set 2: operation content file F02 + input data file F16
Is registered in the case file. The case setting screen 240 will be described later with reference to FIG.

Next, the program generation unit 132 generates an operation test program based on the case file and the operation content file (S226). Specifically, the functions sequentially recorded in the operation content file set in the execution set may be sequentially set in the program code of the operation test program. Further, when a plurality of execution sets are set in one test case, the program generation unit 132 collects and sets the functions recorded in the operation content file of each execution set in one operation test program.

In addition, when the input data itself is set for the function of the operation content file, the program generation unit 132 statically sets the input data in the operation test program. For example, when generating an operation test program, input data is set in advance as a function argument in the program code. If an input data file is specified for the function of the operation content file, the data of the input data file is set to be read when the program is executed.

Finally, the program execution unit 134 executes the operation test program to reproduce a series of operations executed by the user on the web server 400 (S228). For functions that require data in the input data file, the data of the input data file read when the operation test program is executed is passed as an argument. That is, the function is executed based on the input data dynamically acquired from the input data file.

FIG. 15 is a screen diagram of the function setting screen 210. This figure shows a function setting screen 210 that displays “search entry.xls” described later in FIG. The data file column name column 250 is an area in which the column name of the input data file in which the input data set in the parameter is recorded is designated. The user may describe the input data itself in the input value field 217, or instead, may specify the column name of the input data file in the data file column name field 250. When the column name of the input data file is specified in the data file column name field 250, when the operation test program is executed, one or more input data set in the column of the input data file is read and passed to the function. .

FIG. 16 is a screen diagram of the function setting screen 210 in the reference technique 2. In the input value field 217 of the “search part signature” parameter in FIG. 9, input data candidates are listed in a drop-down list 252. The input data candidates displayed in the drop-down list 252 are input data candidates displayed in the drop-down list of the web page when the user inputs data to the web page. The user can select input data from the drop-down list 252 and the burden on the user in setting the input data is reduced.

In order to set the drop-down list 252, the operation detection unit 115 acquires input data candidates displayed in the drop-down list of the web page when the user performs a data input operation on the web page. For example, a candidate for input data in the drop-down list is acquired from web page data acquired from the web server 400, for example, HTML data. The operation recording unit 137 records the input data candidates in the input value field 217 of the operation content file. When the input value field 217 is input, the function setting screen display unit 120 displays a list of input data candidates recorded in the operation content file in the form of a drop-down list.

FIG. 17 is a screen diagram of the input setting screen 230 in the reference technique 2. FIG. 17A shows an input setting screen 230 that displays the input data file “search entry data.xls” in which the column name is specified in the function setting screen 210 of FIGS. 15 and 16. 15 and FIG. 16, the “name kanji” column and the “name kana” column in FIG. 17A are designated. FIG. 17B shows an input setting screen 230 that displays another input data file “login data 2.xls”. The input data file shown in the figure will be described later with reference to FIG.

FIG. 18 is a screen diagram of the case setting screen 240 in the reference technique 2. A basic execution order of each execution set is set in the execution number column 253, and test case identification information is set in the test case ID column 254. In the group ID column 256, identification information indicating a group of execution set repetition processing is set, and in the line number column 258, a reference range of the input data file in the operation test is set. The operation content file column 260 is set with an operation content file for each execution set, and the input data file column 262 is set with an input data file for each execution set. The setting content on the case setting screen 240 is stored as a case file in the case file holding unit 148.

When the test execution button 264 is pressed on the case setting screen 240, the program generation unit 132 generates an operation test program according to the case file and the operation content file. Specifically, based on one or more operation content files in which the same test case ID is set in the case file, one operation test program in which the function recorded in each operation content file is set in the program code Generate. For example, the program generation unit 132 performs one operation test program for executing a registered case and one for executing a search case in accordance with a case file in which the contents of the case setting screen 240 of FIG. 18 are recorded. An operation test program is generated. Note that “one operation test program” means a program executed as one execution unit, in other words, a program executed at one execution opportunity, and the number of physical programs is not limited.

When the program execution unit 134 executes one function recorded in each operation content file, the program execution unit 134 associates the function with the function in the operation content file among the data of the input data file associated with the operation content file. Pass column data to the function.

In addition, the program execution unit 134 handles a plurality of execution sets assigned the same group ID in the same test case as the same group that should execute the same repetition process. Specifically, each execution set with the same group ID is repeatedly executed, and each time it is repeated, the function recorded in the operation content file of each execution set is recorded in the input data file. Pass a certain number of records. In the reference technique 2, one record recorded in the input data file is passed to the function every time it is repeated. Note that the program generation unit 132 may generate an operation test program in which a repetition instruction for executing the above-described processing is set in the program code.

Further, for the execution set in which the line number is designated, the program execution unit 134 passes the input data specified by the line number in the input data file to the function recorded in the operation content file. Therefore, when the repetition process by the group ID is designated and the line number is further designated, the repetition is executed by the designated number of line numbers. For example, in the registration case of FIG. 18, there is no group setting across execution sets, and processing for one input data is executed in each execution set. On the other hand, in the search case of FIG. 18, the process for one input data in each execution set is repeated three times.

Specifically, in the first iteration, login2. 1. Login data for the function recorded in xls Data “t_yamada” on the 35th line of xls is passed. And search entry. Search entry data for the function recorded in xls, specifically, the function No. 1 in FIG. The data “Taro Yamada” in the first line of the xls and full name kanji is passed. For the function No. 2 in FIG. The data “Yamada Taro” in the first line of the xls and the name Kana is passed.
In the second iteration, login 2. 1. Login data for the function recorded in xls The data “h_yamada” on the 36th line of xls is passed. And search entry. Search entry data for the function No. 1 in FIG. Data “Kana Yamada” in the second line of xls and full name kanji is passed. In addition, the search entry data for the function No. 2 in FIG. The data “Yamada Hanako” in the second line of xls and the name Kana is passed.
Similarly, in the third iteration, the input data in the incremented line number is passed to each function.

The operation verification apparatus 100 of the reference technique 2 also has the effects described in the reference technique 1. Specifically, since separation of processing and parameters and GUI-based setting are realized, it is possible to easily generate a stable quality program while ensuring the diversity of logic included in the operation test program. Moreover, it becomes easy to expand the variation of the operation test by setting the input data file. Furthermore, operation tests in various scenarios can be realized by setting case files.

In the operation verification apparatus 100 of Reference Technology 2, the execution engine function that executes the operation test program and the wrapper function that realizes efficient use of the execution engine are seamlessly linked without requiring user intervention. To do. Thereby, the burden on the user for executing the operation test can be reduced. Unlike the case of the operation verification apparatus 100 of the reference technique 1, the user does not need to be aware of the environment for program execution such as a QTP project. That is, an operation content file and an input data file in a format that can be easily understood and operated by the user may be set. As a result, the user can easily perform the operation test.

In the operation verification apparatus 100 of the reference technique 2, one operation test program in which a plurality of execution sets of the same test case are aggregated is generated, and the one operation test program is executed. Compared with execution for each execution set in the operation verification apparatus 100 of the reference technology 1, that is, for each project of QTP generated based on the execution set, overhead when executing a series of operation tests in one test case is reduced. A quick operation test can be realized.

In the operation verification apparatus 100 of the reference technique 2, it is possible to set a repeated group over a plurality of execution sets. As a result, the execution set can be flexibly divided, and an operation test scenario can be flexibly set. Furthermore, the reference range of the input data file can be set. Thereby, even if the input data file is the same, the reference range can be flexibly changed according to the scenario, and various variations of the operation test can be easily realized.

In the reference technique 2 described above, the drop-down list 252 is shown as an example of supporting the value setting on the function setting screen 210. In the modified example, candidates for input data displayed so as to be selectable by a radio button, a check box, a list box, etc. on the web page are displayed so as to be selectable on the function setting screen 210 in a drop-down list 252 or other formats. Also good. That is, input data candidates that are displayed so as to be selectable in various formats for the user when performing a data input operation on the operation confirmation target device are displayed so as to be selectable when setting values on the function setting screen 210. Also good.

In the reference technique 1 described above, the operation verification apparatus 100 creates an operation test program on the condition that a user operation on the web server 400 is accepted by the QTP and a log is generated. In the reference technique 2 described above, the operation verification apparatus 100 detects a user operation on the web server 400 and creates an operation test program. As a modification, an operation test program may be generated when the operation verification apparatus 100 acquires an HTML file from the web server 400. In this case, the operation verification apparatus 100 searches for a form tag included in the HTML file and specifies what input interface is included in the HTML file. If a function for inputting data to each input interface is selected and these functions are arranged in various orders, an operation test program can be generated without an explicit web operation by the user.

(Reference technology 3)
First, an outline of the reference technique 3 will be described.
Currently, many web browsers are open to the public, and various types of web browsers are also used in user PCs. Therefore, it is desirable that the operation verification apparatus can support a web page operation test using a plurality of types of web browsers. In this case, the operation verification apparatus needs to operate the web browser according to the specifications of the web browser, and the architecture is roughly divided into the following two. Note that the web page that is the target of the operation test is also referred to as a “test target page” below.

Method 1. External operation:
By calling an API (Application Programming Interface) published by the web browser to the outside, that is, an interface that accepts an input operation to the test target page being displayed on the web browser (hereinafter also referred to as “operation IF”), Operate from. Specifically, the operation IF of the reference technique 3 is a function for inputting data to the test target page, executing the form of the test target page, and transmitting a GET request or a POST request to the web server. is there.

Method 2. Operation from inside:
A script document (hereinafter also referred to as “control code”) that accepts an input operation to the test target page and causes the web browser to execute the input operation is added to the program code (ie, HTML document) of the test target page. . Then, the web browser is operated from the inside by calling the control code. In other words, the web browser is operated by a script document embedded in advance in a web page displayed on the web browser.

The control code of Reference Technology 3 is a program code in which a function including an interface function that accepts an input operation to the test target page from the outside is described. Further, the program code describes a function including a function for updating the display contents of the test target page by operating the DOM tree of the test target page and a function for transmitting a GET request or a POST request to the web server. The control code may be described in a predetermined script language (for example, JAVA script (“JAVA” is a registered trademark)).

FIG. 19 shows the characteristics of the two operation methods for the web browser. The figure compares the operation method from the outside with the operation method from the inside. Regarding the reliability of the operation test itself, which is assumed to be the most important, it can be said that the operation method from the outside is superior because the operation method from the inside rewrites the test target page. On the other hand, regarding compatibility and versatility, the operation method from the outside depends on the operation IF in the web browser, so the operation method from the inside is superior.

Therefore, in the reference technique 3, when the operation IF of the web browser used for the operation test is disclosed, the web browser is operated from the outside, and when the operation IF is not disclosed, the operation verification device operates the web browser from the inside. Propose. As a result, it is possible to support operation tests using various types of web browsers while ensuring reliability of the operation tests at best effort.

FIG. 20 schematically shows a calling relationship between functions for two operation methods for the web browser. The web server 400 is an information processing apparatus that provides a test target page. Browser A 402a, browser B 402b, browser C 402c, and browser D 402d, which are collectively referred to as browser 402, are web browsers that function as HTTP user agents, and execute display processing / operation processing of a test target page. Note that the browser A 402a and the browser B 402b shown in the figure disclose the operation IF to the outside, and the browser C 402c and the browser D 402d make the operation IF private.

The browser A engine 404a and the browser B engine 404b, which are collectively referred to as the individual browser engine 404, are program codes that call the operation IFs of the browser A 402a and the browser B 402b, respectively. The common IF 406 is an interface for calling the operation IF of each of the plurality of types of browsers 402, and is a program code having a common format interface that does not depend on the operation IFs that are different for each of the plurality of types of browsers 402. The browser A engine driver 408a and the browser B engine driver 408b, which are collectively referred to as the individual browser engine driver 408, convert the data in the common IF 406 format independent of the type of the browser 402 into data in the operation IF format different for each browser 402. By performing the conversion, the operation test program and the individual browser engine 404 are bridged.

The proxy server 410 functions as a reverse proxy that acquires a test target page from a web server as a proxy of the browser 402. After adding the control code to the test target page, the proxy server 410 provides the test target page to the browser C402c and the browser D402d. The internal operation engine 412 calls the control code added to the test target page via the proxy server 410. The internal operation engine driver 414 bridges the operation test program and the internal operation engine 412 by converting data in the common IF 406 format into data in a predetermined format for calling a control code.

The operation of FIG. 20 will be described. When the web browser that displays the test target page is the browser A 402a or the browser B 402b, the operation test program calls the common IF 406. Accordingly, the web browser operation IF is called via the individual browser engine driver 408 and the individual browser engine 404, whereby the web browser is operated, and the input operation to the test target page is realized.

On the other hand, when the web browser displaying the test target page is the browser C402c or the browser D402d, the proxy server 410 adds a control code to the test target page and causes the test browser to display the test target page. In this case as well, the operation test program calls the common IF 406, and the control code added to the test target page is called through the internal operation engine driver 414, the internal operation engine 412, and the proxy server 410. Thereby, the web browser is operated without going through the operation IF of the web browser, and the input operation to the test target page is realized.

20, the difference in the web browser interface is absorbed by the individual browser engine driver 408 and the internal operation engine driver 414. Thus, the operation test program can always operate the test execution browser by calling the common IF 406 regardless of the type of web browser (hereinafter also referred to as “test execution browser”) used in the operation test of the web page. In other words, an operation test using a plurality of types of web browsers can be automatically executed by the same operation test program.

FIG. 21 is a block diagram showing a functional configuration of the operation verification apparatus of Reference Technique 3. Among the functional blocks in the figure, the functional blocks corresponding to the functional blocks of the operation verification apparatus 100 shown in the reference techniques 1 and 2 are denoted by the same reference numerals. Functional blocks with the same reference numerals are described when the functions are different from those already described in Reference Techniques 1 and 2, and similar functions are already described, and thus description thereof is omitted as appropriate.

The operation verification apparatus 100 further includes a browser processing unit 160 in addition to the functional blocks shown in FIG. The browser processing unit 160 corresponds to the browser 402 in FIG. 20 and functions as a web browser. Of course, a single operation verification apparatus 100 may include a plurality of types of browser processing units 160.

The program execution unit 134 selects an external operation method or an internal operation method according to the type of the test execution browser, in other words, the mode of the browser processing unit 160 when executing the operation test program. The unit 160 is operated. FIG. 22 is a block diagram showing details of the program execution unit 134 of FIG. The program execution unit 134 includes a test subject selection unit 162, an external operation execution unit 164, a proxy processing unit 166, an internal operation execution unit 168, and a witness setting unit 170.

The test subject selection unit 162 acquires information indicating the type of the test execution browser, in other words, information indicating the disclosure status of the operation IF in the browser processing unit 160 from a predetermined setting file or an operation test program, and according to the information Determine the execution subject of the operation test. For example, with reference to a table in which various web browsers are associated with the disclosure status of each operation IF, the disclosure status of the operation IF in the test execution browser may be specified.

When the browser processing unit 160 exposes the operation IF to the outside, the test subject selection unit 162 selects an external operation execution unit 164 described later as an execution subject of the operation test. On the other hand, when the browser processing unit 160 makes the operation IF private to the outside, a proxy processing unit 166 and an internal operation execution unit 168 described later are selected as execution subjects of the operation test.

The external operation execution unit 164 corresponds to the individual browser engine 404 in FIG. 20 and detects the call to the common IF 406 by the operation test program when the operation test program is executed. Then, the browser processing unit 160 is operated by calling an operation IF disclosed by the browser processing unit 160 to execute an input operation on the test target page.

The proxy processing unit 166 corresponds to the proxy server 410 in FIG. 20, obtains the test target page requested from the browser processing unit 160 by proxy from the web server 400, and inserts a control code into the HTML code of the test target page. Then, the HTML code of the modified test target page is sent to the browser processing unit 160 for display. The proxy processing unit 166 receives a control code call from an internal operation execution unit 168, which will be described later, and calls the control code added to the test target page. Communication between the proxy processing unit 166 and the test target page may be implemented as asynchronous communication using XMLHttpRequest.

The internal operation execution unit 168 corresponds to the internal operation engine 412 in FIG. 20, and detects the call to the common IF 406 by the operation test program when the operation test program is executed. Then, by calling the control code added to the test target page via the proxy processing unit 166, the browser processing unit 160 is operated to perform an input operation on the test target page. Note that the proxy processing unit 166 and the internal operation execution unit 168 may be implemented using a Selenium (http://seleniumhq.org/) library published as a test automation tool.

In response to a request from the external operation execution unit 164 or the internal operation execution unit 168, the testimony object setting unit 170 sets data (hereinafter also referred to as “testimony object”) that proves the result of the operation test. Is stored in the result holding unit 152. As a proof, a screen snap of the test target page displayed on the display by the browser processing unit 160 may be acquired, and data transmitted and received by HTTP communication with the web server 400 may be acquired. Note that the screen snap acquisition function may be provided by the operating system (hereinafter also referred to as “OS”) of the operation verification apparatus 100, and the HTTP communication data acquisition function is a packet installed in the operation verification apparatus 100. It may be provided by capture software.

The operation of the above configuration will be described below.
FIG. 23 is a flowchart showing the operation of the operation verification apparatus 100. This figure shows an operation when an operation test program is executed in the operation verification apparatus 100. For example, it shows details of S218 in FIG. 12, and also shows details of S228 in FIG.

At the start of the operation test for the test target page, the test subject selection unit 162 selects an operation test execution engine according to the disclosure status of the operation IF in the browser processing unit 160 (S300). The browser processing unit 160 requests the web server 400 to provide a test target page (S302). When the operation IF is not disclosed in the browser processing unit 160 (N in S304), the proxy processing unit 166 obtains the test target page by proxy, adds the control code to the HTML code, and then transmits the test target page to the browser. The data is sent to the processing unit 160 (S306). If the operation IF is disclosed (Y in S304), S306 is skipped. The browser processing unit 160 displays the test target page on a predetermined display (S308). The proof object setting unit 170 sets a proof object of the operation test, such as obtaining a screen snap of the test target page, and stores it in the result holding unit 152 (S310).

If all operations to be executed in the operation test have been executed (Y in S312), the flow of this figure is terminated. When an unexecuted operation remains (N in S312) and the operation IF is disclosed in the browser processing unit 160 (Y in S314), the external operation execution unit 164 applies the test to the test target page by the operation test program. The input operation is executed by calling the operation IF (S316). If the operation IF is not disclosed (N in S314), the internal operation executing unit 168 calls an input operation to the test target page by the operation test program by calling a control code added to the test target page. Execute (S318). When provision of a new web page is requested by an operation on the test target page, for example, when a screen transition occurs due to a GET request, a POST request, or the like (Y in S320), the process returns to S304. If provision of a new web page is not requested (N in S320), the process returns to S310.

The operation verification apparatus 100 of the reference technique 3 is obtained by adding a configuration for supporting a web page operation test in a plurality of types of web browsers on the premise of the configuration of the operation verification apparatus 100 described in the reference techniques 1 and 2. is there. Therefore, the configuration of the operation verification apparatus 100 of the reference technique 3 has the effects described in the reference techniques 1 and 2 as they are, and further has the following effects.

The operation verification apparatus 100 can execute an operation test of web pages in a plurality of types of web browsers regardless of whether or not the operation IF is disclosed in the web browser by implementing an internal operation method as the web browser operation method. . The operation verification apparatus 100 also implements an external operation method as a web browser operation method. When the web browser publishes an operation IF, an operation test is performed by an external operation method. It becomes easy to ensure the reliability of the operation test.

The operation verification device 100 of the reference technique 3 has been described above.
In the reference technique 3, the proxy processing unit 166 corresponding to the proxy server 410 is included in the operation verification apparatus 100. In the modification, the proxy processing unit 166 may be provided in another information processing apparatus different from the operation verification apparatus 100.

As described above, a plurality of types of web browsers may be installed in the single operation verification device 100, and the operation verification device 100 may include a plurality of browser processing units 160 corresponding to the plurality of types of web browsers. Good. In this case, the program execution unit 134 may execute an operation test on the test target page in each web browser. That is, the processing shown in FIG. 23 may be executed in parallel for each web browser. Since an operation test in a plurality of types of web browsers can be performed in the single operation verification apparatus 100, an operation test can be realized efficiently and at low cost.

In Reference Technology 3, either the internal operation method or the external operation method was selected for each type of web browser. In a variant, both formulas may be combined in an operational test with a single web browser. For example, among operations on the test target page, an operation that can be executed using the published operation IF applies an external operation method, that is, the external operation execution unit 164 calls the operation IF. On the other hand, the operation method from the inside is applied to an operation that cannot be executed by the published operation IF, that is, the internal operation execution unit 168 calls a control code. Since script code written in a web page generally allows a wide range of operations, constraints from external operation methods that occur depending on the operation IF specifications are complemented by internal operation methods. can do.

(Reference technology 4)
The reference technique 4 is related to the reference technique 3 and proposes an operation verification apparatus 100 that supports a web page operation test by various types of web browsers. In the following description, “Windows”, “Internet Explorer”, “Firefox”, “Mac”, “Mac OS”, “Safari” (Safari) ) "," JAVA ", etc. are trademarks or registered trademarks of the respective companies.

FIG. 24 schematically shows a call relationship between functions in the web page operation test. The operation verification engine 500 in FIG. 5 shows a core function for an operation test of a web page in the operation verification apparatus 100. A Windows environment 540 indicates an environment in the Windows OS, and a Mac environment 560 indicates an environment in the MacOS. Note that the operation verification apparatus 100 in which Windows is installed in the OS may be configured by a combination of the operation verification engine 500 and the Windows environment 540. Further, the operation verification apparatus 100 in which MacOS is installed in the OS may be configured by a combination of the operation verification engine 500 and the Mac environment 560.

The IE (Internet Explorer) browser 542 in FIG. 24 discloses COM550, which is an operation IF that can be called from an external application. Therefore, the operation method from the outside in the reference technique 3 is adopted for the operation of the web page displayed on the IE browser 542. On the other hand, the FF (Firefox) browser 546 and the Safari browser 562 do not disclose an operation IF that can be called from an external application. Therefore, the operation method from the inside in the reference technique 3 is adopted for the operation on the web page displayed on these browsers.

FIG. 24 shows a selenium module 530 for an operation method from the inside. The selenium module 530 is a server program of Selenium (http://seleniumhq.org/) that corresponds to the internal operation engine 412 and the proxy server 410 of FIG. 20 and is published as a test automation tool. The selenium module 530 assigns a unique identifier to a web browser window (hereinafter also referred to as “browser window”) that displays a web page. Then, by specifying the identifier from the client, the browser window (in other words, the web page) to be operated is uniquely identified.

The common IF 406 corresponds to the common IF 406 in FIG. The IE driver 502 corresponds to the individual browser engine driver 408 of FIG. The IE driver 502 functions as a bridge between an operation test program (hereinafter also referred to as “test program”) and the IE browser 542 in a web page operation test using the IE browser 542. The IE driver 502 includes an IE window manager 504, an IE browser operation unit 506, and an IE dialog operation unit 508.

The IE window manager 504 has an identifier assigned by the user in the test program (hereinafter also referred to as “user assigned ID”) and an identifier assigned by the OS (hereinafter “ The correspondence relationship with the “OS allocation ID”) is managed.

The IE browser operation unit 506 specifies an OS allocation ID corresponding to the user allocation ID specified by the test program for a browser window (hereinafter also referred to as “operation target window”) on which a web page should be operated. At the same time, the COM 550 function corresponding to the operation content specified by the test program is specified. Then, by calling the function of the COM 550 by specifying the OS allocation ID, the IE browser 542 is operated from the outside, and the IE browser 542 is caused to execute the web page operation in the operation target window. Depending on the operation of the web page, the IE browser 542 may display a dialog box by directly using the resources of the Windows OS. Hereinafter, this dialog box is also referred to as an “OS dialog”.

The IE dialog operation unit 508 operates the OS dialog 544 displayed by the IE browser 542. Specifically, when the OS dialog 544 is detected, an operation for the OS dialog 544 is executed by calling a Win32 API 552 function corresponding to the dialog operation specified by the test program. Note that typical operation contents for the OS dialog 544 include input of a character string, selection of a display item, and pressing of a button. The same applies to the operation contents for the FF dialog 548 and the Safari dialog 564 described later.

The FF driver 510 corresponds to the internal operation engine driver 414 in FIG. 20 and functions as a bridge between the test program and the FF browser 546 in the web page operation test using the FF browser 546. The FF driver 510 includes an FF window manager 512, an FF browser operation unit 514, and an FF dialog operation unit 516.

The FF window manager 512 assigns an identifier (hereinafter also referred to as “selenium assignment ID”) assigned by the selenium module 530 to the browser window displaying the web page, and a user assignment ID and an OS assignment ID for the window. Manage the correspondence of.

The FF browser operation unit 514 specifies a selenium assignment ID corresponding to the user assignment ID specified by the test program for the operation target window. At the same time, the function of the selenium module 530 corresponding to the operation content specified by the test program is specified. Then, the function of the selenium module 530 is called by specifying the selenium allocation ID. In response to this, the selenium module 530 specifies an operation target window based on the selenium assignment ID, and calls a control code corresponding to the called function among the control codes of the web page displayed in the operation target window. As a result, a specific JAVA script is operated on the operation target web page, and the FF browser 546 is operated from the inside, and the operation of the web page specified by the test program is realized.

The FF browser operation unit 514 suppresses calling of the selenium module 530 when a predetermined operation to be operated without going through the selenium module 530, typically, a browser window closing process is designated by the test program. Instead, similar to the IE dialog operation unit 508, the operation is executed by calling the Win32 API 552.

Depending on the operation of the web page, the FF browser 546 may display the OS dialog 544 or a dialog box (hereinafter, also referred to as “FF dialog”) using a unique resource of the FF browser 546. is there. Typically, the OS dialog 544 is displayed in cooperation with the OS, such as saving a file, and the FF dialog 548 is displayed as an alert dialog by JAVA script.

The FF dialog operation unit 516 operates the OS dialog 544 and the FF dialog 548. Specifically, when the display of the OS dialog 544 is detected, an operation for the OS dialog 544 is executed by calling a Win32 API 552 function corresponding to the dialog operation specified by the test program. When the display of the FF dialog 548 is detected, an operation for the FF dialog 548 is executed by calling the FF plug-in 554 which is an extension function set in advance in the FF browser 546 and instructing the dialog operation. Details of this will be described later with reference to FIG.

The Safari driver 520 corresponds to the internal operation engine driver 414 in FIG. 20 and functions as a bridge between the test program and the Safari browser 562 in the web page operation test using the Safari browser 562. The Safari driver 520 includes a Safari window manager 522, a Safari browser operation unit 524, and a Safari dialog operation unit 526. The Safari window manager 522 manages the correspondence between the user allocation ID, the OS allocation ID, and the selenium allocation ID for the window in which the web page is displayed. In MacOS, it is the Safari browser 562 that assigns an ID to a browser window, but for simplicity of description, the name is unified with the OS assignment ID.

The Safari browser operation unit 524 specifies a selenium allocation ID corresponding to the user allocation ID specified by the test program for the operation target window. At the same time, the function of the selenium module 530 corresponding to the operation content specified by the test program is specified. Then, the function of the selenium module 530 is called by specifying the selenium allocation ID. As a result, similar to the FF browser operation unit 514, an operation on the operation target web page is executed via the selenium module 530.

The Safari browser operation unit 524 suppresses calling of the selenium module 530 when a predetermined operation to be operated without going through the selenium module 530, typically, a window closing process is designated by the test program. Instead, the operation is executed by calling an API provided by MacOS. Specifically, an accessibility API 566 which is an API group provided for use by an application for a disabled person is called. Details of this will be described later with reference to FIG.

Depending on the operation of the web page, the Safari browser 562 may display a dialog (hereinafter also referred to as “Safari dialog”). When the Safari dialog operation unit 526 detects the display of the Safari dialog 564, the Safari dialog operation unit 526 calls the MacOS API corresponding to the dialog operation specified by the test program to execute the operation on the Safari dialog 564. Also at this time, the above-described accessibility API 566 is called. Details of this will be described later with reference to FIG.

The above FF window manager 512 and Safari window manager 522 are collectively referred to as a selenium window manager 532. FIG. 25 shows an example of data stored in the selenium window manager 532. As shown in the figure, the selenium window manager 532 holds the user assignment ID, selenium assignment ID, and OS assignment ID assigned to each browser window in the web page operation test in association with each other. The operation of the IE browser 542 is not performed through the selenium module 530. Therefore, the IE window manager 504 holds only the user allocation ID and the OS allocation ID in association with each other.

The FF browser operation unit 514 and the Safari browser operation unit 524 are collectively referred to as a selenium browser operation unit 534 that operates a web browser via the selenium module 530.

FIG. 26 shows an operation mechanism for the FF dialog 548 shown in FIG. In the selenium module 530, when the FF browser 546 is activated, a window handler 572 implemented by a JAVA script is preset in the FF profile 570 read by the FF browser 546. When the FF browser 546 opens a new browser window, the function of the window handler 572 is additionally incorporated by reading the window handler 572 into the storage area of the FF plug-in 554.

When the window handler 572 is incorporated as the FF plug-in 554, the listener object for detecting window generation by the FF browser 546 is registered in the window watcher 582 via the XPConnect 578 and the XPCOM 580. When the window handler 572 detects that the FF dialog 548 has been generated via the registered listener object, the window handler 572 performs socket communication with the socket server 574 of the FF dialog operation unit 516 and notifies that the FF dialog 548 has been generated. To do. The notification information includes a display character string in the FF dialog 548 and display mode information indicating a display object such as a button.

When notified that the FF dialog 548 has been generated, the FF dialog operation unit 516 is a function corresponding to the dialog operation specified by the test program, and notifies the window handler 572 of the identification information of the XPCOM 580 function. . The window handler 572 operates the FF dialog 548 by calling the XPCOM 580 function designated by the FF dialog operation unit 516. Note that the FF dialog operation unit 516 may appropriately adjust the operation content for the FF dialog 548 in accordance with the display mode of the FF dialog 548. For example, for the button pressing operation specified by the test program, the ID of the button to be pressed may be added and the operation instruction may be notified to the window handler 572.

As described above, the FF browser 546 displays the FF dialog 548 as an alert dialog of JAVA script. Since the FF dialog 548 is displayed using resources unique to the FF browser 546, the contents of the dialog cannot be determined from the OS side. For example, display character strings and display objects such as buttons in the FF dialog 548 cannot be determined. For this reason, an operation via the Win32 API 552 cannot be performed like an operation on the OS dialog 544. For example, the pressing operation on the button displayed in the FF dialog 548 cannot be performed.

Therefore, in the reference technique 4, by adding the monitoring function of the FF dialog 548 display in the FF browser 546 as a plug-in, the display contents of the FF dialog 548 can be acquired, and an operation for the FF dialog 548 is realized.

Since the FF browser operation unit 514 operates the FF browser 546 via the selenium module 530, a function for dialog operation (for example, a program code similar to the window handler 572) is set in the program code of the web page. You can also However, when the FF dialog 548 is a modal dialog (an application modal dialog or a system modal dialog), the dialog cannot be operated from the code of the web page of the display source. On the other hand, the window handler 572 incorporated as the FF plug-in 554 can continue the operation to the modal dialog. From this viewpoint, it can be said that the window handler 572 is desirably set as an extended function of the FF browser 546.

FIG. 27 shows an operation mechanism for the Safari dialog 564 shown in FIG. An Apple script 584 in the figure is a script for calling an Apple event, which is an inter-application communication technique in MacOS. A system event 586 is an application provided by MacOS, and implements the calling of the accessibility API 566 by the Apple script 584 through the Apple script 584 and the accessibility API 566.

FIG. 27 shows MacOS in four layers, and an upper layer provides an API having a higher abstraction level than a lower layer, in other words, provides a high level API. As shown in the figure, the accessibility API 566 belongs to the highest hierarchy “user experience”. As described above, the accessibility API 566 is an API group related to a UI automation function for ensuring usability for persons with disabilities. The operation contents to be automated include various user interface (screen) operations such as opening a window and pressing a button.

The Safari dialog operation unit 526 specifies the function of the accessibility API 566 corresponding to the dialog operation specified by the test program, and transmits the Apple script 584 specifying the function to the system event 586. Thus, a dialog operation function prepared as the accessibility API 566 is called through the system event 586.

Note that the operation on the Safari dialog 564 can be realized using a lower level API group in MacOS. For example, the Safari dialog operation unit 526 can execute the operation on the Safari dialog 564 by calling the “Quartz” API of the graphics and media hierarchy. However, since the accessibility API 566 provides a high-level function corresponding to the UI operation by the user, the accessibility API 566 can be used to reduce the code amount of the Safari dialog operation unit 526 and to improve the maintainability thereof. Can do.

Although not shown in FIG. 27, the Safari browser operation unit 524 calls the accessibility API 566 for a predetermined operation to be performed without going through the selenium module 530 among the operations on the web page, as in the Safari dialog operation unit 526. This causes the MacOS or Safari browser 562 to execute the operation.

FIG. 28 is a block diagram illustrating a functional configuration of the operation verification apparatus 100 of the reference technique 4. Among the functional blocks in the figure, functional blocks that are the same as or correspond to the functional blocks of the operation verification apparatus 100 shown in the reference techniques 1 to 3 are denoted by the same reference numerals. Functional blocks with the same reference numerals are described when their functions are different from those already described in Reference Techniques 1 to 3, and descriptions of similar functions are omitted as appropriate.

The operation verification apparatus 100 further includes a window ID holding unit 154 and an ID acquisition unit 139. The window ID holding unit 154 and the ID acquisition unit 139 correspond to the IE window manager 504 and the selenium window manager 532 in FIG. That is, the window ID holding unit 154 holds the correspondence relationship between the user assignment ID, selenium assignment ID, and OS assignment ID assigned to the browser window that displays the web page.

The ID acquisition unit 139 acquires an OS allocation ID for the browser window from the OS (Windows OS or MacOS). In the case of Windows OS, the ID acquisition unit 139 acquires a window handle obtained by calling Win32 API. For example, the HWND value returned by the function “FindWindow” is acquired as a long integer and stored in the OS allocation ID field of the window ID holding unit 154. Of the multiple types of HWND values returned by the function “FindWindow”, those not stored in the window ID holding unit 154 may be added as new records in the window ID holding unit 154.

In the case of MacOS, the ID acquisition unit 139 acquires a long value obtained from the Safari browser 562 using the Apple script, and stores it in the OS allocation ID field of the window ID holding unit 154. For example, the following script:
tell application "Safari"
return id of windows
end tell
To obtain a list of long values assigned by the Safari browser 562 to each browser window.

Also, the ID acquisition unit 139 acquires “seleniumWindowName” which is a window attribute value assigned to each browser window by the selenium module 530 (Selenium-CORE). Then, the window attribute value is stored as a selenium allocation ID in the selenium allocation ID field of the window ID holding unit 154. For example, a list of window attribute values managed by the selenium module 530 is acquired by calling the function “getAttributeFromAllWindows (“ seleniumwindowName ”)” of the selenium module 530.

Also, the ID acquisition unit 139 acquires a user allocation ID allocated to the browser window by an ID allocation unit 174 and an ID allocation unit 180, which will be described later, and stores them in the user allocation ID field of the window ID holding unit 154. The ID acquisition unit 139 sequentially stores the user allocation ID, the selenium allocation ID, and the OS allocation ID in the window ID holding unit 154 in the allocation order on the time axis (the acquisition order in the ID acquisition unit 139). Also good. As a result, the ID (for example, selenium allocation ID) previously allocated on the time axis is similarly associated with another ID (for example, user allocation ID) previously allocated on the time axis.

FIG. 29 is a block diagram showing details of the program execution unit 134 of FIG. The program execution unit 134 includes a test environment determination unit 171, an external operation execution unit 164, a proxy processing unit 166, an internal operation execution unit 168, and a witness setting unit 170. Of the functional blocks in the figure, functional blocks that are the same as or correspond to the functional blocks of the program execution unit 134 shown in the reference technique 3 are denoted by the same reference numerals. Functional blocks with the same reference numerals are described when their functions are different from those already described in Reference Technology 3, and descriptions of similar functions are omitted as appropriate.

The test environment determination unit 171 corresponds to the test subject selection unit 162 in the reference technique 3 (FIG. 21), and is based on the environment for the operation test of the web page, specifically based on the combination of the OS and the web browser. Determine the operation test method. For example, a web browser operation method and a dialog operation method are determined.

The external operation execution unit 164 corresponds to the IE browser operation unit 506 in FIG. The external operation execution unit 164 includes an ID assignment unit 174 and a dialog operation unit 176. When the ID allocation unit 174 detects that a function (command) for generating a new browser window is specified in the test program during execution of the test program, the ID allocation unit 174 uses the ID specified as an argument of the command as the user allocation ID. Determine as. When the ID is not specified, a new ID that is unique in the external operation execution unit 164 and the internal operation execution unit 168 may be assigned as a user allocation ID.

The dialog operation unit 176 corresponds to the IE dialog operation unit 508 in FIG. 24 and periodically monitors the dialog settings by the IE browser 542. For example, the above-described FindWindow function of Win32 API 552 may be periodically called to detect that a dialog window has been generated. The dialog operation unit 176 operates the OS dialog 544 displayed on the IE browser 542 via the Win32 API 552.

It should be noted that the external operation execution unit 164 includes a web page operation function that can be specified by the test program, a function defined as the web browser operation IF, a function defined in COM550 or Win32 API 552 in Reference Technique 4. Holds a table that stores the correspondence relationship. The external operation execution unit 164 refers to the table and calls a COM550 or Win32 API 552 function corresponding to the function specified by the test program.

The proxy processing unit 166 corresponds to the selenium module 530 in FIG. As already described in the reference technique 3, the proxy processing unit 166 acquires the operation target web page by proxy. The program code of the web page is a program code implemented by a JAVA script, and a function group ("control code" in Reference Technology 3) prepared in advance for executing various operations on the web page is added. To do. Then, the web page after the addition of the control code is displayed on the browser processing unit 160. Further, in response to an operation instruction from the internal operation execution unit 168, a function of a control code corresponding to the operation instruction is called to execute the web page operation in an intermediary manner.

The proxy processing unit 166 includes an ID issuing unit 178 and an add-on setting unit 179. The ID issuing unit 178 issues identification information for uniquely identifying the browser window in the proxy processing unit 166 when the browser processing unit 160 displays the web page after the addition of the control code. This identification information is acquired as a selenium allocation ID by the ID acquisition unit 139. The add-on setting unit 179 specifies the FF profile 570 including the window handler 572 in FIG. 26 when the FF browser 546 is to be activated in order to display the web page after the addition of the control code on the browser processing unit 160. Start up. As a result, the window handler 572 is incorporated as an extended function of the FF browser 546.

The internal operation execution unit 168 corresponds to the selenium browser operation unit 534 in FIG. The internal operation execution unit 168 includes an ID assignment unit 180 and a dialog operation unit 182. The ID allocation unit 180 has the same configuration as the ID allocation unit 174 of the external operation execution unit 164. The internal operation execution unit 168 holds a table that stores the correspondence between web page operation functions that can be specified by the test program and web page operation functions defined in the selenium module 530. The external operation execution unit 164 refers to the table and calls a function of the selenium module 530 corresponding to the function specified by the test program as an operation instruction to the selenium module 530.

The dialog operation unit 182 corresponds to the FF dialog operation unit 516 and the Safari dialog operation unit 526 in FIG. When the test environment is the Windows OS and the FF browser 546, the dialog operation unit 182 periodically monitors the dialog settings by the FF browser 546. For example, the above-described FindWindow function of Win32 API 552 may be periodically called to detect that a dialog window has been generated. The dialog operation unit 182 operates the OS dialog 544 displayed on the FF browser 546 via the Win32 API 552. Further, the dialog operation unit 182 detects the FF dialog 548 displayed by the FF browser 546 via the FF plug-in 554 and operates it via the FF plug-in 554.

Further, when the test environment is MacOS and the Safari browser 562, the dialog operation unit 182 periodically monitors the dialog settings by the Safari browser 562. For example, the above-mentioned Apple script may be periodically executed to detect a dialog window setting in the Safari browser 562. The dialog operation unit 182 operates the Safari dialog 564 displayed on the Safari browser 562 via the accessibility API 566.

The operation of the above configuration will be described below.
FIG. 30 is a flowchart showing the operation of the operation verification apparatus 100. This figure shows an operation when an operation test program is executed in the operation verification apparatus 100. For example, it shows details of S218 in FIG. 12, and also shows details of S228 in FIG. That is, as illustrated in FIG. 14, the operation verification apparatus 100 supports the operation check of the web page by reproducing a series of input operations performed by the user in advance on the web page. 30 can be said to be a more detailed version of the flowchart of FIG.

When receiving an instruction to start a web page operation test from the user, the test environment determination unit 171 determines the type of the OS and web browser that are the execution environment of the operation test. Then, according to the determination result, the external operation execution unit 164 or the internal operation execution unit 168 is selected as the operation subject of the web page (S400). The program execution unit 134, in other words, the external operation execution unit 164 or the internal operation execution unit 168 selected as the operation subject executes a test program reading process to be described later (S402). When all the operations (all items of the operation test) for the web page described in the test program are completed (Y in S404), the flow of FIG.

Here, as an example of the test program, a test program in which operation commands of the following operations 1 to 10 are described is assumed.
(Operation 1) Open browser window A (ID = 1) (Operation 2) Open browser window B (ID = 2) (Operation 3) Select browser window A (ID = 1)
(Operation 4) Operation on Web Page Displayed in Browser Window A (Operation 5) Operation on Web Page Displayed in Browser Window A (Operation 6) Operation on Dialog Displayed in Operation 5 (Operation 7) Browser Window B Select (Specify ID = 2)
(Operation 8) Operation on the web page displayed in the browser window B (Operation 9) Close the browser window A (specify ID = 1)
(Operation 10) Close browser window B (specify ID = 2)

This test program is an operation test program set by the series of processes shown in FIG. In the operation content file in this case, “waitForNewWindow” is specified as a function for specifying operation 1 and operation 2, and the URL of the web page and the user-specified ID are set as arguments. Further, “selectWindow” is designated as a function for designating operations 3 and 7, and an ID designated by the user is set as an argument thereof. In addition, as functions for specifying the operation 4, the operation 5, and the operation 8, functions for various operations such as WebEditSet and WebElementClick are specified. Further, a predetermined function for dialog operation is designated as a function for designating the operation 6.

In the execution of the test program, the operation commands described in the test program are executed sequentially. When an unexecuted operation command remains in the test program (N in S404), and when the display of a new web page is requested in the test program, that is, when the above operation 1 and operation 2 are to be executed ( The external operation execution unit 164 or the ID assignment unit of the internal operation execution unit 168 selected as the operation subject assigns the ID specified by the operation command as the user assignment ID. The ID acquisition unit 139 acquires the user assignment ID and stores it in the window ID holding unit 154 (S408).

When the operation IF of the web browser is open (Y in S410), the external operation execution unit 164 causes the web browser to acquire the data of the operation target web page by calling the operation IF (S412). Then, the browser window displaying the operation target web page is displayed on the display (S420). In FIG. 24, Y in S410 is when the web browser is the IE browser 542, and the COM 550 corresponds to the operation IF at this time.

When the operation IF of the web browser is not disclosed (N in S410), the internal operation execution unit 168 instructs the proxy processing unit 166 to acquire the operation target web page. The ID issuing unit 178 of the proxy processing unit 166 issues a selenium allocation ID (S414). The proxy processing unit 166 acquires data of the operation target web page (S416), and adds a control code to the data of the web page (S418). The proxy processing unit 166, after executing a web browser activation process described later (S419), passes the web page data to which the control code is added to the web browser, and displays a browser window for displaying the operation target web page on the display. (S420). In FIG. 24, N in S410 is when the web browser is the FF browser 546 or the Safari browser 562.

The ID acquisition unit 139 calls an API provided by the OS, acquires the OS assignment ID assigned by the OS to the browser window, and stores the acquired OS assignment ID in the window ID holding unit 154. When the web browser operation IF is not disclosed, the API provided by the selenium module 530 is called to acquire the selenium assignment ID assigned by the selenium module 530 to the browser window and store it in the window ID holding unit 154. To do. As a result, the user allocation ID, the OS allocation ID, and the selenium allocation ID allocated to the browser window A and the browser window B are associated with each other (S422).

If the display of a new web page is not specified in the test program (N in S406), S408 to S422 are skipped. When the operation of the web page is designated in the test program, that is, when the above operations 3 to 10 are to be executed (Y in S424), the program execution unit 134 executes a web page operation process described later (S426). . If no web page operation is specified in the test program (N in S424), S426 is skipped. After the operation on the web page, the proof object setting unit 170 stores the proof object data such as the screen snapshot in the result holding unit 152 (S428), and the process returns to S404.

Note that it is preferable that the proof object setting unit 170 records the proof object such as a screen snap related to the browser window as the proof object in association with the user assignment ID assigned to the browser window. Thereby, the user who confirms the result of the operation test on the web page can confirm the proof corresponding to the ID designated by the user, and the convenience of the user in the confirmation of the proof can be improved.

FIG. 31 is a flowchart showing in detail the test program reading process in S402 of FIG. The program execution unit 134 reads an operation test program (S430). When a predetermined function indicating a dialog operation is specified in the program (Y in S432), the program execution unit 134 stores the content of the dialog operation (the above operation 6) in a predetermined dialog operation holding table (S434). ). If no dialog operation is specified (N in S432), S434 is skipped.

FIG. 32 is a flowchart showing in detail the web browser activation process in S419 of FIG. The proxy processing unit 166 activates a web browser for the operation test (S440). When the web browser for the operation test is the FF browser 546 (Y in S442), the add-on setting unit 179 adds on the window handler 572 as an extended function of the FF browser 546 (S444). The window handler 572 registers a listener for detecting the FF dialog 548 in the FF browser 546 (S446). If the web browser for the operation test is not the FF browser 546 (N in S442), S444 and S446 are skipped.

FIG. 33 is a flowchart showing in detail the web page operation process of S426 of FIG. When the OS is MacOS in the determination result in the test environment determination unit 171 (Y in S450), the internal operation execution unit 168 executes a Mac operation process described later (S452). If the OS is not MacOS (N in S450), S452 is skipped. When the OS is Windows (Y in S454) and the web browser is the IE browser 542 (Y in S456), the external operation execution unit 164 executes an IE operation process described later (S458). When the web browser is not the IE browser 542 (N in S456) and is the FF browser 546 (Y in S460), the internal operation execution unit 168 executes an FF operation process described later (S462). If the web browser is not the FF browser 546 (N in S460), S462 is skipped, and if the OS is not Windows (N in S454), S456 and subsequent steps are skipped.

FIG. 34 is a flowchart showing in detail the Mac operation process in S452 of FIG. When a window selection operation (the above operations 3 and 7) is specified in the test program (Y in S470), the ID allocation unit 180 of the internal operation execution unit 168 identifies the ID received as an argument as the user allocation ID. (S472). When a predetermined operation to be executed via the Safari browser 562 (the above-described operations 4, 5, and 8, hereinafter also referred to as “browser operation”) is designated instead of the window selection operation (N in S470) (Y in S474) ), The internal operation execution unit 168 specifies the selenium allocation ID associated with the user allocation ID (S476). Then, by notifying the selenium module 530 of an operation instruction designating the selenium allocation ID, the function corresponding to the operation instruction among the control codes added to the web page is called by the selenium module 530 (S478).

When not a browser operation (N in S474) but a predetermined operation to be executed via MacOS (the above operations 9 and 10, hereinafter also referred to as “OS operation”) is designated (Y in S480), an internal operation is executed. The unit 168 identifies the OS assignment ID associated with the user assignment ID (S482). Then, by designating the OS allocation ID and calling a function of the accessibility API 566 corresponding to the operation content, Mac OS is caused to execute the OS operation on the browser window (S484). If it is not an OS operation (N in S480), S482 and S484 are skipped.

When the dialog operation unit 182 of the internal operation execution unit 168 detects the display of the Safari dialog 564 (Y in S486), the operation content for the dialog (the above operation 6) is acquired from the dialog operation holding table (S488). Then, an operation for the Safari dialog 564 is executed by calling a function of the accessibility API 566 corresponding to the operation content (S490). If the display of the Safari dialog 564 is not detected (N in S486), S488 and S490 are skipped.

FIG. 35 is a flowchart showing in detail the IE operation process in S458 of FIG. When a window selection operation (the above operations 3 and 7) is specified in the test program (Y in S500), the ID assignment unit 174 of the external operation execution unit 164 specifies the ID received as an argument as the user assignment ID. (S502). When an operation to be executed via the IE browser 542 (the above-described operations 4, 5, and 8, hereinafter also referred to as “browser operation”) is specified instead of the window selection operation (N in S500) (Y in S504), The external operation execution unit 164 identifies the OS assignment ID associated with the user assignment ID (S506). Then, by designating the OS allocation ID and calling a function of the COM 550 corresponding to the content of the browser operation, the IE browser 542 is caused to execute the operation (S508).

When not a browser operation (N in S504) but an operation to be executed via Windows OS (the above operations 9 and 10, hereinafter also referred to as “OS operation”) is specified (Y in S510), the external operation execution unit 164 Specifies the OS allocation ID associated with the user allocation ID (S512). Then, by designating the OS assignment ID and calling a function of the Win32 API 552 corresponding to the operation content, the OS operation on the browser window is executed by the Windows OS (S514). If it is not an OS operation (N in S510), S512 and S514 are skipped.

When the dialog operation unit 176 of the external operation execution unit 164 detects the display of the OS dialog 544 (Y in S516), the operation content for the dialog (the above operation 6) is acquired from the dialog operation holding table (S518). Then, an operation for the OS dialog 544 is executed by calling a function of the Win32 API 552 corresponding to the operation content (S520). If the display of the OS dialog 544 is not detected (N in S516), S518 and S520 are skipped.

FIG. 36 is a flowchart showing in detail the FF operation process in S462 of FIG. When a window selection operation (the above operations 3 and 7) is specified in the test program (Y in S530), the ID allocation unit 180 of the internal operation execution unit 168 identifies the ID received as an argument as the user allocation ID. (S532). When a predetermined operation to be executed via the FF browser 546 (the above-described operations 4, 5, and 8, hereinafter also referred to as “browser operation”) is designated instead of the window selection operation (N in S530) (Y in S534) ), The internal operation execution unit 168 identifies the selenium allocation ID associated with the user allocation ID (S536). Then, by notifying the selenium module 530 of an operation instruction specifying the selenium allocation ID, the selenium module 530 is called a function corresponding to the operation instruction among the control codes added to the web page (S538).

When not a browser operation (N in S534) but a predetermined operation to be executed via Windows OS (the above operations 9 and 10, hereinafter also referred to as "OS operation") is designated (Y in S540), an internal operation is executed. The unit 168 identifies the OS assignment ID associated with the user assignment ID (S542). Then, by designating the OS allocation ID and calling a function of the Win32 API 552 corresponding to the operation content, the OS operation on the browser window is executed by the Windows OS (S544). If it is not an OS operation (N in S540), S542 and S544 are skipped.

In this paragraph, it is assumed that the dialog displayed by the operation 5 is the FF dialog 548. When the dialog operation unit 182 of the internal operation execution unit 168 detects the display of the FF dialog 548 based on the notification from the window handler 572 (Y in S546), the dialog operation holding table displays the operation content (the above operation 6) for the dialog. (S548). Then, by notifying the window handler 572 of the operation content, the window handler 572 is caused to call a function of the FF browser 546 corresponding to the operation content (S550). As a result, an operation according to the test program is executed for the FF dialog 548. If the display of the FF dialog 548 is not detected (N of S546), S548 and S550 are skipped.

In this paragraph, it is assumed that the dialog displayed by the operation 5 is the OS dialog 544. When the dialog operation unit 182 of the internal operation execution unit 168 detects that the OS dialog 544 is displayed (Y in S552), the operation content for the dialog (the above operation 6) is acquired from the dialog operation holding table (S554). ). Then, an operation on the OS dialog 544 is executed by calling a Win32 API 552 function corresponding to the operation content (S556). If the display of the OS dialog 544 is not detected (N in S552), S554 and S556 are skipped.

According to the operation verification apparatus 100 of the reference technique 4, as already described in the reference technique 3, by using the selenium module 530, web pages on various web browsers can be displayed regardless of whether or not the operation IF is disclosed. Operation check is possible. Further, the user can assign an arbitrary ID to the browser window in the test program and the operation content file or function setting screen that is the generation source of the test program. In other words, the user can appropriately set the browser window ID according to the convenience of the test, while the selenium module 530 does not need to be aware of the selenium allocation ID necessary for identifying the browser window, thereby improving the user convenience. It can be improved.

Also, the operation verification apparatus 100 can support an operation on a browser window that should be performed without using the selenium module 530. In this case as well, the user can arbitrarily set the ID of the browser window according to the convenience of the test, but the user does not need to be aware of the ID assigned to the browser window by the OS or the web browser. Can be improved.

Furthermore, according to the operation verification apparatus 100, a dialog detection function is added on as an extended function of the web browser (FF browser 546). As a result, even if the function of the OS cannot acquire the contents of the dialog because the web browser uses its own resource to display the dialog, the contents of the dialog are acquired via the dialog detection function that is added, and the dialog Can be operated. In addition, even when the web browser can generate both the first dialog using its own resources and the second dialog using the OS's native resources, the operation for either dialog can be performed by appropriately switching the dialog operation method. Also make it possible. In other words, the first dialog is operated through the add-on dialog detection function, and the second dialog is operated through the OS native function, thereby supporting a series of operations on the web page. Can be reproduced.

Furthermore, according to the operation verification apparatus 100, the selenium module 530 is used in many operations on the browser window, and the OS API is directly called or a dialog operation module is used in exceptional operations. In this way, by combining the techniques described in Reference Technology 4 as appropriate, various test environments not mentioned in Reference Technology 4 (ie, combinations of OS and web browser) can be supported, and support for web page operation tests. it can.

(First embodiment)
An outline of the operation verification apparatus 100 according to the first embodiment of the present invention (hereinafter also referred to as “first embodiment”) will be described. FIG. 37 shows a display example of web content in the browser window. This figure shows a situation in which a web page (typically a test target page) provided from a web server is displayed on a browser window 632 on a display screen 630. In the first embodiment, the web browser activated in the browser window is the Safari browser 562.

The browser window 632 includes a client area 634 and a header area 636. The header area 636 is an area in which information related to the Safari browser 562 is mainly displayed, and various menus and icons for operating the Safari browser 562 are displayed. Also, meta information of the web page such as URL and title is displayed instead of the content of the web page itself.

On the other hand, the client area 634 is an area where the content of the web page itself is displayed. The client area 634 includes a plurality of frames each displaying a different web page. In the first embodiment, a frame a638, a frame b640, and a frame c642 are included. In the browser window 632, the header area 636 is arranged above the client area 634.

The vertical scroll bar 646 is arranged in the frame a638 because the entire vertical length including the non-display portion exceeds the vertical length of the visible range. The frame a638 includes an element 644. The “element” in the first embodiment is a GUI component that is arranged in a frame and can be set with a scroll bar, and includes, for example, a text box and a list box. The vertical scroll bar 645 is disposed in the element 644 in the drawing because the entire vertical length including the non-display portion exceeds the vertical length of the visible range. In the frame b640, the horizontal width including the non-display portion exceeds the horizontal length of the visible range, so the horizontal scroll bar 650 is arranged. The frame b640 includes a table 648.

The operation verification apparatus 100 according to the first embodiment stores a display image of the browser window 632 as one piece of image data for each frame. That is, in a screen capture of a web page composed of a plurality of frames, in other words, a web page that displays a plurality of web pages divided into frames (hereinafter also referred to as “top page” for convenience), for each frame. Save captured images. Further, when the contents of the entire frame cannot be displayed at a time in the browser window 632 and scrolling is necessary, a plurality of pieces of image data each including a part of the frame are acquired, and the image data obtained by connecting them is obtained as a final captured image. Save as.

FIG. 38 shows a captured image of the browser window of FIG. FIG. 38A shows a captured image of the frame a638. In the image capture process of the first embodiment, the entire frame a638 including the area not displayed in FIG. 37 is stored as one image. FIG. 38B shows a captured image of the element 644 in the frame a638. In the image capture process of the first embodiment, the entire element 644 including the area not displayed in FIG. 37 is stored as one image. FIG. 38C shows a captured image of the frame b640. In FIG. 37, the entire frame b640 including the non-displayed area is stored as one image. Although not shown in FIG. 38, a captured image of the frame c642 is also acquired. Since no scroll bar is arranged in the frame c642, a captured image indicating the display content in FIG. 37 is acquired.

Hereinafter, a browser window that should capture a display image is called a “capture target window”, a frame included in the capture target window is referred to as a “capture target frame”, and an element included in the capture target frame is referred to as a “capture target element”. A web page displayed in each frame on the top page is referred to as a “frame page”.

The main functional configuration of the operation verification apparatus 100 in the first embodiment is the same as the functional configuration of the operation verification apparatus 100 in the reference technique 4 shown in FIGS. Hereinafter, the description of the configurations already described in the reference techniques 1 to 4 will be omitted as appropriate, and the configuration for screen capture processing of web content will be described in detail.

FIG. 39 is a block diagram showing a functional configuration related to screen capture processing in the first embodiment. As described above, the operation verification apparatus 100 includes the evidence setting unit 170, the proxy processing unit 166, and the browser processing unit 160. The witness setting unit 170 calls various functions described later provided by the proxy processing unit 166. The proxy processing unit 166 appropriately communicates with the control code added to the program code of the web page, and causes the browser processing unit 160 to execute the JAVA script corresponding to the function called by the evidence setting unit 170. In other words, the witness setting unit 170 operates the web page using various functions provided by the proxy processing unit 166, and acquires information necessary for image capture of the web page from the browser processing unit 160 and the proxy processing unit 166. To do.

FIG. 40 shows functions provided by the proxy processing unit 166 to the witness setting unit 170. In other words, it can be said that these functions are functions that the selenium module 530 in FIG. 24 exposes to the outside. It should be noted that the functions other than “getOffsetPositionOfWindow” in the figure may be functions provided by the proxy processing unit 166, that is, the selenium module 530 as standard.

The proxy processing unit 166 of the first embodiment acquires the top page to be displayed on the Safari browser 562 as a proxy, and adds a control code to the top page. In addition, a frame page to be displayed in the frame of the top page is obtained by proxy, and a control code is added to each frame page. At that time, a unique ID is assigned to the browser window displaying the web page, and an ID is also assigned to each frame displayed on the web page. Then, the ID assigned to the window and the ID assigned to the frame are managed in association with each other.

39, the proxy processing unit 166 includes an image position specifying unit 600. The witness setting unit 170 includes an object specifying unit 612, a frame processing unit 614, an element processing unit 619, an image acquisition unit 625, and an image composition unit 627. Each function block in the figure can also be collectively referred to as a screen capture processing unit 629 as a function block for executing a screen capture process of web content. Each functional block of the screen capture processing unit 629 may be arranged in the proxy processing unit 166 according to the function of the function provided by the proxy processing unit 166 to the witness object setting unit 170. It may be arranged.

Proxy processing unit 166:
The image position specifying unit 600 realizes the function of the function “getOffsetPositionOfWindow” in FIG. That is, the display position on the screen 630 is derived for the frame (capture target frame) specified by the argument of the function, and information indicating the display position is notified to the evidence object setting unit 170. The image position specifying unit 600 includes a window display position acquiring unit 602, a header area specifying unit 604, a client area specifying unit 606, a frame relative position acquiring unit 608, and a frame display position specifying unit 610.

The window display position acquisition unit 602 acquires the display position on the screen 630 of the browser window 632 in which the capture target frame is displayed. In the first embodiment, the coordinates in the Cartesian coordinate system with the upper left corner of the screen 630 as the origin and the coordinates of the upper left corner of the browser window (point 659 in FIG. 37) are acquired.

Specifically, the window display position acquisition unit 602 communicates with the control code of the top page including the capture target frame, and causes “window.screenX” to be executed on the top page. Then, as a return value, the abscissa value of the point 659 (abscissa 652 in FIG. 37) is acquired. Similarly, “window.screenY” is executed on the top page, and the ordinate value of the point 659 (ordinate 654 in FIG. 37) is acquired as a return value.

The header area specifying unit 604 specifies the size of the header area 636 in the browser window 632. In the first embodiment, since the header area 636 is disposed above the client area 634, the header area specifying unit 604 specifies the vertical length of the header area 636.

Specifically, the header area specifying unit 604 communicates with the control code of the top page including the capture target frame, and causes “window.outerHeight” to be executed on the top page. Then, the vertical length of the browser window 632 (vertical width 656 in FIG. 37) is acquired as a return value. Similarly, “window.innerHeight” is executed on the top page, and the vertical length of the client area 634 (vertical width 658 in FIG. 37) is acquired as a return value. The header area specifying unit 604 derives the vertical length of the header area 636 by subtracting the vertical length of the client area 634 from the vertical length of the browser window 632.

The client area specifying unit 606 specifies the display position of the client area 634 on the screen 630 according to the display position of the browser window 632 on the screen 630 and the size of the header area 636. Specifically, the vertical length of the header area 636 is added to the ordinate value of the coordinates of the upper left corner (point 659) of the browser window 632, while the abscissa value of the coordinates of the point 659 is used as it is. The coordinates of the upper left corner (point 660) of the area 634 are derived.

The frame relative position acquisition unit 608 acquires the position of the capture target frame in the client area 634. In the first embodiment, the coordinates are in an orthogonal coordinate system with the upper left corner of the client area 634 (point 660 in FIG. 37) as the origin, and the coordinates of the upper left corner of the capture target frame (hereinafter also referred to as “coordinate within the client area”). .) For example, when the capture target frame is the frame b640, the coordinates in the client area of the point 662 in FIG. 37 are acquired.

Specifically, the frame relative position acquisition unit 608 communicates with the control code of the top page including the capture target frame, and causes “frame name.frameElement.offsetLeft” to be executed on the top page. Then, as a return value, the abscissa value of the coordinates in the client area of the capture target frame is acquired. Similarly, “frame name.frameElement.offsetTop” is executed on the top page, and the ordinate value of the coordinates in the client area of the capture target frame is acquired as a return value.

The frame display position specifying unit 610 determines the capture target frame on the screen 630 according to the display position of the client area 634 on the screen 630 derived by the client area specifying unit 606 and the position of the capture target frame on the client area 634. Specify the display position. Specifically, by adding the ordinate value and the abscissa value of the coordinates in the client region of the capture target frame to the ordinate value and the abscissa value of the upper left coordinate of the client region 634, the upper left end of the capture target frame The coordinates (coordinates on the screen 630) are derived.

Evidence setting unit 170:
The target specifying unit 612 determines target content to be image-captured from the web content displayed on the screen 630. Specifically, the capture target window, the capture target frame, and the capture target element are sequentially specified.

In the first embodiment, a user sets a screen capture function (for example, “WebScreen Capture” in FIG. 7) on the function setting screen 210, and sets a user allocation ID of a browser window to be captured as an argument thereof. The target specifying unit 612 identifies the browser window corresponding to the user assignment ID specified by the argument of the screen capture function as the capture target window.

Also, the target specifying unit 612 calls the function “getAllFrame” by specifying the selenium allocation ID corresponding to the user allocation ID specified by the argument of the screen capture function. Then, the proxy processing unit 166 obtains frame list information (hereinafter also referred to as “frame list”) included in the capture target window. This frame list includes the ID of each frame (hereinafter also referred to as “frame ID”) assigned by the selenium module 530 to each frame included in the capture target window. The target specifying unit 612 identifies each frame included in the frame list as a capture target frame.

Also, the target identifying unit 612 calls the function “getScrollableElement” by designating each ID of the capture target frame. Then, scrollable element list information (hereinafter also referred to as “element list”) included in the capture target frame is acquired from the proxy processing unit 166. This element list includes the ID of each scrollable element included in the capture target frame (hereinafter also referred to as “element ID”). The target specifying unit 612 specifies each element included in the element list as a capture target element.

The frame processing unit 614 executes a preparation process for capturing a capture target frame. The frame processing unit 614 includes a frame display position acquisition unit 615, a frame range acquisition unit 616, and a frame scroll processing unit 617.

The frame display position acquisition unit 615 calls the function “getOffsetPositionOfWindow” by specifying the frame ID of the capture target frame. As a result, the image position specifying unit 600 of the proxy processing unit 166 specifies the coordinates indicating the display position of the capture target frame on the screen 630, and the coordinate value is acquired from the proxy processing unit 166.

The frame range acquisition unit 616 specifies the frame ID of the capture target frame and calls the function “getScrollRectangleOfElement”. Then, the invisible portion of the capture target frame, in other words, the entire size including the portion displayed on the screen 630 for the first time by scrolling, here the horizontal length (horizontal width) and the vertical length of the entire capture target frame. (Vertical width) is acquired from the proxy processing unit 166.

Note that when the proxy processing unit 166 receives a call to the function “getScrollRectangleOfElement”, the proxy processing unit 166 may execute different JAVA scripts on the top page depending on the type of the browser processing unit 160. That is, in the case of the Safari browser 562 as in the first embodiment, the execution result of “frame name. Document. Width” may be specified as the horizontal width of the entire capture target frame. Similarly, the execution result of “frame name. Document. Height” may be specified as the vertical width of the entire capture target frame. On the other hand, when the browser processing unit 160 is the FF browser 546, the execution result of “frame name.window.innerWidth” + the execution result of “frame name.window.scrollMaxX” is specified as the horizontal width of the entire capture target frame. Also good. Similarly, the execution result of “frame name.window.innerHeight” + the execution result of “frame name.window.scrollMaxY” may be specified as the vertical width of the entire capture target frame.

Also, the frame range acquisition unit 616 specifies the frame ID of the capture target frame and calls the function “getClientRectangle”. The visible portion of the capture target frame, in other words, the size of the portion displayed on the screen 630 without scrolling, that is, the horizontal width and the vertical width of the visible portion in the capture target frame are acquired from the proxy processing unit 166.

When the proxy processing unit 166 receives a call to the function “getClientRectangle”, the proxy processing unit 166 executes a JAVA script corresponding to the function on the top page. For example, the execution result of “frame name. Document. DocumentElement. ClientWidth” may be specified as the horizontal width of the visible portion in the capture target frame. Similarly, the execution result of “frame name. Document. DocumentElement. ClientHeight” may be specified as the vertical width of the visible portion in the capture target frame.

The frame scroll processing unit 617 divides the horizontal width of the entire capture target frame by the horizontal width of the visible portion (the remainder is rounded up), and determines the result as the maximum number of horizontal scrolls. Further, the vertical width of the entire capture target frame is divided by the vertical width of the visible portion (the remainder is rounded up), and the result is determined as the upper limit number of vertical scrolls.

Further, when the image of the capture target frame is acquired, the frame scroll processing unit 617 vertically scrolls the frame or vertically scrolls and horizontally scrolls before acquiring the next image. Specifically, in order to execute vertical scrolling, the frame ID of the capture target frame, the horizontal scroll amount is 0, the vertical width of the visible portion of the capture target frame is specified as the vertical scroll amount, and the function “doScrollFrameTo” is set. call. Also, in order to execute horizontal scrolling, the function “doScrollFrameTo” is called by specifying the frame ID of the capture target frame, the horizontal width of the visible portion of the capture target frame as the horizontal scroll amount, and 0 as the vertical scroll amount.

In the first embodiment, scrolling is executed in a so-called inverted N type. That is, for each image acquisition process, the frame scroll processing unit 617 first repeats the vertical scroll once for the upper limit number of times while fixing the horizontal position. Next, the vertical position is returned to the initial position and horizontal scrolling is executed once. Then, the vertical scrolling is repeated once each up to the upper limit number while the horizontal position is fixed again. When both the vertical scroll count and horizontal scroll count reach the upper limit count, the scroll process for the capture target frame is terminated.

The element processing unit 619 executes a preparation process for capturing the capture target element. The element processing unit 619 includes an element relative position acquisition unit 620, an element display position specification unit 621, an element range acquisition unit 622, and an element scroll processing unit 623.

The element relative position acquisition unit 620 calls the function “getOffsetPositionOfElement” by specifying the element ID of the capture target element. Then, the coordinates in the Cartesian coordinate system having the origin at the upper left corner of the frame including the capture target element and the coordinates of the upper left corner of the capture target element (hereinafter also referred to as “in-frame coordinates”) are acquired. For example, for the element 644 included in the frame a638, the upper left corner coordinate of the element 644 with the point 662 as the origin is acquired.

When the proxy processing unit 166 receives a call to the function “getOffsetPositionOfElement”, the proxy processing unit 166 causes the JAVA script corresponding to the function to be executed on the frame page. For example, “element name.getBoundingClientRect.left” may be executed, and the abscissa value of the in-frame coordinates of the capture target element may be acquired as the return value. Similarly, “element name.getBoundingClientRect.top” may be executed, and the ordinate value of the in-frame coordinates of the capture target element may be acquired as the return value.

The element display position specifying unit 621 acquires coordinates of the display position of the capture target frame on the screen 630 acquired by the frame display position acquisition unit 615, and the frame of the capture target element acquired by the element relative position acquisition unit 620. The display position of the capture target element on the screen 630 is specified according to the internal coordinates. Specifically, by adding the ordinate and abscissa values of the in-frame coordinates of the capture target element to the ordinate and abscissa values of the upper-left corner coordinates of the capture target frame, the upper-left corner of the capture target element Coordinates (coordinates on the screen 630) are derived.

The element range acquisition unit 622 specifies the element ID of the capture target element and calls the function “getScrollRectangleOfElement”. Then, the entire size including the invisible part of the capture target frame, here, the horizontal width and the vertical width of the entire capture target frame are acquired from the proxy processing unit 166.

Note that when the proxy processing unit 166 receives a call to the function “getScrollRectangleOfElement”, the proxy processing unit 166 causes the JAVA script corresponding to the function to be executed on the frame page. For example, “element name. ScrollWidth” may be executed, and the horizontal width of the entire capture target element may be acquired as the return value. Similarly, “element name. ScrollHeight” may be executed, and the vertical width of the in-frame coordinates of the capture target element may be acquired as the return value.

Also, the element range acquisition unit 622 calls the function “getClientRectangleOfElement” by specifying the element ID of the capture target element. Then, the size of the visible portion of the capture target element, here, the horizontal width and vertical width of the visible portion in the capture target frame is acquired from the proxy processing unit 166.

When the proxy processing unit 166 receives a call to the function “getClientRectangleOfElement”, the proxy processing unit 166 executes a JAVA script corresponding to the function on the frame page. For example, the execution result of “element name. ClientWidth” may be acquired as the width of the visible portion of the capture target element. Similarly, the execution result of “element name. ClientHeight” may be acquired as the vertical width of the visible portion of the capture target element.

The element scroll processing unit 623 divides the horizontal width of the entire capture target element by the horizontal width of the visible portion (rounds up the remainder), and determines the result as the maximum number of horizontal scrolls. Further, the vertical width of the entire capture target element is divided by the vertical width of the visible portion (the remainder is rounded up), and the result is determined as the upper limit number of vertical scrolls.

In addition, when the image of the capture target element is acquired, the element scroll processing unit 623 vertically scrolls the element or vertically scrolls and horizontally scrolls before acquiring the next image. Specifically, in order to execute vertical scrolling, the element ID of the capture target element, the horizontal scroll amount is 0, the vertical width of the visible portion of the capture target element is specified as the vertical scroll amount, and the function “doScrollElementTo” is executed. call. In order to execute horizontal scrolling, the function “doScrollElementTo” is called by specifying 0 as the element ID of the capture target element, the horizontal width of the visible portion of the capture target element, and the vertical scroll amount as the horizontal scroll amount. The scroll mode of the element is the same as the scroll mode of the frame described above.

The image acquisition unit 625 captures the display image of the capture target frame according to the display position of the capture target frame on the screen 630. Further, the display image of the capture target element is captured according to the display position of the capture target element on the screen 630. The image acquisition unit 625 of the first embodiment excludes the header area 636 of the browser window 632 from the capture target. In other words, the display image of the capture target frame is obtained purely from which the display image of the header area 636 is excluded.

For example, the image acquisition unit 625 specifies the coordinate value of the upper left corner of the capture target frame on the screen 630 as the start point, specifies the horizontal width and vertical width of the visible portion of the capture target element as the capture range, and performs screen capture of the OS. A function (API) may be called. Then, the image data of the visible portion of the capture target element may be stored in the memory (clipboard).

Further, the image acquisition unit 625 repeats the capture processing of the capture target frame until the number of vertical and horizontal scrolls determined by the frame scroll processing unit 617 reaches the upper limit. Similarly, the capture processing of the capture target element is repeated until the number of vertical and horizontal scrolls determined by the element scroll processing unit 623 reaches the upper limit. Therefore, for a frame or element in which a scroll bar is set, data of a plurality of captured images is taken into the memory.

The image composition unit 627 synthesizes the data of a plurality of captured images acquired by a plurality of image acquisition processes for one capture target frame into one image data. For one capture target frame, one piece of image data is stored in the result holding unit 152 as a proof of the operation test. The same applies to the element. For one capture target frame, one piece of image data is stored in the result holding unit 152 as a proof of the operation test. If there is an overlapping portion between a plurality of captured images, the overlapping portion is trimmed from one captured image and then combined into a single image.

The operation of the above configuration will be described below.
FIG. 41 is a flowchart showing the operation of image capture processing in the operation verification apparatus 100. In the execution of the operation test, when a web page image capture request is received (Y in S600), the target specifying unit 612 acquires a list of frames included in the capture target window specified by the request from the proxy processing unit 166. (S602). The target specifying unit 612 specifies one of the frames included in the frame list as a capture target frame (S604). The frame display position acquisition unit 615 designates a capture target frame and causes the proxy processing unit 166 to execute a frame display position specifying process described later (S606). The frame range acquisition unit 616 acquires the size of the entire capture target frame and the size of the visible portion (S608). The frame scroll processing unit 617 determines the maximum number of horizontal and vertical scrolls to be performed on the capture target frame according to the size of the entire capture target frame and the size of the visible portion (S610).

The image acquisition unit 625 temporarily stores the image of the visible portion of the capture target frame in the memory (S612). If the upper limit number of scrolls has not been reached (N in S614), the frame scroll processing unit 617 changes the display content (that is, the visible portion) of the capture target frame by scrolling the capture target frame by one unit. , The process returns to S612 (S616). When the upper limit number of scrolls is reached (Y in S614), the image composition unit 627 acquires a plurality of capture images temporarily stored for the capture target frame, and sets one image data obtained by composing them. Then, the combined image data is stored in the result holding unit 152 as a proof of the operation test (S618). If there is one captured image, the image is stored in the result holding unit 152 as a proof as it is.

The target specifying unit 612 acquires a list of elements included in the capture target frame from the proxy processing unit 166. When an uncaptured element (that is, a capture target element) remains (N in S620), the element relative position acquisition unit 620 acquires the position of the capture target element in the capture target frame (S622). The element display position specifying unit 621 derives the display position of the capture target element on the screen according to the display position of the capture target frame on the screen and the position of the capture target element in the capture target frame (S624). ). Then, the process returns to S608.

In the element capture process, the element range acquisition unit 622 acquires the size of the entire capture target element and the size of the visible portion (S608). The element scroll processing unit 623 determines the maximum number of vertical and horizontal scrolls to be performed on the capture target element according to the size of the entire capture target element and the size of the visible portion (S610). The image acquisition unit 625 temporarily stores the image of the visible portion of the capture target element in the memory (S612). If the upper limit number of scrolls has not been reached (N in S614), the element scroll processing unit 623 scrolls the capture target element by one unit and returns to S612 (S616). When the upper limit number of scrolls is reached (Y in S614), the image composition unit 627 acquires a plurality of captured images temporarily stored for the capture target element, and sets one image data obtained by composing them. S618).

When the capture process for all the capture target elements is completed (Y in S620), if the capture process for some frames in the frame list is not completed (N in S626), the process returns to S604 and a new capture target frame is determined. And processing continues. When the capture processing for all the frames in the frame list is completed (Y in S626), the flow of FIG. If a web page image capture request is not accepted (N in S600), the subsequent processing is skipped and the flow of FIG.

FIG. 42 is a flowchart showing in detail the frame display position specifying process in S606 of FIG. The window display position acquisition unit 602 acquires the display position on the screen of the capture target window including the capture target frame (S630). The header area specifying unit 604 derives the size of the header area 636 by calculating the difference between the size of the entire capture target window and the size of the client area 634 (S632). The client area specifying unit 606 derives the display position of the client area 634 on the screen according to the display position of the capture target window on the screen and the size of the header area 636 (S634). The frame relative position acquisition unit 608 acquires the position of the capture target frame in the client area 634 (S636). The frame display position specifying unit 610 derives the display position of the capture target frame on the screen according to the display position of the client area 634 on the screen and the position of the capture target frame on the client area 634 (S638).

According to the operation verification apparatus 100 of the first embodiment, only the display image of the web page necessary for confirming the success or failure of the operation test is stored as a proof. Specifically, it is the content of the web page displayed in the frame itself, and presents the captured image to the user, excluding the extra part such as the header area, in other words, the part that would rather get in the way of confirming the result of the operation test. it can. Thereby, an efficient operation test can be supported.

(Second Embodiment)
The second embodiment of the present invention (hereinafter also referred to as “second embodiment”) differs from the first embodiment in that the web browser activated in the browser window 632 is the FF browser 546.

FIG. 43 shows a display example of web content in the browser window. Of the elements in the figure, elements that are the same as or correspond to the elements in FIG. 37 described above are denoted by the same reference numerals as in FIG. 37, and the following description is omitted. As shown in FIG. 43, unlike the browser window 632 of the first embodiment, the browser window 632 of the second embodiment includes a fixed-length (for example, 26 pixels) status bar at the bottom of the client area 634. Further, the browser window 632 of the second embodiment includes a frame line 666 in which a certain amount of width (for example, a fixed length of 4 pixels) is set as a boundary with an object outside the browser window (another window, background, etc.). .

The main functional configuration of the operation verification apparatus 100 in the second embodiment is the same as that of the first embodiment, and the main operation is also the same as that of the first embodiment. Hereinafter, the operation verification device 100 according to the second embodiment will be described while referring to differences from the first embodiment.

The header area specifying unit 604 of the image position specifying unit 600 specifies the size of the header area 636 in consideration of the size (width) of the status bar 664. Specifically, by subtracting the vertical width of the client area 634 from the vertical width of the browser window 632 and subtracting the vertical length of the status bar 664, which is a predetermined 26 pixels in this case, the header area 636 is subtracted. The vertical length of is derived. Strictly speaking, the fixed value of 26 pixels can be said to be the total value of the vertical length of the status bar 664 and the width of the frame line 666.

The client area specifying unit 606 of the image position specifying unit 600 specifies the display position of the client area 634 on the screen 630 in consideration of the size (width) of the frame line 666. Specifically, the vertical length of the header area 636 is added to the ordinate value of the upper left coordinate of the browser window 632, while the width of the frame line 666, which is predetermined here, is added to the abscissa value of the coordinate. The coordinates of the upper left corner of the client area 634 are derived by adding the four pixels.

According to the operation verification apparatus 100 of the second embodiment, the operation test using the web browser in which the status bar 664 and the frame line 666 having a certain width are arranged in the browser window 632 is the same as in the first embodiment. The effect of can be realized. In other words, in addition to the header area, a captured image that excludes a portion that hinders confirmation of the result of the operation test, such as a status bar and a frame line, can be presented to the user.

The present invention has been described based on the first embodiment and the second embodiment. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there. Hereinafter, a modification is shown.

A first modification will be described. In the above embodiment, the header area 636 is arranged above the client area 634, but the mode of the browser window 632 is not limited to this. For example, the header area 636 may be disposed on the left side of the client area 634. In this case, the header area specifying unit 604 specifies the horizontal length of the header area 636 by acquiring the horizontal length of the browser window 632 and the horizontal length of the client area 634 and calculating the difference between them. May be. Then, the client area specifying unit 606 adds the horizontal length of the header area 636 to the abscissa value of the upper left corner of the browser window 632, while using the ordinate value as it is, the upper left corner of the client area 634. Coordinates may be derived.

A second modification will be described. Although not particularly mentioned in the above embodiment, the first embodiment and the second embodiment may be combined and realized in the same operation verification apparatus 100. For example, when the proxy processing unit 166 captures the display image of the browser window in which the FF browser 546 is activated and the first image position specifying unit that is called when capturing the display image of the browser window in which the Safari browser 562 is activated. May include a second image position specifying unit to be called. Then, the proxy processing unit 166 may operate one of them depending on the type of browser window to be captured.

As another configuration, the header area specifying unit 604 and the client area specifying unit 606 of the image position specifying unit 600 perform the operation described in the first embodiment or the second embodiment according to the type of browser window to be captured. Any of the operations described may be performed. Note that the browser window type may be specified by the user in an argument at the start of the operation test or in various setting files, or may be dynamically read from the OS or the browser processing unit 160.

A third modification will be described. In the above embodiment, the client area 634 is divided into a plurality of frames. However, the client area 634 is not necessarily divided into a plurality of frames. In this case, the frame processing unit 614 may operate in the same manner as in the embodiment, regarding the entire client area as one frame. Further, the element processing unit 619 may operate in the same manner as in the embodiment, regarding elements included in the entire client area as elements included in one frame.

A fourth modification will be described. In the above embodiment, the Safari browser 562 and the FF browser 546 are illustrated as web browsers. However, the scope of application of the technical idea of the present invention is not limited to these types of web browsers. For example, the technology of the first embodiment can be applied to a web browser in which the status bar 664 is not arranged in the browser window 632 and the width of the frame line 666 is less than a predetermined value. Further, the technology of the second embodiment can be applied to a web browser in which the status bar 664 is arranged and the width of the frame line 666 is a predetermined value or more. Further, when the width of the frame line 666 is less than a predetermined value when the status bar 664 is disposed, or conversely, when the width of the frame line 666 is not greater than the predetermined value when the status bar 664 is not disposed, Those skilled in the art will understand that the configurations of the first embodiment and the second embodiment may be appropriately selected and combined.

A fifth modification will be described. Although not particularly mentioned in the above embodiment, the operation verification apparatus 100 further includes a verification image holding unit that holds a verification image for verifying whether the captured images acquired in the operation test match. Also good. This collation image is image data set in units of frames and elements, as in the capture image of the embodiment.

The operation verification apparatus 100 may further include a verification unit that determines whether the captured image as the evidence and the verification image match. The captured image as the evidence is a captured image in the operation test, in other words, an image synthesized by the image synthesizing unit 627, and in other words, a captured image stored in the result holding unit 152. The collation unit may store the determination result as to whether the captured image and the collation image match in a predetermined storage area (file, DB, etc.) and notify the user. According to this modification, it is possible to support the efficient operation test by automating the determination of the normality of the operation. Further, in the above embodiment, pure web content contents with the header area excluded are captured, so that the exclusion setting of the verification area (so-called mask processing) becomes unnecessary, and the captured image matches the verification image. It can be easily determined whether or not to do so.

Any combination of the above-described reference techniques, embodiments, and modifications is also useful as an embodiment of the present invention. The new embodiment resulting from the combination has the effects of the combined reference technology, embodiment, and modification.

It is understood by those skilled in the art that the functions to be fulfilled by the constituent elements described in the claims are realized by a single element or a combination of the constituent elements shown in the reference technology, the embodiments, and the modifications. By the way.

100 operation verification device, 160 browser processing unit, 166 proxy processing unit, 170 evidence object setting unit, 600 image position specifying unit, 602 window display position acquisition unit, 604 header area specifying unit, 606 client area specifying unit, 608 frame relative position Acquisition unit, 610 frame display position specifying unit, 612 target specifying unit, 625 image acquiring unit, 627 image compositing unit, 630 screen, 632 browser window, 634 client area, 636 header area, 664 status bar, 666 frame line.

This invention can be used for an apparatus for testing the operation of a product.

Claims (8)

1. A window position acquisition unit for acquiring a display position of the browser window on a screen for a browser window including a client area in which a web page is displayed and an additional area different from the client area;
   Image position specification for specifying the display position of the client area on the screen according to the display position of the browser window on the screen, the size of the additional area of the browser window, and the size of the status bar of the browser window And
   An image acquisition unit that captures a display image of the client area according to a display position on the screen specified by the image position specifying unit;
   An image acquisition apparatus comprising:
2. The image acquisition apparatus according to claim 1, wherein the image acquisition unit excludes an additional area of the browser window from a target to be captured.
3. The image position specifying unit includes:
   By obtaining the length of the browser window and the length of the client area, by subtracting the length of the client area and the length predetermined as the size of the status bar from the length of the browser window, An additional area specifying unit for deriving the length of the additional area;
   A client area specifying unit that specifies, as the display position of the client area on the screen, a position obtained by shifting the display position of the browser window on the screen according to the length of the additional area;
   The image acquisition apparatus according to claim 1, further comprising:
4. In the browser window, the additional area is arranged at the top of the client area, and the status bar is arranged at the bottom of the client area,
   The window position acquisition unit acquires the upper left coordinates of the browser window in an orthogonal coordinate system as the display position of the browser window on the screen,
   The additional area specifying unit specifies a vertical length of the additional area,
   The client area specifying unit derives the upper left coordinates of the client area in the orthogonal coordinate system by adding the vertical length of the additional area to the ordinate value in the upper left coordinates of the browser window. The image acquisition apparatus according to claim 3.
5. The client area specifying unit derives the ordinate value in the upper left coordinate of the client area by adding the vertical length of the additional area to the ordinate value in the upper left coordinate of the browser window, while the browser window Deriving the abscissa value in the upper left coordinate of the client area by adding a predetermined length as the width of the frame line indicating the boundary between the browser window and the outside to the abscissa value in the upper left coordinate of The image acquisition apparatus according to claim 4.
6. The client area includes a plurality of frames,
   The apparatus further includes a frame position acquisition unit that acquires a position of the capture target frame in the client area for the capture target frame among the plurality of frames.
   The image position specifying unit specifies the display position of the capture target frame on the screen according to the display position of the client area on the screen and the position of the capture target frame on the client area,
   The image acquisition unit according to claim 1, wherein the image acquisition unit captures a display image of the capture target frame according to a display position of the capture target frame on the screen. apparatus.
7. A method performed by an image acquisition device,
   Obtaining a display position of the browser window on a screen for a browser window including a client area in which a web page is displayed and an additional area different from the client area;
   Identifying the display position of the client area on the screen according to the display position of the browser window on the screen, the size of the additional area of the browser window, and the size of the status bar of the browser window;
   Capturing a display image of the client area according to a display position on the screen specified in the specifying step;
   An image acquisition method comprising:
8. In the image acquisition device,
   A browser window including a client area in which a web page is displayed and an additional area different from the client area, a function of acquiring a display position of the browser window on a screen;
   A function for specifying the display position of the client area on the screen according to the display position of the browser window on the screen, the size of the additional area of the browser window, and the size of the status bar of the browser window;
   A function of capturing a display image of the client area according to a display position on the screen specified by the specifying function;
   Computer program for realizing.

Images