JP2010231594A - Test program and testing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a test program and a testing device for reusing a test scenario. <P>SOLUTION: An execution means 12 reads a test scenario stored in a scenario DB 11, and makes test object integrated equipment 20 execute an operation based on the test scenario. In executing the moving operation of a focus on a display screen, a re-execution means 14 determines whether or not the component of the movement destination of focus based on the operation result information is matched with a target component. When it is determined that those components are not matched, the integrated equipment 20 is made to execute the moving operation of the focus until the focus moves to the target component, and a procedure to confirm the component of the movement destination is repeated. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a test program and a test apparatus, and more particularly to a test program and a test apparatus that perform a test based on a test scenario that describes a test procedure for a target device.

Conventionally, in the development of software installed in embedded devices such as mobile phones, testing has been automated in order to shorten test time and reduce development costs.
As a method of automatically performing the test, prepare a test scenario describing the test procedure of the device under test and an expected value in advance, and extract the operation target that automatically executes key operations and the test target status. There is a technique for causing a test apparatus including a measuring unit to execute the method. In such a test equipment, the test scenario is executed by the operation unit, the test scenario execution result observed by the measurement unit is compared with the expected value, and if there is a difference, an error is detected, and the test execution to judgment is automatically performed. (See, for example, Patent Document 1).

Japanese Patent Laid-Open No. 2005-196672

However, the conventional test apparatus has a problem that it is not easy to reuse the test scenario.
In product development, a plurality of models with different installed functions may be developed at the same time. In addition, slight specification changes may occur due to version upgrades. For example, in mobile phones, it is common to develop a plurality of models depending on differences in function and time. In such a case, even in the same application, the menu configuration differs depending on the presence or absence of the function to be installed and the specification change.

  Thus, when there are a plurality of models having slightly different specifications, if a test scenario created once for a certain model can be reused for another model, the burden required for the test can be greatly reduced. However, in the conventional test apparatus, if the specifications are slightly different, the description of the test scenario has to be corrected and it is difficult to reuse.

  An example will be described. The model A menu screen has “register” and “delete” operation buttons arranged in a vertical direction, and “registration” is focused when the menu screen is opened. Hereinafter, a predetermined range on the display screen associated with an operation command that prompts the target device to perform a predetermined process, such as an operation button, is collectively referred to as a part. The focus means that the operation of the part is selected and instructed, and usually a part or a character string on the part is highlighted. For example, the user can easily see that the character is selected, such as changing the color of the character on the part to another part, or setting the cursor under the character string. When testing the delete operation on the menu screen of this model A, it is necessary to move the focus of “Registration” to “Delete” arranged below, and select “Delete” to execute it. . At this time, the test scenario is described as “the operation of pressing“ down key ”and pressing“ decision ”.”

  Here, as the next-generation model B, it is assumed that the specification is changed to add an editing function in addition to the registration function and the deletion function. The registration function and the deletion function are the same as model A. On the menu screen of the model B, an “edit” operation button is added between “registration” and “deletion”, and “registration”, “edit”, and “deletion” are arranged in the vertical direction. In this case, when the above test scenario is executed, “edit” is selected by the “down key” operation, and the editing function is executed. In order to test the delete function, the test scenario had to be rewritten to press the “down key” twice. In this case, the test scenario is rewritten, for example, “Press the“ down key ”, press the“ down key ”, and press“ decision ””. Such a test scenario correction must be performed manually for each model, which is a heavy burden on the tester.

  In view of such problems, it is an object to provide a test program and a test apparatus that can reuse a test scenario if a difference in specifications is small.

  In order to solve the above-described problem, a test program for performing a test based on a test scenario describing a test procedure of a target device is provided. The test program is applied to the test apparatus, and causes the computer to function as an execution unit, an operation result acquisition unit, and a re-execution unit. The execution means includes a scenario storage means for storing a test scenario group including a test scenario in which a movement operation command for moving the focus of the part on the display screen to the target part is described in association with the identification information of the target part. Read test scenario. Then, the target device is caused to execute an operation based on the read test scenario. The operation result acquisition unit acquires, from the target device, operation result information including identification information of a movement destination component focused after the execution of the movement operation when the execution unit performs the focus movement operation. The re-execution unit determines whether or not the identification information of the destination component included in the operation result information acquired by the operation result acquisition unit matches the identification information of the target component described in the test scenario. When it is determined that they do not match, the target device is caused to execute a focus movement operation, and the identification information of the destination component and the identification information of the target component included in the operation result information acquired via the operation result acquisition unit Repeat the procedure for checking. Then, the identification information of the destination component and the identification information of the target component are matched.

  A computer that executes such a test program reads a test scenario stored in the scenario storage means, and causes the target device to execute an operation based on the test scenario. If the instruction to be executed is a focus movement operation, the identification information of the component to which the focus is moved is obtained from the operation result information, and it is determined whether or not it matches the identification information of the target component. When it is determined that they do not match, the target device is caused to execute the focus movement operation and the procedure for collating the identification information of the destination component after execution with the identification information of the target component is repeated. To match the desired part.

  According to the disclosed test program and test apparatus, when the focus movement operation described in the test scenario is executed, the target component to be focused after the movement operation is determined, and the target component is focused. The focus movement operation is repeated. As a result, even when the arrangement of components on the display screen changes due to a specification change or the like, the difference can be absorbed and the target component can be focused. As a result, the test scenario can be reused on a plurality of models.

It is the figure which showed the outline | summary of invention. It is the figure which showed the structural example of the test system. It is the figure which showed the hardware constitutions of the test apparatus. It is the figure which showed an example of the menu screen and the test scenario. It is the figure which showed an example of expected value data. It is the figure which showed the menu screen after a specification change. It is the figure which showed an example of the menu screen of depth 2, and a test scenario. It is the flowchart which showed the processing procedure of the automatic test. It is the flowchart which showed the procedure of the menu execution process. It is the flowchart which showed the procedure of the message execution process in consideration of the case where a command menu character string does not exist. It is the flowchart which showed the procedure of the message execution process which considered the case where the command depth of a test scenario was larger than a menu. It is the flowchart which showed the procedure of the message execution process which considered the case where the command depth of a test scenario was smaller than a menu.

Hereinafter, embodiments will be described with reference to the drawings. First, an outline of the invention will be described, and then specific contents will be described.
FIG. 1 is a diagram showing an outline of the invention.

  The test apparatus 10 is an apparatus that automatically performs an application test of the embedded device 20. The embedded device 20 is a device to be tested. In addition, since it is an apparatus for performing a test, it is not necessary to be exactly the same as a product mechanically. Even if some functions not related to the test are omitted, an emulator that virtually reproduces the embedded device 20 itself on the computer may be used. In addition, the test apparatus 10 and the embedded device 20 can exchange information by an information transmission unit (not shown). For example, if the test apparatus 10 and the embedded device 20 are separate apparatuses, information is exchanged via communication means. When they exist in the same device, information is exchanged via a common memory or the like.

  First, the embedded device 20 to be tested will be described. The embedded device 20 includes an application processing unit 21, an input unit 22, a control unit 23, and a result output unit 24, and executes a process according to an operation command input from the test apparatus 10.

The application processing unit 21 executes an application program according to the control unit 23 and realizes a predetermined application function of the embedded device 20.
The input unit 22 acquires an operation command from the execution unit 12 and the re-execution unit 14 of the test apparatus 10, converts it into a format that can be interpreted by the control unit 23, and sends the operation command to the control unit 23.

  The control unit 23 controls the entire embedded device 20 and causes the application processing unit 21 to execute processing according to the operation command input from the input unit 22. Some processing is done by itself. In addition, management processing for managing components focused on the display screen displayed by the application processing unit 21 is also performed. In the focus management process, when a focus movement operation command is executed, a component that is a focus target after execution is specified, and the identification information is notified to the test apparatus 10 via the result output means 24 as operation result information. Notification may be made directly to the test apparatus 10.

The result output unit 24 outputs operation result information describing the result of the operation of the application processing unit 21 and the control unit 23 to the test apparatus 10.
Next, the test apparatus 10 for testing the built-in device 20 will be described. The test apparatus 10 includes a scenario database (hereinafter referred to as DB) 11, execution means 12, operation result acquisition means 13, re-execution means 14, and depth output injection means 15.

  The scenario DB 11 is a scenario storage unit that stores a test scenario describing a test procedure to be executed by the embedded device 20 to be tested. In the test scenario, an operation command for causing the embedded device 20 to perform an operation to be tested, and a determination command for comparing the result executed by the operation command with an expected value are described. Further, the operation instructions include a movement operation instruction for moving the focus of a component on the display screen and other operation instructions. The move operation command instructs to move the focus to a component prepared on the display screen in order to instruct execution of a predetermined function. For this reason, the movement operation command described in the test scenario is associated with the identification information of the target component that is the focus movement destination.

  The execution means 12 reads one line of the test scenario stored in the scenario DB 11. Then, an operation execution command based on the read test scenario is output to the input unit 22 of the embedded device 20.

  The operation result acquisition unit 13 acquires operation result information output from the result output unit 24 of the embedded device 20. The operation result information includes identification information of a destination component to which the focus has moved after the embedded device 20 has performed the focus movement operation.

  The re-execution unit 14 includes identification information of a component to which the focus is moved included in the operation result information acquired by the operation result acquisition unit 13, identification information of a target component of the movement operation command described in the corresponding test scenario, and To determine whether or not they match. If it is determined that they match, the target part is focused, and no particular processing is performed. When it is determined that they do not match, the focus is moved so that the target component is focused. For example, if the embedded device 20 is provided with a direction key for instructing the direction in which the focus is moved, the same operation command as when the direction key is operated is instructed to the embedded device 20. The embedded device 20 moves the focus in the direction specified in accordance with this operation command. The control means 23 identifies the part to be focused that has been moved by this operation, and sets its identification information in the operation result information. The test apparatus 10 obtains identification information of the destination component based on the operation result information input via the operation result acquisition unit 13. Then, the identification information of the destination component and the identification information of the target component are collated, and if they do not match, the procedure of executing the movement operation again is repeated. This procedure is repeated until the destination part matches the target part.

  The depth output injection means 15 analyzes an application program that executes the processing of the application processing unit 21 of the embedded device 20 to be tested, and detects a command that requests a menu display command existing in the application program. Then, a menu call log output command for notifying the menu depth is injected into the detected menu display command. The menu here refers to an operation menu displayed on the application screen in order to perform an operation related to a predetermined application. When operating the embedded device 20, an operation menu screen is opened, a component corresponding to the operation displayed on the menu screen is selected, and processing corresponding to the component is started. On the operation menu screen, selectable operations are displayed as a list. The menu screen may have a hierarchical structure. For example, in order to perform more detailed settings for the operation selected on the menu screen displayed first, when a component is selected on the first menu screen, a menu screen related to the operation of this component is further opened. This number of layers is called the menu depth. The output command injected into the menu display command is set for all the menu display commands existing in the application program. Then, the application program in which the output command is injected into the menu display command is stored in the storage unit of the embedded device 20.

An application test of the embedded device 20 using the test apparatus 10 having such a configuration will be described.
In the application test of the embedded device 20, an operation command similar to that when the user operates the embedded device 20 is output to the embedded device 20 according to the test scenario, and the test target operation is executed. The operation instruction is performed by displaying a menu screen, selecting a part corresponding to a desired operation from the operation list displayed on the menu screen, and operating the enter key.

  When the menu display command is called by the depth output injection means 15, the application program installed in the embedded device 20 is injected with a menu call log output command for notifying the depth of the menu, and then the embedded device 20. Forwarded to The scenario DB 11 stores a test scenario describing a test procedure. In the movement operation command included in this test scenario, identification information of the target part is set.

  When the application test of the embedded device 20 by the test apparatus 10 is started, the execution unit 12 reads the test scenario from the scenario DB 11 and outputs an operation command according to the test procedure based on the test scenario to the embedded device 20. An operation command from the test apparatus 10 is received by the input unit 22 of the embedded device 20 and transferred to the control unit 23. The control unit 23 causes the application processing unit 21 to execute processing corresponding to the execution instruction. At this time, if the execution instruction is a menu display request, the application processing unit 21 displays the requested menu, and the depth of the menu is set as a menu call log to the test apparatus 10 via the result output unit 24. Output. In the case of a focus movement operation command including a focus movement including a menu display command, the control unit 23 outputs the identification information of the focus target component moved after the command execution via the result output unit 24.

  In the test apparatus 10, the operation result acquisition unit 13 acquires operation result information of the embedded device 20 and sends it to the re-execution unit 14. When the operation command that the execution unit 12 has caused the embedded device 20 to execute is a focus movement operation command including a menu display command, the re-execution unit 14 identifies identification information of the movement destination component included in the operation result information. And the identification information of the target part of the test scenario are compared. If they match, it is determined that the currently focused component is the target component, and the process ends. If they do not match, it is determined that the currently focused component is not the target component, and the focus is moved. A focus movement operation command is transmitted to the embedded device 20, operation result information after execution of the focus movement operation command is acquired, and identification information of the component to which the focus is moved is confirmed. This procedure is repeated until the acquired identification information of the destination component matches the identification information of the target component. Thus, after the focused component becomes the target component, the processing of the re-execution unit 14 ends.

Subsequently, the execution means 12 reads the next test scenario from the scenario DB 11 and repeats the above procedure.
As described above, according to the test apparatus 10, when a moving operation command including a focus movement including a menu display command is executed, the focused component is monitored, and the target component is focused by the re-execution unit 14. Until the focus movement operation command is repeated. Thus, the target part is focused, and the next test procedure is executed by the execution means 12.

  Thereby, even when the arrangement of components on the display screen is shifted due to a specification change or the like, the shift can be absorbed and the target component can be focused, and the test procedure of the test scenario can be continued. The test scenario can be reused on a plurality of models. As described above, according to the test apparatus 10, the man-hours for creating and managing the test scenario can be reduced, the test time can be shortened, and the development cost can be reduced. As a result, it becomes possible to provide a high-quality product at an early stage.

Next, an embodiment will be described by taking as an example the case where the test apparatus 10 shown in FIG. 1 is applied to a test system for testing mobile phone applications.
FIG. 2 is a diagram illustrating a configuration example of the test system.

  The test system includes a test device 100 that executes a test, a mobile phone device 200 to be tested, and a development environment 300 that generates an application program 410 that incorporates functions used when testing the mobile phone device 200. In the figure, the test device 100, the mobile phone device 200, and the development environment 300 are configured as separate devices. However, each processing function may be realized on the same computer. For example, a test unit that realizes the processing function of the test apparatus 100, an emulator unit that reproduces the operation of the mobile phone device 200, and a development environment unit that generates the application program 410 are mounted on a computer. Each device will be described.

The test apparatus 100 includes a scenario DB 110, a scenario drive unit 120, an expected value DB 130, and a measurement unit 140.
In the application test, an operation command is input to the mobile phone device 200 in the same manner as when the user performs an operation, and it is determined whether or not the result of the process executed according to the operation matches an expected value. And do it. The scenario DB 110 stores a test scenario describing a test procedure for an application test of the mobile phone device 200. In the test scenario, a command representing an operation on the mobile phone device 200 and information related to the command are described. In the procedure of opening the menu screen and selecting an operation, a command for instructing display of the menu screen and identification information of a target component selected from the menu screen are set. Note that the component identification information is a character string representing a predetermined operation displayed on the component. For example, when a part on the display screen on which a character string “delete” is displayed is selected, an operation command for deleting the corresponding information is generated in the mobile phone device 200.

  The scenario driving unit 120 includes an execution unit 12, an operation result acquisition unit 13, and a re-execution unit 14, and sequentially executes a test operation based on the test scenario in the scenario DB 110. First, the test scenarios stored in the scenario DB 110 are read line by line to interpret the test scenarios. Based on the test scenario, an operation command is issued to the mobile phone device 200 via the input unit 220. When the operation instruction is a focus movement operation instruction including a menu display request, after the movement operation instruction is performed, the movement destination component is confirmed based on the menu call log 501 and the menu log 502. When the destination component is not the target component, the focus movement operation and the determination as to whether or not the destination component is the target component are repeated to move the focus to the target component.

  The measurement unit 140 monitors output information of the output unit 230 of the mobile phone device 200. As a measurement method, for example, characters and graphics displayed on the screen are acquired and compared with expected value data stored in the expected value DB 130. The comparison result is displayed on the display unit of the test apparatus 100.

The mobile phone device 200 includes an OS (Operating System) 210, an input unit 220, an output unit 230, and an application unit 240.
In addition to a general operating system, the OS 210 includes programs such as middleware, a framework, and a common library that provide general-purpose functions to applications, and controls the entire mobile phone device 200. In particular, the menu mechanism unit 211 has a function of outputting text information of the selected menu as the menu log 502 when the menu is displayed. In the menu log 502, at least a character string on a component focused on the menu display screen is recorded.

  The input unit 220 is a mechanism for a user to operate such as a keyboard and a touch panel. Furthermore, a device for detecting sound such as a microphone, an input via wired or wireless communication, a signal input from a sensor such as a GPS (Global Positioning System), an acceleration sensor, and the like are also included. In the following, an explanation will be given mainly on an input signal from a keyboard on which a user gives an operation instruction. The keyboard includes a menu key for displaying a menu, an arrow key indicating a direction in which the focus is moved, and a determination key for starting an operation corresponding to the focused component.

  The output unit 230 refers to a screen output, a light emission output such as an LED, a mechanism for notifying the user of information such as sound and vibration, and an output by communication. Here, a description will be given mainly for screen output.

  The application unit 240 executes application processing based on the application program 410 generated by the development environment 300. The application unit 240 provides individual functions such as a telephone function and an address book. At that time, in order to improve development efficiency and provide a group of functions with a sense of unity, a program is created using the functions provided by the OS 210. For example, in an address book application, a menu key handler (program) 412 is generated when a menu is displayed on the screen when a menu key is pressed. Among them, a menu display command 411 is created as a program for calling the menu mechanism unit 211 of the OS 210. These programs cause the computer to operate as a menu key handler 242 and a menu display command 241 respectively.

  The development environment 300 includes a depth output injection unit 310 and injects a depth output command into the target application program 400 to generate the application program 410. For example, the menu display command 401 is a command for requesting the menu mechanism unit 211 to display a menu. In the depth output injection unit 310, a depth output function for outputting the menu depth at that time to the menu log 502 is added, and a menu display command 411 is generated. This function addition is applied to all menu display commands 401 included in the application program 400. The function addition method can analyze a source program of the application program 400 as a preprocessor and generate a new application program 410 as source code. Moreover, you may inject | pour in a binary level using an aspect-oriented language.

  In this way, the application program 410 including the menu display command 411 and the menu key handler 412 to which the depth output function is added is created from the application program 400 including the menu display command 401 and the menu key handler 402.

Here, the hardware configuration of the test apparatus 100 will be described. FIG. 3 is a diagram illustrating a hardware configuration of the test apparatus.
The entire test apparatus 100 is controlled by a CPU (Central Processing Unit) 101. A random access memory (RAM) 102, a hard disk drive (HDD) 103, a graphic processing device 104, an input interface 105, and a communication interface 106 are connected to the CPU 101 via a bus 107.

  The RAM 102 temporarily stores at least a part of OS programs and application programs to be executed by the CPU 101. The RAM 102 stores various data necessary for processing by the CPU 101. The HDD 103 stores an OS and application programs. A monitor 108 is connected to the graphic processing device 104, and an image is displayed on the screen of the monitor 108 in accordance with a command from the CPU 101. A keyboard 109 a and a mouse 109 b are connected to the input interface 105, and signals transmitted from the keyboard 109 a and the mouse 109 b are transmitted to the CPU 101 via the bus 107. The communication interface 106 is connected to a network (not shown), and can send and receive data to and from other devices via the network.

  With such a hardware configuration, the processing function of the test apparatus 100 can be realized. 3 shows the hardware configuration of the test device, the hardware configurations of the development environment 300 and the mobile phone device 200 are the same.

  An application test of the mobile phone device 200 by the test system will be described in detail. The application test tests whether the mobile phone device 200 operates according to the specification according to the user's operation. The user's procedure is to press a menu key, call up an operation menu, and select a desired operation from the displayed operation menu screen. The operation menu screen includes a case where a desired operation is displayed when the menu screen is opened and a case where the menu screen has a hierarchical structure. In the case of a hierarchical structure, when the menu key displayed on the first menu screen displayed by pressing the menu key is selected, the second menu screen is opened and several menu keys are displayed. Is done. In a menu display screen having a hierarchical structure, which level of the menu screen is displayed is represented by a parameter called depth. The screen displayed by pressing the menu key is set to depth 1, and the depth value is increased each time the menu key is expanded. Note that the single layer can be regarded as the same as in the case of depth 1.

Hereinafter, a single-layer (depth 1) menu process will be described as the first embodiment, and a multi-layer menu process will be described as the second embodiment.
An application test for a single-level (depth 1) menu screen according to the first embodiment will be described.

The application test is performed in the following cycle.
(1) Source code creation, editing, compilation, build, depth output function injection, deployment to mobile phone devices (2) Creation or reuse of test scenarios and expected values (3) Automatic test execution The processing procedure will be described in order.

First, (1) processing from creation of source code to deployment to a mobile phone device will be described.
The application program 410 is created in the development environment 300 such as a personal computer. First, an original application program 400 including a menu display command 401 and a menu key handler (program b) 402 is prepared. In the development environment 300, an executable file is created from the application program 400 using an editor, a compiler, and a linker prepared in the development environment 300. At this time, the depth output injection unit 310 detects the menu display command 401 included in the application program 400, adds a function of outputting the current menu depth to the menu call log, and generates the application program 410. . This process is performed at any stage until the execution format file of the application program 410 is created. The generated executable file is transferred to the mobile phone device 200.

Next, (2) processing up to creation or reuse of test scenarios and expected values will be described.
FIG. 4 is a diagram illustrating an example of a menu screen and a test scenario. The example in the figure shows a menu screen with a menu depth of 1.

  The menu screen 2000 is assumed to be a menu opened to delete one piece of address information from the address list. For example, when the menu key is pressed while the address list display screen is displayed, it is displayed as a pop-up screen. The menu screen 2000 includes a component 2001 including a character string “registration” representing a function for newly registering address information in the address list, and a component 2002 including a character string “delete” representing an operation of deleting address information from the address list. Are arranged in the vertical direction. Hereinafter, they are referred to as “register” 2001 and “delete” 2002, respectively. The focused component is referred to as a focus menu character string.

  When the enter key is operated while “Registration” 2001 is focused, an application for registration processing is started and the input address information is registered. When the enter key is operated while “Delete” 2002 is focused, an application for deletion processing is started and the selected address information is deleted.

  The test scenario 1000 is an instruction for opening a menu screen from the address list display screen and instructing deletion of one line. “Menu menu key“ delete ”” 1001 is an operation command for pressing a menu key to open a menu screen and selecting “delete” 2002 from the menu screen. Here, “Menu” is a menu display operation command for requesting display of a menu screen. The “menu key“ delete ”” is a command for instructing to select “delete” from the displayed menu key. Note that the character string delimited by the symbols ““ ”and“ ”” is a character string that represents a target operation instruction described in the command, and is a command menu character string in the following description.

  The “Assert address list” 1002 is a determination command for determining whether the “address list” screen after executing the process of the “Menu menu key“ delete ”” 1001 is correct, and whether it matches with the expected value. .

  Thus, the test scenario used by the test apparatus 100 does not indicate an actual key operation up to the menu key such as “down key operation”, but only the target menu key is described, and the procedure of the key operation in the middle is described. Not.

The expected value data at this time will be described.
FIG. 5 is a diagram illustrating an example of expected value data. Further, FIG. 5 also shows the result obtained by the measurement unit 140.

When the measurement unit 140 executes the test scenario and acquires the result, the measurement unit 140 collates the acquired result 2300 with the expected value data 1300 prepared in advance.
The expected value data 1300 is created in advance by a tester and is stored in the expected value DB 130. The expected value data 1300 in the figure is referred to when the “Assert address list” 1002 of the test scenario 1000 shown in FIG. 4 is executed.

  In the screen 1301, a screen name described in the test scenario is registered. Here, the screen name “address list” corresponding to the “Assert address list” 1002 of the test scenario 1000 is set. Further, the control 1302 stores the target type, and the content 1303 stores the comparison data item and the expected value data in association with each other. For example, the expected value data 1300 describes that the number of rows of the table is 4.

  In the result 2300, the state of the “address list” screen acquired by the measurement unit 140 after execution of the test scenario is set. The item corresponds to the content 1303 of the expected value data 1300.

(3) The automatic test execution process will be described.
The mobile phone device 200 stores an application program 410 including an output processing procedure for the menu call log 501. Further, it is assumed that test scenario 1000 is stored in scenario DB 110 of test apparatus 100 and expected value data 1300 is stored in expected value DB 130.

  In the test apparatus 100, the scenario driving unit 120 reads one line of the test scenario from the scenario DB 110, interprets this, and issues an execution command to the mobile phone device 200. When “Menu menu key“ delete ”” 1001 is read, first, an instruction to press the menu key is transmitted to the mobile phone device 200. In the mobile phone device 200, a command for pressing the menu key is transmitted to the OS 210 via the input unit 220. The OS 210 calls the menu key handler 242 of the application unit 240, and the menu display command 241 is executed by the menu key handler 242. A menu call message is output to the menu call log 501 before the menu display process is executed by the menu display command 241. When the menu display command 241 executes processing, the menu mechanism unit 211 of the OS 210 is called and a focus menu character string is output to the menu log 502. As a result, the menu screen 2000 shown in FIG. 4 is displayed on the display screen of the mobile phone device 200, and the upper “registration” 2001 is focused.

  The scenario driving unit 120 of the test apparatus 100 acquires the menu call log 501 and the menu log 502. Based on the menu log 502, it is detected that the current focus menu character string is “registration”. This focus menu character string “registration” is compared with the command menu character string “delete” of the test scenario “Menu menu key“ delete ”” 1001. Since they do not match here, a focus movement operation command is output to the mobile phone device 200. For example, a lower key operation command is output to the input unit 220. Upon receiving the down key operation command via the input unit 220, the OS 210 performs a process of moving the focus downward, and the menu mechanism unit 211 outputs “delete” to the menu log 502. Use of the call log 501 will be described later.

  The scenario driver 120 collates the focus menu character string “delete” in the menu log 502 with the command menu character string “delete” in the test scenario “Menu menu key“ delete ”” 1001. Here, since they match, a command for operating the enter key is output to the mobile phone device 200 to execute the deletion process.

The cellular phone device 200 receives the determination key operation command, and the application unit 240 executes the deletion process according to the OS 210. Details of the deletion process are omitted.
Thus, the address list on which the deletion process has been executed is displayed on the display screen of the mobile phone device 200.

  Subsequently, the scenario driving unit 120 reads the next line of the test scenario 1000. The “Assert address list” 1002 is read, and the measurement unit 140 acquires the result from the output unit 230. Assume that the result 2300 shown in FIG. 5 is obtained. The expected value DB 130 is searched using the “address list” described in the “Assert address list” 1002 as a key, and the expected value data 1300 is extracted. The content 1303 of the extracted expected value data 1300 is collated with the result 2300 obtained by actually executing the instruction, and it is determined whether or not all the contents match. When they match, the test result in the determination instruction is determined to be acceptable. However, when a location different from the expected value is detected even at one location, the position of the determination command in the test scenario and the mismatched location in the expected value are recorded. The contents are also notified to the person in charge of the test.

  When the contents 1303 of the expected value data 1300 and the result 2300 are collated, the name of the label 3 of the expected value data 1300 is “Uchino”, whereas in the result 2300, it is “Unno”, which does not match. Therefore, the data of the part that does not match is recorded as error information, and an error is notified to the person in charge of the test.

The scenario driver 120 reads the next test scenario from the scenario DB 110 and executes the same procedure.
As described above, the test apparatus 100 focuses the character string displayed on the corresponding menu screen based on the test scenario in which the menu display command and the target focus character string are defined. Move to column.

  In the above description, the down key operation is performed when the same character string as that in the test scenario is not detected, but the operable direction is appropriately selected. For example, if the menu screen is configured to display the menu key in the horizontal direction, the key operation in the other direction is performed when the menu key is not detected by the down key operation.

Here, it is assumed that the specification change of the display screen has occurred. FIG. 6 is a diagram showing a menu screen after the specification is changed. The depth of 1 is the same as before the specification change.
In the menu screen 2100 after the specification change, “Register” 2101, “Edit” 2102, and “Delete” 2103 are prepared as menus. “Edit” 2102 is added to the menu screen 2000 before the specification change shown in FIG.

Since the test scenario 1100 does not change the test contents, the same test scenario 1000 as that before the specification change can be reused.
The (3) automatic test execution process of the test apparatus 100 will be described. Since the processing from when the menu key is pressed until the menu screen 2100 is opened is the same as before the specification change, description thereof will be omitted.

  When the menu screen 2100 after the specification change is opened, the focus menu character string of the menu log 502 output from the mobile phone device 200 is “registered” as before the specification change. The scenario driving unit 120 compares “registration” acquired from the menu log 502 with “deletion” of the test scenario “Menu menu key“ deletion ”1101 and does not match. To do. Assume that a lower key operation command is output to the input unit 220. Upon receiving the down key operation command via the input unit 220, the OS 210 performs a process of moving the focus downward, and the menu mechanism unit 211 outputs “edit” to the menu log 502.

  The scenario driving unit 120 compares “edit” acquired from the menu log 502 with “delete” of the test scenario “Menu menu key“ delete ”” 1101 and does not match. To order. A down key operation command is output to the input unit 220. The OS 210 that receives the down key operation command via the input unit 220 performs a process of moving the focus downward, and the menu mechanism unit 211 outputs “delete” to the menu log 502. Here, since the focus menu character string acquired from the menu log 502 matches the command menu character string of the test scenario, the scenario driving unit 120 outputs a command for operating the determination key to the mobile phone device 200 and performs the deletion process. Let it run. Since the subsequent processing is the same as before the specification change, the description is omitted.

  As described above, the test apparatus 100 collates the focus menu character string with the command menu character string described in the test scenario even if the specification is changed, and finds the character string that matches the command menu character string from the menu screen. Can be detected. For this reason, even if the display order in the menu screen changes, an automatic test can be performed without correcting the test scenario. Thus, since the test scenario can be used as it is even if the model to be tested is changed, the number of items required for the test can be reduced, and the terminal development cost can be reduced and early collection can be achieved.

Next, a case where the menu depth is 2 will be described as a second embodiment. FIG. 7 is a diagram illustrating an example of a menu screen having a depth of 2 and a test scenario.
The menu screen 2200 includes a first menu screen 2210 displayed when the menu key is pressed, and a second menu screen 2220 displayed when “Delete” is selected on the first menu screen. .

  The second menu screen 2220 has menu keys of “cancel” 2221, “one case” 2222, and “all cases” 2223. “Cancel” 2221 is a menu key for selecting an operation to cancel “delete” selected on the first menu screen 2210. “One item” 2222 is a menu key for selecting an operation for deleting one item selected from the address list. “All items” 2223 is a menu key for selecting an operation for deleting all address information registered in the address list.

  The test scenario 1210 is a command for instructing deletion of one line from the address list display screen. “Menu menu key“ Delete. “1 case” ”1201 is that the menu key is pressed to open the first menu screen,“ Delete ”is selected from the first menu screen, and“ 1 ”is selected from the second menu screen. Operation instruction. Also, the “Assert address list” 1202 includes a “Menu menu key“ delete. This is a determination command for determining whether the “address list” screen after the execution of the process of “one case” 1201 is correct and whether it matches with the expected value.

  When the menu screen has a hierarchical structure, the test scenario for selecting an operation from the menu describes the character strings to be selected side by side. In this example, a plurality of character strings in each layer are arranged by separating the layer breaks with a delimiter character “.”. The character string described first is a character string selected on the first layer (a menu screen opened by pressing the menu key). A character string selected on a menu screen (hereinafter referred to as a submenu) that is sequentially expanded after the delimiter “.” Is described. For example, “Delete. 1 item” indicates that the menu key “Delete” is selected on the menu screen 2210 displayed first, and the menu key “1 item” is selected on the submenu 2220 of the next layer. ing. When the menu screen 2210 in the first layer is displayed, the depth of the menu is counted as 1. When the next submenu 2220 is displayed, the menu depth is counted as 2. Here, the case of depth 2 is shown, but the menu itself can be made as deep as possible. The delimiter is not limited to “.”, And can be arbitrarily set according to the system.

A description will be given of (3) automatic test execution processing when the menu screen has a hierarchical structure.
Processing after the menu key 22 is pressed to open the menu screen 2210 will be described.

  In the mobile phone device 200, when the menu screen 2210 is opened, the scenario driving unit 120 of the test device 100 acquires the menu call log 501 and the menu log 502. The menu log 502 indicates that the current focus menu character string is “registration”. The menu call log 501 is added with a menu call date and time, an ID number, etc. to distinguish it from other log messages from the computer. For example, “2008/9/24 10: 30: 41.768 Menu called” In addition to the menu call, the date and time is given in milliseconds so that it can be distinguished from other logs. . The scenario driving unit 120 stores a “depth variable” indicating the hierarchy of the current menu screen in a storage area for storing temporary data. The depth variable is set to “1” when the menu call log 501 indicating that the first menu screen is displayed is acquired. Thereafter, each time the menu call log 501 notifies that the menu hierarchy has been deepened, it is incremented by one.

  The scenario driving unit 120 sets “1” to the depth variable based on the menu call log 501. Then, the registration of the focus menu character string set in the menu log 502 and the test scenario “Menu menu key“ delete. Compared with “delete” of “1 case” ”1201 and does not match, the mobile phone device 200 is instructed to move the focus. Note that the depth menu indicates that the current menu screen is in the first layer, so it is collated with “delete” in the first layer. Subsequently, a lower key operation command is output to the input unit 220. The OS 210 that receives the down key operation command via the input unit 220 performs a process of moving the focus downward, and the menu mechanism unit 211 outputs “delete” to the menu log 502. Since the menu is not called and the state of the depth variable is “1”, the test scenario “Menu menu key“ Delete. The command menu character string “delete” of “1 case” ”1201 is collated. Here, they match. The scenario driving unit 120 outputs a command for operating the determination key to the mobile phone device 200.

  The cellular phone device 200 receives the enter key operation command and displays the submenu 2220 of the next layer. At this time, a new menu call message is output to the menu call log 501 by the menu display command 241. Further, “cancel” is output to the menu log 502 as the focus menu character string.

  The scenario driving unit 120 increments the depth variable by 1 based on the menu call log 501 and sets it to “2”. Based on the depth variable, the test scenario “Menu menu key“ Delete. The command menu character string “1 case” in the second layer of “1 case” ”1201 is compared with the focus menu character string“ cancel ”in the menu log 502. Since they do not match, a lower key operation command is output to the mobile phone device 200 to move the focus. When the next operation is executed, the focus menu character string “1” is obtained from the menu log 502. Since this matches the command menu character string “one case” in the second layer of the test scenario, the determination key operation is instructed to execute the deletion process.

As described above, even when the depth of the menu is 2, it is possible to search for a part to be tested and detect a corresponding menu key.
In the above description, the processing is performed assuming that there is a menu key having a command menu character string described in the test scenario on the menu screen, but it is also assumed that there is no corresponding menu key on the menu screen. . Therefore, the character string first detected as the focus menu character string on the menu screen is stored. If the same character string is detected again as the focus menu character string after performing the focus movement operation, an error is assumed.

  It is also assumed that the test scenario command depth (the number of menu key character strings described in the delimiter character string) does not match the actual menu depth. For example, the test scenario is “Menu Menu Key“ Delete. This is a case where “1” is displayed, but the submenu is not displayed after “Delete” is selected. In such a case, the menu call message in the menu call log 501 is kept waiting. In order to prevent this, a certain waiting time is provided. If the submenu is not displayed after this time has elapsed, an error is assumed.

  Contrary to the above example, it is also assumed that the actual menu depth is deeper than the test scenario command depth. For example, although the test scenario is “Menu menu key“ delete ””, the sub-menu is actually displayed after “delete” is selected. Since deletion is detected, the enter key is operated, but the deletion process is not executed. In order to prevent this, the scenario driving unit 120 monitors the menu call log 501 for a certain period of time, and if the menu call log 501 is called in a hierarchy in which no delimiter is set, an error occurs.

For each of the above errors, different error messages are output so that the contents of the error can be understood.
In this way, it is possible to accurately identify an abnormal state that may occur during execution of the menu command, output a different error message, notify the test executor, and stop the test.

Next, an automatic test processing procedure of the test apparatus 100 will be described using a flowchart.
FIG. 8 is a flowchart showing the processing procedure of the automatic test.

An automatic test execution for a test case is instructed, and the process is started.
[Step S11] When the process is started, first, a “depth variable” indicating how many levels the current menu screen is is initialized to zero. The “depth variable” is temporarily stored in the storage means.

  [Step S12] One line of the test scenario of the target test case stored in the scenario DB 110 is read. At this time, the test scenario with the earliest execution order among the unexecuted test scenarios is read out.

  [Step S13] It is determined whether or not the test scenario read in step S12 is the last line. In the last line, for example, a command for instructing the end of the test, such as “end” and “result display”, is described. If it is the last row, the process proceeds to step S14. If it is not the last line, the process proceeds to step S15.

[Step S14] Since the row read in step S12 is the last row, the test result is displayed on the display screen, and the test of this test case is completed.
[Step S15] When the line read in step S12 is not the last line, it interprets what command the read test scenario is.

  [Step S16] In step S15, it is determined whether the command interpreted from the test scenario is an operation command. If it is determined as an operation command, the process proceeds to step S17. If it is not determined to be an operation command, the process proceeds to step S20.

  [Step S17] When it is determined that the command is a move operation command, it is determined whether the command is a menu display command. Here, it is determined whether or not the command of the read test scenario is “Menu”. If it is a menu display command, the process proceeds to step S18 to execute the menu process. If it is not a menu display command, the process proceeds to step S19, and the menu process is not executed.

  [Step S18] When the read command is a menu display command, a menu screen is displayed and a menu key displayed on the menu display screen is selected. Of the displayed menu keys, the focus is moved to the menu key corresponding to the command menu character string of the menu display command. Details of the menu execution process will be described later. By the menu execution process, the focus is set to a menu key that instructs execution of a desired operation. After the completion, the process returns to step S12, and the next test scenario line is processed.

  [Step S19] An execution command for processing described in the test scenario is output to the target mobile phone device 200. After the completion, the process returns to step S12, and the next test scenario line is processed.

[Step S20] When the command is not determined to be an operation command, since it is a determination command, the measurement unit 140 obtains output information.
[Step S21] Expected value data corresponding to the test item described in the test scenario is acquired from the expected value DB.

  [Step S22] The measurement result acquired in step S20 is collated with the expected value data acquired in step S21. If all match, this test item is judged to pass. If even one location does not match, this test item is determined to be an error. After the determination, the process returns to step S12, and the next line of the test scenario is processed.

By executing the above processing procedure, an operation command and a determination command are sequentially executed based on the test scenario, and an automatic test of the mobile phone device 200 is executed.
Next, details of the menu execution process will be described.

FIG. 9 is a flowchart showing a procedure of menu execution processing.
It is executed when the read test scenario is a menu display command.
[Step S101] An operation command for pressing a menu key for calling a menu is output to the mobile phone device 200. In the mobile phone device 200, when the menu display command 241 of the application unit 240 is executed, a new menu call message is output to the menu call log 501. Further, the menu mechanism unit 211 outputs a focus menu character string focused at the present time as the menu log 502.

  [Step S102] The scenario drive unit 120 monitors the output of the menu call log 501 and detects that a menu call message has been newly output. When a new menu call message is detected, 1 is added to the depth variable.

[Step S <b> 103] The scenario driving unit 120 extracts a focus menu character string from the menu log 502.
[Step S104] Based on the depth variable, a command menu character string set in the corresponding menu hierarchy is acquired from the test scenario. When the depth variable is “1”, the command menu character string described first is read. When the depth variable is “2”, the next command menu character string delimited by the delimiter “.” Is read.

  [Step S105] The focus menu character string acquired in step S103 is collated with the command menu character string acquired in step S104. If it is determined that they match, the process proceeds to step S106. If not, the process proceeds to step S109.

  [Step S106] When the focus menu character string matches the command menu character string, the target menu character string is focused, and therefore an operation command for pressing the enter key is output to the mobile phone device 200.

  [Step S107] It is determined whether or not the depth variable matches the command depth variable. The depth variable is a value counted every time a new menu call is recorded in the menu call log 501 in step S102. The command depth variable is a depth variable extracted from the test scenario command. Since the command menu character string is delimited by a delimiter for each hierarchy, the number of command menu character strings corresponds to the command depth variable. If the depth variable matches the command depth variable, the process proceeds to step S108. If they do not match, the process returns to step S102, waits for the next level menu to be called, and performs processing from monitoring the menu call log 501.

  [Step S108] When the depth variable matches the command depth variable, the depth variable is reset to zero. Since the focus menu character string matches the command menu character string, the processing is ended in order to start the processing for the focus menu character string on the displayed menu screen.

  [Step S109] If the focus menu character string does not match the command menu character string, a process of pressing an arrow key for moving the focus is output to the mobile phone device 200. Then, the process returns to step S103 to perform processing from the focus menu character string extraction.

By executing the above processing procedure, the command menu character string corresponding to the target operation described in the test scenario is focused.
The processing procedure shown in FIG. 9 is based on the premise that a character string corresponding to the command menu character string always exists on the menu screen. However, it is assumed that it may not exist. Therefore, a menu execution process including a process when a command menu character string is not detected on the menu screen will be described.

FIG. 10 is a flowchart showing the procedure of the message execution process taking into account the case where the command menu character string does not exist.
It is executed when the read test scenario is a focus movement operation command.

  [Step S201] An operation command for pressing a menu key for calling a menu is output to the mobile phone device 200. In the mobile phone device 200, when the menu display command 241 of the application unit 240 is executed, a new menu call message is output to the menu call log 501. Further, the menu mechanism unit 211 outputs a focus menu character string focused at the present time as the menu log 502.

  [Step S202] The scenario driver 120 monitors the output of the menu call log 501 and detects that a menu call message has been newly output. When a new menu call message is detected, 1 is added to the depth variable.

  [Step S203] When a menu call message is newly detected, the top menu character string is reset and initialized. The first menu character string is a focus menu character string that is first detected on the menu screen when a new menu screen is displayed. It is stored in the storage means as temporary information.

[Step S <b> 204] The scenario driving unit 120 extracts a focus menu character string from the menu log 502.
[Step S205] When the menu screen stored in the first menu character string is opened, the focus menu character string detected first and the current focus menu character string detected in step S204 are collated and matched. It is determined whether or not. If they match, it is determined that the corresponding character string has not been detected on the menu screen even after the search for the command menu character string has been completed. Therefore, it is determined that “the menu corresponding to the command menu character string does not exist”, and the process proceeds to step S206. If not, the process proceeds to step S207.

  [Step S206] When the focus menu character string matches the first menu character string, the test result is set to failure, the automatic test process is interrupted, and the process proceeds to branch point A shown in FIG. In the subsequent processing, a notification that “the menu corresponding to the command menu character string does not exist” is output as the error reason, and the error occurrence in the test scenario is displayed on the display screen or the like.

  [Step S207] If the focus menu character string does not match the first menu character string, and the first menu character string is blank, the focus menu character string acquired in step S204 is set as the first menu character string.

[Step S208] Based on the depth variable, a command menu character string set in the corresponding menu hierarchy is acquired from the test scenario.
[Step S209] The focus menu character string acquired in step S203 and the command menu character string acquired in step S208 are collated. If it is determined that they match, the process proceeds to step S210. If not, the process proceeds to step S213.

  [Step S210] When the focus menu character string matches the command menu character string, the target menu key is focused, and therefore an operation command for pressing the enter key is output to the mobile phone device 200.

  [Step S211] It is determined whether or not the depth variable matches the command depth variable. If the depth variable matches the command depth variable, the process proceeds to step S212. If they do not match, the process returns to step S202, and processing from monitoring the menu call log 501 waiting for the next level menu to be called is performed.

  [Step S212] When the depth variable matches the command depth variable, the depth variable is reset to zero. Since the focus menu character string matches the command menu character string, the processing is ended in order to start the processing for the focus menu character string on the displayed menu screen.

  [Step S213] When the focus menu character string does not match the command menu character string, a process of pressing an arrow key for moving the focus is output to the mobile phone device 200. Then, the process returns to step S204 to perform processing from extraction of the focus menu character string.

  By executing the above processing procedure, in addition to the menu execution process shown in FIG. 9, if a menu key that matches the command menu character string is not searched even if the menu key is cycled on the menu screen, an error test is executed. Will be notified. This can prevent the command menu character string from being searched indefinitely.

Also, the test scenario command depth may not match the actual menu depth.
FIG. 11 is a flowchart showing the procedure of message execution processing taking into account the case where the command depth of the test scenario is larger than the menu.

  It is executed when the read test scenario is a focus movement operation command. [Step S301] An operation command for pressing a menu key for calling a menu is output to the mobile phone device 200. Also, a timer is started and a waiting time until a menu call message is output is set. In the mobile phone device 200, when the menu display command 241 of the application unit 240 is executed, a new menu call message is output to the menu call log 501. Further, the menu mechanism unit 211 outputs a focus menu character string focused at the present time as the menu log 502.

  [Step S302] It is determined whether the waiting time set in step S301 has elapsed. If the waiting time has elapsed, the process proceeds to step S303. If the waiting time has not elapsed, the process proceeds to step S304.

  [Step S303] When the waiting time has elapsed, it is determined that the menu screen of the next layer is not displayed. The test result is set to failure, the automatic test process is interrupted, and the process proceeds to branch point A shown in FIG. In the subsequent processing, a notification that “the menu is not displayed” is output as the error reason, and the error occurrence in the test scenario is displayed on a display screen or the like.

  [Step S304] When the waiting time has not elapsed, the scenario driving unit 120 monitors the output of the menu call log 501 and detects that a menu call message has been newly output. When a new menu call message is detected, 1 is added to the depth variable.

  [Step S <b> 305] When a menu call message is newly detected from the menu call log 501, the scenario driving unit 120 extracts a focus menu character string from the menu log 502.

[Step S306] Based on the depth variable, a command menu character string set in the corresponding menu hierarchy is acquired from the test scenario.
[Step S307] The focus menu character string acquired in step S305 and the command menu character string acquired in step S306 are collated. If it is determined that they match, the process proceeds to step S308. If it is determined that they do not match, the process proceeds to step S311.

  [Step S308] When the focus menu character string matches the command menu character string, the target menu key is focused, and therefore an operation command for pressing the enter key is output to the mobile phone device 200.

  [Step S309] It is determined whether or not the depth variable matches the command depth variable. If the depth variable matches the command depth variable, the process proceeds to step S310. If they do not match, the process returns to step S302 to perform processing from monitoring the menu call log 501 waiting for the next hierarchy menu to be called.

  [Step S310] When the depth variable matches the command depth variable, the depth variable is reset to zero. Since the focus menu character string coincides with the command menu character string, the process ends to start the processing for the menu key that is focused.

  [Step S311] If the focus menu character string does not match the command menu character string, a process of pressing an arrow key for moving the focus is output to the mobile phone device 200. Then, the process returns to step S305 to perform processing from the focus menu character string extraction.

  By executing the above processing procedure, in addition to the menu execution process shown in FIG. 9, when the test scenario command depth is greater than the menu depth, an error is notified to the test executor. Thereby, it is possible to prevent the scenario driving unit 120 from waiting for the menu screen of the next layer.

Conversely, the test scenario command depth may be smaller than the menu.
FIG. 12 is a flowchart showing the procedure of message execution processing taking into account the case where the command depth of the test scenario is smaller than the menu.

It is executed when the read test scenario is a focus movement operation command.
[Step S401] An operation command for pressing a menu key for calling a menu is output to the mobile phone device 200. In the mobile phone device 200, when the menu display command 241 of the application unit 240 is executed, a new menu call message is output to the menu call log 501. Further, the menu mechanism unit 211 outputs a focus menu character string focused at the present time as the menu log 502.

  [Step S402] The scenario driver 120 monitors the output of the menu call log 501 and detects that a menu call message has been newly output. When a new menu call message is detected, 1 is added to the depth variable.

[Step S403] The scenario drive unit 120 extracts a focus menu character string from the menu log 502.
[Step S404] Based on the depth variable, a command menu character string set in the corresponding menu hierarchy is acquired from the test scenario.

  [Step S405] The focus menu character string acquired in step S403 is collated with the command menu character string acquired in step S404. If it is determined that they match, the process proceeds to step S406. If not, the process advances to step S412.

  [Step S406] When the focus menu character string matches the command menu character string, the target menu character string is focused, and therefore an operation command for pressing the enter key is output to the mobile phone device 200.

  [Step S407] It is determined whether or not the depth variable matches the command depth variable. If the depth variable matches the command depth variable, the process proceeds to step S408. If they do not match, the process returns to step S402 to perform processing from monitoring the menu call log 501 that waits for the next hierarchical menu to be called.

  [Step S408] When the depth variable matches the command depth variable, the menu call log 501 is monitored to check whether a new call message has been detected. If detected, the process proceeds to step S409. If not detected, the process proceeds to step S410.

  [Step S409] If a menu call message is detected even though the depth variable matches the command depth variable, it is determined that there is a menu screen of the next layer not described in the command. The test result is set to failure, the automatic test process is interrupted, and the process proceeds to branch point A shown in FIG. In the subsequent processing, a notification that “the menu depth is deeper than the command” is output as the error reason, and the error occurrence in the test scenario is displayed on a display screen or the like.

  [Step S410] When the depth variable matches the command depth variable, and no menu call message is detected, it is determined whether or not a predetermined time has elapsed from the operation command for pressing the enter key. If the certain time has elapsed, the process proceeds to step S411. If the predetermined time has not elapsed, the process returns to step S408, and the process from monitoring the menu call message is performed.

  [Step S411] Since a new menu call message is not detected even after a certain period of time has elapsed, it is determined that the menu depth matches the command depth. Then, the depth variable is reset to zero. In addition, since the focus menu character string matches the command menu character string, the processing is ended in order to start processing for the focus menu character string on the displayed menu screen.

  [Step S412] If the focus menu character string does not match the command menu character string, a process of pressing an arrow key for moving the focus is output to the mobile phone device 200. Then, the process returns to step S403 to perform processing from the focus menu character string extraction.

  By executing the above processing procedure, in addition to the menu execution process shown in FIG. 9, when the test scenario command depth is smaller than the menu depth, an error is notified to the test executor. As a result, it is possible to prevent the scenario driving unit 120 from proceeding to the menu screen of the next hierarchy, but performing processing as the menu screen of the previous hierarchy.

In the above description, each error detection process has been described individually. However, error detection may be performed by appropriately combining them.
The above processing functions can be realized by a computer. In that case, a program describing the processing contents of the functions that the test apparatus should have is provided. By executing the program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium.

  When distributing the program, for example, portable recording media such as a DVD (Digital Versatile Disc) and a CD-ROM (Compact Disc Read Only Memory) on which the program is recorded are sold. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Further, each time the program is transferred from the server computer, the computer can sequentially execute processing according to the received program.

Regarding the above embodiment, the following additional notes are disclosed.
(Supplementary note 1) In a test program for testing based on a test scenario that describes the test procedure of the target device,
Computer
The test scenario from a scenario storage means for storing a test scenario group including a test scenario in which a move operation command for moving the focus of the component on the display screen to the target component is described in association with the identification information of the target component Execution means for causing the target device to execute an operation based on the test scenario,
An operation result acquisition unit that acquires, from the target device, operation result information including identification information of a movement-destination part that has been focused after the execution of the movement operation when the execution unit performs the focus movement operation;
Determining whether or not the identification information of the destination component included in the operation result information acquired by the operation result acquisition means matches the identification information of the target component described in the test scenario; When it is determined that they do not match, the target device is caused to execute the focus movement operation, and the identification information of the destination part included in the operation result information acquired via the operation result acquisition unit and the Re-execution means for repeating the procedure of collating with the identification information of the target part, and matching the identification information of the destination part with the identification information of the target part;
A test program characterized by functioning as

(Supplementary Note 2) The execution unit and the re-execution unit output an operation command corresponding to pressing of an operation key that instructs an operation mounted on the target device to the target device.
The test program according to supplementary note 1, characterized by:

(Supplementary Note 3) The target device is built on an emulator that realizes at least a part of the functions of the target device on a computer.
The test program according to supplementary note 1, characterized by:

(Supplementary Note 4) The target component to which an operation command including the movement operation command is associated is a menu key arranged in a menu screen displayed on the display screen by a menu display operation command,
The execution means reads the menu display operation command from the test scenario in which the menu display operation command and a character string on the menu key corresponding to a target operation are described. A menu display operation command is performed, and after the menu key corresponding to a target operation is focused by the re-execution unit, an operation execution command corresponding to the menu key is output to the target device,
The re-execution unit includes notification information for notifying that the menu display operation command acquired as the operation result information by the operation result acquisition unit is executed, and the menu key focused after the menu display operation command is executed. A focus menu character string, and when it is detected that the menu display operation command is executed based on the notification information, the focus menu character string and the target menu described in the test scenario Determine whether or not the character string of the key matches, and instruct the target device to move the focus until it matches,
The test program according to supplementary note 1, characterized by:

(Additional remark 5) The said computer acquires the program of the said object apparatus, analyzes it, detects the menu display command in the said program which displays the said menu screen according to the said menu display operation command, The said menu display command A depth output injection means for adding a notification process for outputting a call notification of the menu screen each time the menu screen is called, and transferring the program to which the notification process is added to the target device;
The test program according to appendix 4, wherein the test program is made to function as:

(Supplementary Note 6) The menu screen includes the menu screen from the first menu screen first displayed by the menu display operation command described in the test scenario to the menu screen on which the target menu key is displayed. When taking a hierarchical structure called by operating a predetermined menu key on the screen, the test scenario is based on the calling order from the first menu screen to the menu screen on which the target menu key is displayed. The character string on the menu key that calls the menu screen of the next hierarchy is described in the calling order by separating it with a predetermined delimiter,
The execution means collates the depth of the menu screen based on the notification information that is output every time the menu screen is called and switched, and the depth of the menu screen based on the description of the test scenario, and is displayed. The test program according to appendix 4, wherein the menu screen confirms whether the menu screen is the menu screen on which the target menu key is displayed.

(Supplementary Note 7) The re-execution unit stores the focus menu character string detected first when the notification information output every time the menu screen is switched is stored as a first menu character string, and The focus menu character string acquired until the menu screen is switched and the first menu character string are compared, and if they match, it is determined that the target menu key is not detected on the menu screen, and processing is performed. Cancel and notify you of any errors that occur,
The test program according to supplementary notes 4 to 6, characterized in that.

(Supplementary Note 8) The re-execution means cannot obtain the menu screen call notification from the target device even after a predetermined waiting time has elapsed since the menu key to be called by the menu screen at another level has been selected. When it is determined that there is no error in the menu screen, the process is stopped and the error that has occurred is notified.
The test program according to supplementary notes 4 to 6, characterized in that.

(Supplementary Note 9) The re-execution unit monitors the output of the call notification of the menu screen until a predetermined time has elapsed after the target menu key is selected, and when the call notification of the menu screen is acquired, Determining that the depth of the menu screen is deeper than the depth of the menu screen described in the test scenario, canceling the process, and notifying the error that has occurred,
The test program according to supplementary notes 4 to 6, characterized in that.

(Supplementary Note 10) In a test apparatus that performs a test based on a test scenario that describes a test procedure of a target device,
A scenario storage means for storing a test scenario group including a test scenario in which a movement operation command for moving the focus of a component on the display screen to the target component is described in association with the identification information of the target component;
Execution means for reading the test scenario from the scenario storage means and causing the target device to execute an operation based on the test scenario;
When the focus movement operation is executed by the execution means, an operation result acquisition means for acquiring operation result information including identification information of a moving destination component focused after the movement operation is performed,
Determining whether or not the identification information of the destination component included in the operation result information acquired by the operation result acquisition means matches the identification information of the target component described in the test scenario; When it is determined that they do not match, the target device is caused to execute the focus movement operation, and the identification information of the destination part included in the operation result information acquired via the operation result acquisition unit and the A re-execution unit that repeats the procedure of collating the identification information of the target part and reconciles the identification information of the destination part and the identification information of the target part;
A test apparatus characterized by comprising:

10 Test equipment 11 Scenario database (DB)
DESCRIPTION OF SYMBOLS 12 Execution means 13 Operation result acquisition means 14 Re-execution means 15 Depth output injection means 20 Embedded equipment 21 Application processing part 22 Input means 23 Control means 24 Result output means

Claims (5)

In a test program that tests based on a test scenario that describes the test procedure of the target device,
Computer
The test scenario from a scenario storage means for storing a test scenario group including a test scenario in which a move operation command for moving the focus of the component on the display screen to the target component is described in association with the identification information of the target component Execution means for causing the target device to execute an operation based on the test scenario,
An operation result acquisition unit that acquires, from the target device, operation result information including identification information of a movement-destination part that has been focused after the execution of the movement operation when the execution unit performs the focus movement operation;
Determining whether or not the identification information of the destination component included in the operation result information acquired by the operation result acquisition means matches the identification information of the target component described in the test scenario; When it is determined that they do not match, the target device is caused to execute the focus movement operation, and the identification information of the destination part included in the operation result information acquired via the operation result acquisition unit and the Re-execution means for repeating the procedure of collating with the identification information of the target part, and matching the identification information of the destination part with the identification information of the target part;
A test program characterized by functioning as The target component associated with the operation instruction including the movement operation instruction is a menu key arranged in a menu screen displayed on the display screen by a menu display operation instruction,
The execution means reads the menu display operation command from the test scenario in which the menu display operation command and a character string on the menu key corresponding to a target operation are described. A menu display operation command is performed, and after the menu key corresponding to a target operation is focused by the re-execution unit, an operation execution command corresponding to the menu key is output to the target device,
The re-execution unit includes notification information for notifying that the menu display operation command acquired as the operation result information by the operation result acquisition unit is executed, and the menu key focused after the menu display operation command is executed. A focus menu character string, and when it is detected that the menu display operation command is executed based on the notification information, the focus menu character string and the target menu described in the test scenario Determine whether or not the character string of the key matches, and instruct the target device to move the focus until it matches,
The test program according to claim 1, wherein: The menu screen has a predetermined range on the menu screen from the first menu screen first displayed by the menu display operation command described in the test scenario to the menu screen on which the target menu key is displayed. When a hierarchical structure called by operating the menu key is taken, the next hierarchy is determined based on the calling order from the first menu screen to the menu screen on which the target menu key is displayed in the test scenario. The character string on the menu key for calling the menu screen is described in the calling order by separating it with a predetermined delimiter,
The execution means collates the depth of the menu screen based on the notification information that is output every time the menu screen is called and switched, and the depth of the menu screen based on the description of the test scenario, and is displayed. 3. The test program according to claim 2, wherein it is confirmed whether or not the menu screen is the menu screen on which the target menu key is displayed. The re-execution means stores the focus menu character string detected first when the notification information output every time the menu screen is switched is acquired as a first menu character string, and the menu screen is switched. The focus menu character string acquired up to and the first menu character string are compared, and when they match, it is determined that the target menu key is not detected on the menu screen, and the process is stopped, Notify the error that occurred,
The test program according to claim 2 or 3, wherein In test equipment that performs tests based on test scenarios that describe the test procedure of the target device,
A scenario storage means for storing a test scenario group including a test scenario in which a movement operation command for moving the focus of a component on the display screen to the target component is described in association with the identification information of the target component;
Execution means for reading the test scenario from the scenario storage means and causing the target device to execute an operation based on the test scenario;
When the focus movement operation is executed by the execution means, an operation result acquisition means for acquiring operation result information including identification information of a moving destination component focused after the movement operation is performed,
Determining whether or not the identification information of the destination component included in the operation result information acquired by the operation result acquisition means matches the identification information of the target component described in the test scenario; When it is determined that they do not match, the target device is caused to execute the focus movement operation, and the identification information of the destination part included in the operation result information acquired via the operation result acquisition unit and the A re-execution unit that repeats the procedure of collating the identification information of the target part and reconciles the identification information of the destination part and the identification information of the target part;
A test apparatus characterized by comprising:

Images