JP6198529B2 - Test execution system, test execution device, test execution method, and test execution program - Google Patents

Description

  The present invention relates to a test execution system, a test execution apparatus, and a test execution that discriminate a test result by converting the behavior of the product into response information and receiving the response information in the test of the product that behaves based on the operation of the tester. The present invention relates to a method and a test execution program.

Conventionally, in performing a product test, it is common for the tester to check the behavior of the product with respect to the operation by the tester and to make a pass / fail decision.
For the operation by the tester, the operation procedure is clearly set in the test procedure manual, and the tester communicates the operation to the product using a transmitter or the like.
In the pass / fail judgment, the expected value for judgment is set in the test procedure document in advance, and the tester confirms whether the behavior of the product matches the expected value for judgment set in the test procedure document. As a result, pass / fail is determined.

  The test apparatus disclosed in Patent Document 1 is an invention of an apparatus that performs an operation test for determining whether or not an electronic component serving as a peripheral simulator operates correctly. Therefore, when performing an operation test of electronic components and software incorporated as a product, it is necessary to perform the test targeting the product.

  The test apparatus shown in Patent Document 2 is an invention for automating a repetitive test for the purpose of a reproduction test. It is characterized by an action log, and a predetermined error is stored as log information, the result is determined based on whether or not an error has occurred, and the behavior result is not determined.

JP-A-9-091031 JP 2007-52734 A

  In the test apparatus shown in Patent Document 1, a test is performed by supplying a simulated input signal to the control apparatus to determine whether the output electronic component as a peripheral simulator operates. Since the test results are judged from the output of the electronic component, it does not appear in the output of the electronic component, and the test of the behavior of the product cannot be judged. Become.

  The test apparatus disclosed in Patent Document 2 describes a reproduction test of an actual machine test in a product when a product failure occurs. The product is operated correctly according to the test procedure in the actual machine test in the product. There is no automation.

  The present invention has been made to solve the above-described problems, and automates the behavior test according to the test procedure manual with respect to the actual machine test in the product, thereby reducing the number of test steps without reducing the number of test items. The purpose is to do. Another object of the present invention is to automatically generate a test procedure document for the purpose of reducing test man-hours and to reduce the man-hours required for tests in test design.

A test execution system according to the present invention includes a target device that is a test target, receives a command, operates based on the received command, and a test execution device that executes a test on the target device. In the test execution system provided,
The test execution device includes:
The operation correspondence that stores the command to be transmitted to the target device and the expected operation information indicating the operation of the target device expected when the target device receives the command in association with each other and stores it in the storage device as the operation correspondence information An information storage unit;
A command transmission unit that retrieves the command from the motion correspondence information stored in the motion correspondence information storage unit and transmits the retrieved command to the target device by a transmission device;
The target device is
A command receiver for receiving the command from the command transmitter of the test execution device;
An operation execution unit that executes an operation based on the command received by the command receiving unit;
An operation information transmission unit that transmits operation information indicating the operation executed by the operation execution unit to the test execution device;
The test execution device further includes:
An operation information receiving unit that receives the operation information from the operation information transmitting unit of the target device;
The expected motion information associated with the command transmitted by the command transmission unit is extracted from the motion response information stored in the motion response information storage unit, and the retrieved expected motion information and the motion information reception Based on the operation information received by the unit, the processing device determines whether the operation indicated by the operation information is an operation of the target device expected when the target device receives the command. An expected motion determination unit is provided.

  According to the test execution system of the present invention, the instruction transmission unit extracts an instruction from the operation correspondence information stored in the operation correspondence information storage unit, transmits the extracted instruction to the target device by the transmission device, and an expected operation determination unit Whether or not the operation indicated by the operation information is expected operation of the target device when the target device receives the command based on the expected operation information and the operation information transmitted from the target device. Since the determination is made by the processing device, the test of the operation according to the command is automated, and the test man-hour can be reduced without reducing the number of test items.

1 is a diagram illustrating an example of a test automation system 500 according to Embodiment 1. FIG. 1 is a block diagram showing a configuration of a test automation system 500 according to Embodiment 1. FIG. 2 is a diagram illustrating an example of a hardware configuration of a product 100 and a test automation apparatus 200 according to Embodiment 1. FIG. 4 is a flowchart showing a flow of a test automation method of the test automation system 500 according to the first embodiment. 1 is an internal configuration diagram of a test automation apparatus 200 according to Embodiment 1. FIG. 3 is a configuration diagram of a test procedure manual 210 according to Embodiment 1. FIG. 2 is an internal configuration diagram of a product 100 according to Embodiment 1. FIG. 6 is a diagram illustrating abstraction of behavior information by the information output unit 1132 according to Embodiment 1. FIG. FIG. 6 is a block diagram illustrating a configuration of a test automation apparatus 201 according to a second embodiment. It is a figure which shows the example of modeling of preparation of the test procedure manual 210 concerning Embodiment 3. FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the description of each embodiment, the directions such as “up”, “down”, “left”, “right”, “front”, “back”, “front”, “back” are However, it is not intended to limit the arrangement or orientation of devices, instruments, parts, or the like.

Embodiment 1 FIG.
FIG. 1 is a diagram illustrating an example of a test automation system 500 according to the present embodiment.
The test automation system 500 (test execution system) according to the present embodiment is a system for automating the test of the product 100 (target device).

The test automation system 500 includes a product 100 to be tested and a test automation device 200 (test execution device) that executes a test of the product 100. As shown in FIG. 1, the product 100 is, for example, a television, a DVD recorder, or the like.
The product 100 receives the operation information (command) from the test automation apparatus 200, performs a behavior (operation) according to the operation information, and performs a behavior result (behavior information (operation information)) on the test automation apparatus 200. Tell.
The test automation apparatus 200 analyzes the behavior information transmitted from the product 100, determines whether the behavior is correct, and outputs a test result.

  In FIG. 1, the test automation apparatus 200 is composed of a computer device and a transmitter. The motherboard and the transmitter built in the computer device are connected by, for example, a wired connection. The motherboard incorporated in the computer device and the product 100 are connected by, for example, a wired connection. The configuration of the test automation system 500 is not limited to this, and the computer device and the transmitter may be integrated. The connection between the computer device and the transmitter and the connection between the computer device and the product 100 may be any communication method such as wireless communication as long as information can be transmitted and received.

FIG. 2 is a block diagram showing the configuration of the test automation system 500 according to this embodiment.
The configuration of the test automation system 500 according to this embodiment and the function of each component will be described with reference to FIG.

The test automation system 500 includes a product 100 and a test automation device 200.
The product 100 includes an operation input interface unit 115, an operation execution unit 116, a behavior information output unit 113, and a behavior output interface unit 114.

  The operation input interface unit 115 executes interface processing when inputting the operation information 1e from the test automation apparatus 200 to H / W (hardware) or S / W (software) to be tested. The operation input interface unit 115 is an example of a command receiving unit that receives the operation information 1e (command) from the automatic operation unit 220 of the test automation apparatus 200.

  The operation execution unit 116 executes an operation based on the operation information 1e received by the operation input interface unit 115. The operation execution unit 116 is composed of H / W (hardware) and S / W (software) to be tested.

The behavior information output unit 113 outputs behavior information that is a result of the behavior of the test target.
The behavior output interface unit 114 receives the behavior information that is the result of the behavior of the test target from the behavior information output unit 113, executes interface processing when transmitting the behavior information to the test automation device 200, and transmits the behavior information to the test automation device 200. Send behavior information of. The behavior output interface unit 114 is an example of an operation information transmission unit that transmits behavior information 1f (operation information) indicating the behavior (operation) executed by the operation execution unit 116 to the test automation apparatus 200.

The test automation apparatus 200 includes a test procedure manual 210, an automatic operation unit 220, a behavior input interface unit 230, a behavior input determination unit 240, an expected operation interpretation unit 250, a result output unit 260, and a procedure manual creation unit 270.
The automatic operation unit 220 includes an operation interpretation unit 221 and a transmitter 222.

  The test procedure manual 210 associates a test procedure portion indicating an operation procedure for the product 100 with an expected value portion (expected operation information) indicating a behavior expected when the product 100 receives the operation procedure. Is stored in The test procedure manual 210 is an example of operation correspondence information, and is stored in the storage device by the operation correspondence information storage unit.

The automatic operation unit 220 extracts the test procedure part (operation) from the test procedure document 210 stored in the operation correspondence information storage unit, and transmits operation information based on the extracted test procedure part (operation) to the transmitter 222 (transmission device). ) Is an example of a command transmission unit that transmits to the product 100.
The automatic operation unit 220 generates an operation information 1e indicating an operation on the product 100 based on the test procedure part (operation) of the test procedure manual 210 and the time information on the test procedure part (operation). 221.

  The operation configured by the transmitter 222, the operation interpretation unit 221, and the test procedure document 210 is transmitted to the product 100 (test target) as operation information 1e, and the operation information 1e to the product 100 is an operation input incorporated in the product 100. The information is transmitted to the inside of the product 100 via the interface unit 115, and the product 100 performs a behavior corresponding to the operation information 1e. The behavior of the product 100 is transmitted to the behavior information output unit 113, and the behavior information 1 f is transmitted to the test automation apparatus 200 via the behavior output interface unit 114.

The behavior information 1 f is transmitted to the behavior input determination unit 240 via the behavior input interface unit 230 of the test automation apparatus 200. The behavior input interface unit 230 is an example of an operation information receiving unit that receives the behavior information 1 f from the behavior output interface unit 114 of the product 100.
The behavior input determination unit 240 receives the behavior information 1 f from the behavior input interface unit 230 and transmits the behavior information 1 f to the expected operation interpretation unit 250.

  The expected action interpretation unit 250 takes out the expected value part associated with the operation information 1e transmitted from the automatic operation part 220 from the test procedure manual 210 stored in the action correspondence information storage part, and extracts the expected value part Based on the received behavior information 1f, the processing device determines whether the operation indicated by the behavior information 1f is the behavior expected when the operation information 1e transmitted by the product 100 is received. That is, the expected behavior of the test procedure document 210 is transmitted as an expected value to the expected behavior interpretation unit 250, and the expected behavior interpretation unit 250 automatically collates the behavior with the expected value. The expected action interpretation unit 250 is an example of an expected action determination unit.

The result output unit 260 outputs the result of the automatic verification in the expected action interpretation unit 250.
The procedure manual creation unit 270 automatically generates the test procedure manual 210. The procedure manual creation unit 270 may be omitted.
As described above, in the test automation system 500, the test from the test operation to the result confirmation is automatically executed.

FIG. 3 is a diagram illustrating an example of a hardware configuration of the product 100 and the test automation apparatus 200 according to the present embodiment.
A hardware configuration example of the product 100 and the test automation apparatus 200 will be described with reference to FIG.

  The product 100 and the test automation device 200 include a computer, and each element of the product 100 and the test automation device 200 can be realized by a program.

  As hardware configurations of the product 100 and the test automation apparatus 200, an arithmetic device 901, an external storage device 902, a main storage device 903, a communication device 904, and an input / output device 905 are connected to the bus.

The arithmetic device 901 is a CPU (Central Processing Unit) that executes a program.
The external storage device 902 is, for example, a ROM (Read Only Memory), a flash memory, or a hard disk device.
The main storage device 903 is a RAM (Random / Access / Memory).

  The communication device 904 is, for example, a communication board or the like, and is connected to a LAN (Local / Area / Network) or the like. The communication device 904 is not limited to a LAN, but includes an IP-VPN (Internet / Protocol / Virtual / Private / Network), a wide-area LAN, an ATM (Asynchronous / Transfer / Mode) network, a WAN (Wide / Area / Network), or the Internet. It does not matter if it is connected to. LAN, WAN, and the Internet are examples of networks.

  The input / output device 905 is, for example, a mouse, a keyboard, a display device, or the like. Instead of the mouse, a touch panel, touch pad, trackball, pen tablet, or other pointing device may be used. The display device may be an LCD (Liquid / Crystal / Display), a CRT (Cathode / Ray / Tube), or another display device.

The program is normally stored in the external storage device 902, and is loaded into the main storage device 903 and sequentially read into the arithmetic device 901 and executed.
The program is a program that realizes a function described as “unit” shown in FIG.
Further, an operating system (OS) is also stored in the external storage device 902. At least a part of the OS is loaded into the main storage device 903, and the computing device 901 executes the OS while displaying “˜ part” shown in FIG. ”Is executed.
An application program is also stored in the external storage device 902, and is sequentially executed by the arithmetic device 901 while being loaded in the main storage device 903.
Information such as “˜table” is also stored in the external storage device 902.

In the description of the present embodiment, “determining”, “determining”, “extracting”, “detecting”, “setting”, “registering”, “selecting” ”,“ Generate ”,“ Input of ”,“ Output of ”, etc., information, data, signal values, and variable values indicating the results of the processing are stored in the main storage device 903 as files. Yes.
In addition, data received by the product 100 and the test automation device 200 is stored in the main storage device 903.
Further, the encryption key / decryption key, random number value, and parameter may be stored in the main storage device 903 as a file.

  3 is merely an example of the hardware configuration of the product 100 and the test automation device 200, and the hardware configuration of the product 100 and the test automation device 200 is not limited to the configuration illustrated in FIG. Other configurations may be used.

FIG. 4 is a flowchart showing a flow of a test automation method (test execution method) of the test automation system 500 according to the present embodiment.
The processing operation of the test automation system 500 will be described with reference to FIG. The test automation method of the test automation system 500 is realized by executing a test automation program (test execution program) using hardware resources such as the product 100 and the test automation apparatus 200, for example.

In S100, a procedure manual creation process by the procedure manual creation unit 270 is executed. The procedure manual creation process will be described later. Here, it is assumed that the test procedure document 210 is created in advance and stored in the operation correspondence information storage unit.
In S101, the test of the product 100 is started.
In S102, a test procedure manual interpretation process is executed.
The test procedure manual interpretation process will be described below.

<Test Procedure Interpretation Processing: S102>
FIG. 5 is an internal configuration diagram of the test automation apparatus 200 according to the present embodiment.
FIG. 6 is a configuration diagram of the test procedure manual 210 according to the present embodiment.

As shown in FIG. 5, the operation interpretation unit 221 includes a transmission OK / NG processing unit 2211, an interpretation result transmission data conversion unit 2212, and a test procedure interpretation unit 2213.
As shown in FIGS. 5 and 6, the test procedure manual 210 includes a test procedure unit 2141, an expected value unit 2142, and a test result unit 2143. The test procedure manual 210 is, for example, a tabular test procedure manual represented by “Microsoft Excel”, and includes position information in which each of the test procedure section 2141 and the expected value section 2142 is described. In addition, the test result section 2143 displays an OK or NG judgment that automatically compares the behavior result of the product 100 with respect to the “operation” and the expected value.

The test procedure interpreting unit 2213 distinguishes the test procedure part (test procedure unit 2141) of the test procedure manual 210 by the processing device between operation and non-operation.
For example, as shown in FIG. 6, in the test procedure part 2141 of the first record of the test procedure document 210, “operation button” is set as “operation” and “next operation” is information other than the operation. Is set to “60 seconds”. Information other than the operation is an example of time information related to the operation.
Therefore, the test procedure interpretation unit 2213 interprets “press the power button” as “operation” and “60 seconds” as “interval until next operation”.

  The interpretation result transmission data conversion unit 2212 converts the “operation” portion into operation data that can be received by the product 100 based on the interpretation result by the test procedure interpretation unit 2213. In other words, the interpretation result transmission data conversion unit 2212 generates operation information 1e indicating an operation command for the product 100 based on the identified “operation” (test procedure) and information other than the operation.

The test procedure interpretation unit 2213 confirms the position information indicating which part of the test procedure manual 210 “operation” (test procedure) is described, and specifies and loads “operation” (test procedure).
For example, the test procedure interpreting unit 2213 may be able to distinguish “operation” from the standing of the test procedure unit 41 using “XX button” as a keyword. Further, by setting an interval until the next “operation” as the time information, it is possible to consider the time until the behavior of the product 100 is completed.

  As described above, the automatic operation unit 220 can detect a keyword related to an operation and time information related to an operation from a test procedure document 210 created and stored in advance, and convert the detected operation information into operation information that can be received by a product. The time information related to the operation is, for example, the interval between the operation and the operation of the next record, the time for which the button is long pressed, and the like.

<Transmission OK / NG processing: S103>
The transmission OK / NG processing unit 2211 determines whether or not the operation data converted by the interpretation result transmission data conversion unit 2212 can be output to the transmitter 222 by the processing device. The transmission OK / NG processing unit 2211 receives the NG information 301 or the OK information 302, and executes processing based on the received NG information 301 or the OK information 302.
For example, when the transmission OK / NG processing unit 2211 receives the NG information 301 (see FIG. 5) from the expected operation interpretation unit 250, the transmission OK / NG processing unit 2211 determines that the transmission is NG (NO in S103) and ends the process.

  The case where the transmission OK / NG processing unit 2211 receives the NG information 301 from the expected behavior interpretation unit 250 corresponds to the record before the record of the test procedure manual 210 to be transmitted in S103 (referred to as the processing target record). This means that the behavior of the product 100 is NG in the “operation” procedure. In this case, the NG information 301 is transmitted from the expected operation interpretation unit 250 to the transmission OK / NG processing unit 2211. When the transmission OK / NG processing unit 2211 of the operation interpreting unit 221 receives the NG information 301, the transmission of the operation data is stopped and the processing is ended.

For example, when the transmission OK / NG processing unit 2211 receives the OK information 302 (see FIG. 4) from the expected operation interpretation unit 250, the transmission OK / NG processing unit 2211 determines that the transmission is OK (YES in S103), and advances the processing to S104.
The case where the transmission OK / NG processing unit 2211 receives the OK information 302 from the expected operation interpretation unit 250 corresponds to the record before the record of the test procedure manual 210 to be transmitted in S103 (referred to as the processing target record). It means that the behavior of the product 100 is OK in the “operation” procedure.
A process in which the expected action interpretation unit 250 transmits the NG information 301 or the OK information 302 to the operation interpretation unit 221 (transmission OK / NG processing unit 2211) will be described later.

  Further, the transmission OK / NG processing unit 2211 may not be a process of determining whether or not the operation information 1e can be transmitted based on the NG information 301 and the OK information 302. For example, the transmission OK / NG processing unit 2211 refers to the test result unit 2143 of the record before the processing target record in the test procedure manual 210, and when the test result unit 2143 of the record before the processing target record is NG. The process may be terminated.

<Operation information transmission process (command transmission process): S104>
As shown in FIG. 5, the transmitter 222 includes a transmission processing unit 2221.
The transmitter 222 outputs the operation information 1e to the product 100. The transmission processing unit 2221 executes processing for matching the operation information 1e with the transmission format of the transmitter 222. For example, the transmission processing unit 2221 transmits the operation information 1e to the product 100 using communication processing represented by a remote controller.
As a result, for example, the same operation can be realized as if the tester performs operations such as a power supply operation, a reproduction operation, and a scheduled recording operation on the product 100 such as a television by using a remote controller or the like.

FIG. 7 is an internal configuration diagram of the product 100 according to the present embodiment.
The processing operation of the product 100 that has received the operation information 1e will be described with reference to FIGS. The operation of the product 100 that has received the operation information 1e includes an operation input process (S105 in FIG. 4), a behavior information collection process (S106 to S107 in FIG. 4), and a behavior information transmission process (S108 in FIG. 4).

As shown in FIG. 7, the product 100 includes an operation receiving module 25, a function 1 module (SW module 21 (software module), HW module 23 (hardware module)), function, as an operation execution unit 116 (see FIG. 2). Two modules (SW module 22) are provided.
The SW module 21 to the operation receiving module 25 are modules inherent to the product 100. The function 1 module 24 is an example composed of the SW module 21 and the HW module 23, and may be composed of only the SW module 22 like the function 2 module. The operation receiving module 25 is originally provided in the product 100 represented by the receiving unit of the remote controller, and is a module that receives an operation.

  The operation input interface unit 115 includes an operation conversion unit 1151 and an operation interpretation unit 1152. The behavior information output unit 113 includes an information collection unit 1131 and an information output unit 1132. The behavior output interface unit 114 includes a behavior information conversion unit 1141 and a behavior information transmission unit 1142.

<Operation receiving process (command receiving process): S105>
In S105, the operation input interface unit 115 receives the operation information 1e. The operation input interface unit 115 converts the received operation information 1e into a format suitable for the operation receiving module 25 inherent to the product 100.
The operation interpretation unit 1152 interprets the content of the operation of the received operation information 1e by the processing device. The operation conversion unit 1151 converts the content of the operation interpreted by the operation interpretation unit 1152 into a format that can be received by the operation reception module 25 inherent to the product 100 by the processing device. When the operation information 1e can be directly received by the operation receiving module 25, the operation input interface unit 115 does not need to be incorporated.

<Behavior information collection process (operation execution process): S106 to S107>
The operation receiving module 25 receives an operation (information obtained by converting the operation information 1e) from the operation input interface unit 115. Similar to the case where the operation receiving module 25 receives an operation from the user by the remote controller, the product 100 is configured to function 1 module (SW module 21, HW module 23) and function 2 module (SW module 22) according to the received operation. Send a signal to That is, the product 100 performs a behavior corresponding to the received operation.

  In S <b> 106, the information collection unit 1131 collects behavior information of the product 100 from the function 1 module 24 and the SW module 22 that are inherent to the product 100.

In S107, the information output unit 1132 converts the behavior information collected by the information collection unit 1131 into a data format that can be received by the test automation device 200 by the processing device.
If a large amount of information is output in the behavior information output process, the performance of the product 100 may be deteriorated. Therefore, the information output unit 1132 performs a process of abstracting necessary behavior information by the processing device.

FIG. 8 is a diagram showing abstraction of behavior information by the information output unit 1132 according to the present embodiment.
As shown in FIG. 8, the configuration of the behavior information 1f ′ output by the information output unit 1132 is the same as the configuration of the expected value unit 2142 of the test procedure manual 210 of FIG.
The behavior information 1f ′ (operation information) is information including status information indicating the operation state of the product 100 and setting information set in the product 100.
The behavior information 1f ′ includes 96 bits of status 8 bits, channel 16 bits, display 8 bits, OSD (display setting display / adjustment function) 16 bits, display time 8 bits, cursor position 8 bits, recording mode 8 bits, and reaction (behavior) (elapsed) time 16 bits. It is digitized by. FIG. 8 shows an example of the numerical value setting for each item.

  As described above, the behavior information 1 f ′ includes the behavior elapsed time 51 and outputs the operation time required for the behavior. The behavior elapsed time 51 is used for the pass / fail judgment together with the behavior in the expected behavior interpretation processing by the expected behavior interpretation unit 250 of the test automation apparatus 200 described later. The behavior elapsed time 51 may be an internal operation time of the product 100 depending on the content of the test.

<Behavior information transmission process (operation information transmission process): S108>
In S <b> 108, the behavior output interface unit 114 receives the behavior information 1 f ′ output from the information output unit 1132. The behavior information conversion unit 1141 of the behavior output interface unit 114 converts and adapts the input behavior information 1f ′ to an interface physically connected to the test automation apparatus 200, and outputs it as behavior information 1f. The behavior information transmission unit 1142 transmits, to the test automation apparatus 200, behavior information 1f adapted to a communication interface represented by, for example, RS232C, by the behavior information conversion unit 1141. When the product is inherently provided, the behavior output interface unit 114 need not be incorporated.

  Next, the operation of the test automation apparatus 200 that has received the behavior information 1f will be described with reference to FIGS.

<Behavior information reception process (operation information reception process): S201>
The behavior input interface unit 230 (behavior information processing unit) performs processing in accordance with the format of the communication input interface provided in the test automation apparatus 200. The behavior input interface unit 230 is an example of an operation information receiving unit that receives the behavior information 1 f transmitted from the behavior output interface unit 114.

  The behavior input determination unit 240 (behavior information determination processing unit) decodes the behavior information 1 f abstracted by the behavior information output unit 113. The behavior input determination unit 240 prepares data so that it can be easily compared with an expected value by decoding the behavior information 1f. The behavior input determination unit 240 outputs the decoded behavior information 1 f to the expected operation interpretation unit 250.

<Expected motion interpretation process (expected motion determination step): S202 to S203>
The expected behavior interpretation unit 250 includes an expected value interpretation processing unit 251 and an expected value / behavior verification processing unit 252.
The expected value interpretation processing unit 251 extracts the expected value part 2142 corresponding to the test procedure part 2141 that is the source of the transmitted operation information 1e from the expected value part 2142 of the test procedure document 210. The expected value interpretation processing unit 251 converts the extracted expected value unit 2142 into the same output data format as the behavior information 1f converted by the behavior input determination unit 240.

The expected value / behavior verification processing unit 252 includes behavior information 1f decoded by the behavior input determination unit 240 and an expected value unit 2142 converted into the same output data format as the behavior information 1f by the expected value interpretation processing unit 251. Verification is performed by the processing device.
The expected value / behavior verification processing unit 252 expects the behavior result of the product 100 based on the behavior information 1 f input from the behavior input determination unit 240 and the expected value unit 2142 input from the expected value interpretation processing unit 251. It is determined by the processing device whether or not the result is satisfied. The expected value / behavior verification processing unit 252 outputs NG information 301 to the operation interpretation unit 221 when the verification determination result is NG. The expected value / behavior verification processing unit 252 outputs OK information 302 to the operation interpretation unit 221 when the verification determination result is OK.
However, there is no need for the expected value / behavior verification processing unit 252 to output the NG information 301 or the OK information 302 to the operation interpretation unit 221.

  If the collation determination result is OK (YES in S203), the process proceeds to S204. If the collation determination result is NG (NO in S203), the process proceeds to S205.

<Result display processing: S204 to S206>
The result output unit 260 includes an OK / NG display processing unit 261 and an NG warning processing unit 262.
The OK / NG display processing unit 261 performs OK display or NG display, and sets OK or NG in the test result portion 2143 column of the test procedure manual 210.
The NG warning processing unit 262 executes a process of giving a warning by using an LED, sound, or the like.

  In S204, the OK / NG display processing unit 261 displays an OK display on the display device, and the process proceeds to S206. In S205, the OK / NG display processing unit 261 displays an NG display on the display device, and the NG warning processing unit 262 issues a warning with an LED, a voice, or the like, and the process is terminated. Thereby, when the test result is NG, the test can be interrupted, and a warning can be given to the tester to notify that the test was NG.

  In S206, the result output unit 260 determines whether the test is completed. For example, if the test procedure to be processed is the last record in the test procedure manual 210, the result output unit 260 determines that the test is completed, and if the test procedure is not the last record, the result output unit 260 determines that the test is not completed. to decide.

If the test is completed (YES in S206), the process ends.
If the test has not been completed (NO in S206), the process is executed from S102 on the test procedure of the next record to be processed in the test procedure document 210.

  This is the end of the description of the test automation process (method) of the test automation system 500.

  In the present embodiment, the process is terminated when the collation determination result is NG, but the present invention is not limited to this. For example, the test may be executed up to the test procedure of the last record in the test procedure manual 210 without ending the process.

  Further, in this embodiment, automatic operation means (unit), expected action interpretation means (part), behavior input determination means (part), expected action interpretation means (part), and result output means (part) are one test automation. Since the device is integrated, a compact device can be provided.

According to the test automation system 500 according to the present embodiment, the operation given to the product is interpreted based on the test procedure document, and the operation information can be transmitted to the product. Can be shortened.
In addition, since the behavior of the product is output from the product and input to the test input / output device, it is possible to reliably determine whether the test has passed or failed. In addition, the pass / fail judgment is automated, and it is not necessary for the tester to be present at the time of the test execution, and the test execution confirmation time can be greatly shortened.

  In the behavior information output means (unit) of the test automation system 500 according to the present embodiment, the operation state of the product and the setting information of the product are output as the behavior of the product. Further, the behavior information output means (unit) of the test automation system 500 according to the present embodiment can output the elapsed time information from the operation to the behavior of the product and the elapsed time of the internal operation of the product. Therefore, it is possible to improve the completeness of the test and improve the quality by outputting even the internal behavior of the product that cannot be output to the peripheral simulator.

Embodiment 2. FIG.
In the present embodiment, differences from the first embodiment will be mainly described.
In the present embodiment, a test automation apparatus 201 in which the transmitter 222 and components other than the transmitter 222 are separated will be described.

  FIG. 9 is a block diagram showing a configuration of the test automation apparatus 201 according to the present embodiment. Components having the same functions as those described in Embodiment 1 may be denoted by the same reference numerals and description thereof may be omitted.

As shown in FIG. 9, the test automation apparatus 201 includes a transmitter 222 and a personal computer 61. Of the components described in the first embodiment, the personal computer 61 has components other than the transmitter 222. The personal computer 61 and the transmitter 1a are connected by communication means 62 such as RS232C.
Therefore, the general personal computer 61 can be used for storing the test procedure manual 210 and determining the expected operation.
In the present embodiment, the operation interpretation unit 221 performs transmission processing in accordance with the communication unit 62.

  According to the test automation apparatus 201 according to the present embodiment, in the automatic operation unit, the operation interpretation unit that interprets the operation of the test procedure manual and the transmitter that transmits the operation to the product are separated. The test automation apparatus 201 can be configured using a general personal computer 61.

Embodiment 3 FIG.
In the present embodiment, differences from Embodiments 1 and 2 will be mainly described.
FIG. 10 is a diagram showing an example of modeling for creating the test procedure manual 210 according to the present embodiment. Components having the same functions as those described in the first and second embodiments are denoted by the same reference numerals and description thereof may be omitted.
The present embodiment aims to automate the creation of the test procedure manual 210a, and the steps after the creation of the test procedure manual 210a are the same as in the first and second embodiments. FIG. 10 is an example of a model for automatically generating the test procedure manual 210a from the test model 82.

In S100 of FIG. 4, the procedure manual creation unit 270 (see FIG. 2) of the test automation apparatus 200 executes a procedure manual creation process. The procedure manual creation unit 270 automatically creates the test procedure manual 210a.
Although the configuration of the test procedure manual 210 shown in FIG. 6 is different from the configuration of the test procedure manual 210a shown in FIG. 10, which item is used among the items of the automatically created test procedure documentation 210a. It can be appropriately changed as to whether it is stored as the document 210.

First, the procedure manual creation unit 270 performs modeling of the design specification that is the basis of the test procedure 210a, and generates a test model 82 (specification model). The design specifications are stored in advance in a storage device included in the test automation apparatus 200.
The procedure manual creation unit 270 extracts setting values to be set for each item of the test model 82 from the design specification, and sets the extracted setting values for each item of the test model 82.

  The test viewpoint 83 is generated from the required specifications and design specifications, prevents test leakage at the viewpoint level, and realizes traceability with the test procedure document 210a by corresponding to the test viewpoint of the test procedure document 210a. .

The test condition 84 includes an environment, a start condition, and the like. The test conditions 84 correspond to the pre-preparation, test environment, and activation conditions of the test procedure document 210a as preparation for performing the test procedure.
The test procedure 85 corresponds to the test procedure in the test procedure manual 210a, and the advance state and the operation before the behavior is changed are set.

The behavior 86 corresponds to the criterion (expected value) of the test procedure document 210a, and describes the change in behavior from the starting state.
The result 87 corresponds to the result of the test procedure manual 210a.

  The procedure manual creation unit 270 generates the test model 82 as described above, and performs automatic conversion from the generated test model 82 to the test procedure manual 210a by the processing device.

  As described above, the first to third embodiments are examples of a test automation device that determines the result of operation automation and product behavior by creating a test procedure manual. It can be applied to a system test of a product having a man-machine interface such as a television, a Blu-ray (registered trademark) recorder, and a navigation.

  In the description of the above embodiment, the product 100 and the test automation apparatus 200 are configured by being divided into a plurality of functional blocks. However, the configuration of the functional blocks is not limited to the above embodiment, and functions in any other combination. A block may be configured.

As mentioned above, although embodiment of this invention was described, you may implement one part among this embodiment. Or you may implement combining two or more parts among this embodiment.
The above-described embodiments are essentially preferable examples, and are not intended to limit the scope of the present invention, its application, and uses, and various modifications can be made as necessary. .

  1e operation information, 1f behavior information, 21, 22 SW module, 23 HW module, 24 function 1 module, 25 operation reception module, 51 behavior elapsed time, 61 personal computer, 62 communication means, 82 test model, 83 test viewpoint, 84 Test conditions, 85 test procedure, 86 behavior, 87 results, 100 products, 113 behavior information output unit, 114 behavior output interface unit, 115 operation input interface unit, 116 operation execution unit, 200 test automation device, 210 test procedure manual, 220 automatic operation unit, 221 operation interpretation unit, 222 transmitter, 230 behavior input interface unit, 240 behavior input determination unit, 250 expected operation interpretation unit, 251 expected value interpretation processing unit, 252 expected value / Behavior verification processing unit, 260 result output unit, 261 OK / NG display processing unit, 262 NG warning processing unit, 270 procedure manual creation unit, 301 NG information, 302 OK information, 500 test automation system, 901 arithmetic unit, 902 External storage device, 903 main storage device, 904 communication device, 905 input / output device, 1131, information collection unit, 1132, information output unit, 1141, behavior information conversion unit, 1422, behavior information transmission unit, 1151 operation conversion unit, 1152 operation interpretation unit, 2141 Test procedure part, 2142 Expected value part, 2143 Test result part, 2211 Transmission OK / NG processing part, 2212 Interpretation result transmission data conversion part, 2213 Test procedure interpretation part, 2221 Transmission processing part.

Claims (7)

In a test execution system comprising a target device that is a test target, the target device receiving a command and operating based on the received command, and a test execution device that executes a test on the target device,
The test execution device includes:
A test protocol part including the time and to continue the operation and the operation against the target device, the operation of the subject device in which the operation is expected when performed with respect to the target device included in the test procedures section A storage device that associates the expected operation information to be displayed with the test result unit that sets the operation result of the target device when the operation included in the test procedure unit is performed on the target device. An operation correspondence information storage unit stored in
Removed and time to continue the operation as the operation before Symbol motion correspondence information, based on time and to continue the operation and the operation to generate operation information for executing the operation to the target device, A command transmission unit that transmits the generated operation information to the target device as the command;
The target device is
An operation execution unit that executes the operation based on the instruction that the or instruction transmitting unit et reception of the test execution device,
An operation information transmission unit that transmits operation information indicating the operation executed by the operation execution unit to the test execution device;
The test execution device further includes:
In operation with the operation information received from the operation information transmission unit of the target device, based on said expected operation information included in the operation correspondence information, operation indicated by the operation information indicated by the expected operation information An expected motion determination unit that determines whether or not there is a processing device ; and
Lutein strike execution system comprising when the operation is not an operation indicated by the expected operation information, and suspends warning processing section tests for the target device outputs a warning to <br/> indicated by the operation information. The operation information transmitter is
Based on the operation of the operation execution unit executes, and generates the operation information of the expected operation information of the same configuration that is stored in the operation correspondence information storage unit, that sends the generated the operation information 請 Motomeko test execution system according to 1. The operation information transmitter is
Test execution system according to 請 Motomeko 2 that generates the operation information including the setting information set in the target device and status information indicating the operating state of the target device. The operation correspondence information is a test procedure document composed of a plurality of operations and a time for continuing the operations .
The test execution system further includes:
It generates a specification model that models the design specifications of the target device, based on the generated specification model, from 請 Motomeko 1 Ru comprising the steps drafting unit for generating the processing device said test procedures 3 test execution system according to any one of. In a test execution device that is a target device to be tested, receives a command, and executes a test on the target device that operates based on the received command.
A test protocol part including the time and to continue the operation and the operation against the target device, the operation of the subject device in which the operation is expected when performed with respect to the target device included in the test procedures section A storage device that associates the expected operation information to be displayed with the test result unit that sets the operation result of the target device when the operation included in the test procedure unit is performed on the target device. An operation correspondence information storage unit stored in
Removed and time to continue the operation as the operation before Symbol motion correspondence information, based on time and to continue the operation and the operation to generate operation information for executing the operation to the target device, A command transmitting unit that transmits the generated operation information to the target device as the command;
An operation information receiving unit for receiving operation information indicating an operation that is executed on the basis of the instruction that the instruction transmitting unit or we received from the target device,
And the expected operation information included in the prior SL motion correspondence information, based on said operation information the operation information receiving unit receives, whether the operation indicated by the operation information is the operation indicated by the expected operation information An expected action determination unit for determining whether or not by a processing device;
If the operation indicated by the operation information is not operation indicated by the expected operation information, lutein stringent execution unit includes a warning unit and the <br/> interrupt the test for the target device outputs a warning. In a test execution method of a test execution system, which is a target device that is a test target, and includes a target device that receives a command and operates based on the received command, and a test execution device that executes a test on the target device ,
The test execution device is expected when the operation included in the test procedure unit is performed on the target device, and a test procedure unit including an operation on the target device and a time for continuing the operation. The operation is performed by associating expected operation information indicating the operation of the target device with a test result unit that sets a result of the operation of the target device when the operation included in the test procedure unit is performed on the target device. An operation correspondence information storage unit that stores the correspondence information in the storage device,
The test execution device extracts the operation and the time for which the operation is continued from the operation correspondence information, and performs the operation on the target device based on the operation and the time for which the operation is continued. Information is generated, and the generated operation information is transmitted to the target device as the command,
The target device transmits operation information indicating an operation executed based on the command received from the test execution device to the test execution device,
The test execution device is an operation in which an operation indicated by the operation information is indicated by the expected operation information based on the operation information received from the target device and the expected operation information included in the operation correspondence information. A test execution method that determines whether or not there is a processor and outputs a warning and interrupts the test on the target device when the operation indicated by the operation information is not the operation indicated by the expected operation information. A target device is tested, receives a command, the test executive program to be executed by a Rute list execution unit to execute the test for the target device computer that operates based on the command received,
The test execution device is expected when the operation included in the test procedure unit is performed on the target device, and a test procedure unit including an operation on the target device and a time for continuing the operation. The operation is performed by associating expected operation information indicating the operation of the target device with a test result unit that sets a result of the operation of the target device when the operation included in the test procedure unit is performed on the target device. An operation correspondence information storage unit that stores the correspondence information in the storage device,
Removed and time to continue the operation as the operation before Symbol motion correspondence information, based on time and to continue the operation and the operation to generate operation information for executing the operation to the target device, A command transmission process for transmitting the generated operation information to the target device as the command;
And operation information reception process of receiving from the target device operation information indicating an operation that is executed on the basis of the instruction that the instruction transmission process or found received,
Whether the operation indicated by the operation information is an operation indicated by the expected operation information based on the expected operation information included in the operation correspondence information and the operation information received by the operation information reception process Expected operation determination processing for determining by the processing device,
A test execution program comprising: a warning process for outputting a warning and interrupting a test for the target device when the operation indicated by the operation information is not the operation indicated by the expected operation information .