JP7294043B2 - Program and method for device test automation and script generator - Google Patents

Description

TECHNICAL FIELD The present disclosure relates to testing of a device having an operation unit, and more particularly to technology for generating an automatic execution script based on the details of operation of the device.

2. Description of the Related Art In recent years, devices such as MFPs (Multifunction Peripherals) that accept input operations from users have become multifunctional. Therefore, the number of man-hours for operation tests of these devices continues to increase. Especially in software development, an increase in test man-hours is a heavy burden on manufacturers.

In the software development process, it is important to ensure the quality of software by repeating operation tests. However, the number of man-hours required for the operation test is limited, and in order to improve the development speed, the efficiency of the operation test is required. In order to improve the efficiency of operation tests, a test tool is sometimes used that records the details of operations and creates an automatic execution script that executes similar operations based on the details of the operations.

Regarding a test tool that automatically executes recorded operation contents, for example, Japanese Patent Application Laid-Open No. 2017-117309 (Patent Document 1) describes that "The application management of the management server transmits the application into which the operation event processing code is injected to the smartphone terminal. , install.When the smartphone terminal operation is executed on the client terminal, the equivalent operation is executed on the smartphone terminal, and the operation log information is saved in the script file by the script generation means.Instruct test execution on the client terminal Then, the terminal management sends the application with the operation event processing code injected to the smartphone terminals for installation, and the script execution means reads the script from the script file, converts it into terminal operation information, and sends it to the smartphone terminal. However, on the smartphone terminal, the application is executed based on the received terminal operation information.” (See [Abstract]).

In addition, techniques and the like for operation history of operation contents are disclosed in Japanese Unexamined Patent Application Publication No. 2015-121869 (Patent Document 2) and Japanese Unexamined Patent Application Publication No. 2015-153183 (Patent Document 3).

JP 2017-117309 A JP 2015-121869 A JP 2015-153183 A

According to the techniques disclosed in Patent Literatures 1 to 3, if there is an error in the operation during recording for automatic execution script generation, the operation needs to be re-recorded from the beginning, resulting in an increase in test man-hours. Therefore, there is a need for a technique for correcting an operation record without interrupting operation recording even when an error occurs in the operation being recorded.

The present disclosure has been made in view of the above background, and an object in one aspect is to correct the operation record without interrupting the operation record even if there is an error in the operation during recording. It is to provide technology.

According to one embodiment, a computer-executable program is provided. The program causes the computer to execute steps of detecting a user's operation on the operation screen of the target device and determining a plurality of user's operation modes in the operation. The plurality of operational modes includes a first operational mode and a second operational mode. The program further causes the computer to execute a step of generating a manipulation record that excludes the manipulations executed according to the second manipulation mode from the manipulations executed according to the first manipulation mode.

In one aspect, the step of determining the plurality of operation modes includes determining whether each operation mode corresponds to the first operation mode or the first operation mode based on whether the finger touching the operation screen is the finger of the right hand or the finger of the left hand. It includes a step of determining which of the two operating modes.

In one aspect, the step of determining a plurality of operation modes includes determining whether each operation mode is the first operation mode or the second operation mode based on the shape or color of the touch pen touching the operation screen. including the step of

In one aspect, the step of determining the plurality of operation modes is the step of determining whether each operation mode is the first operation mode or the second operation mode based on the shape of the object touching the operation screen. including.

In one aspect, the program instructs the computer to exclude the first operation from recording if the first operation is performed by the second operation mode after the first operation is performed by the first operation mode. further steps to be taken.

In one aspect, the program instructs the computer, when the first operation is executed according to the second operation mode, to perform a second operation included in the operation record when the first operation is executed according to the first operation mode. replacing the first operation performed according to the first operation mode before the performance with the first operation mode with the first operation performed according to the first operation mode after the second operation mode let it run.

In one aspect, in the step of deleting the first operation from the operation record, when the operation screen transitions due to the execution of the first operation, the operation on the operation screen after the transition due to the execution of the first operation is also deleted from the operation record. including the step of removing from

In one aspect, the program instructs the computer, in the case where the operation screen transitions due to the execution of the second operation, when the second operation is executed in the second operation mode, after the transition due to the execution of the second operation A step of excluding each operation executed on the operation screen from the operation record regardless of whether it was executed in the first operation mode or the second operation mode is further executed.

In one aspect, the plurality of operational modes further includes a third operational mode. The program further causes the computer to delete from the operation record all operations performed after the third operation is performed when the third operation is performed according to the third operation mode.

In one aspect, the program further causes the computer to output information for each operation performed in the second mode of operation.

According to another embodiment, a method is provided for generating operational records for a target device. This method includes the steps of detecting a user's operation on the operation screen of the target device, and determining a plurality of user's operation modes in the operation. The plurality of operational modes includes a first operational mode and a second operational mode. The method further includes the step of generating a manipulation record in which among the manipulations executed according to the first manipulation mode, the manipulations executed according to the second manipulation mode are excluded.

In one aspect, the step of determining the plurality of operation modes includes determining whether each operation mode corresponds to the first operation mode or the first operation mode based on whether the finger touching the operation screen is the finger of the right hand or the finger of the left hand. It includes a step of determining which of the two operating modes.

In one aspect, the step of determining a plurality of operation modes includes determining whether each operation mode is the first operation mode or the second operation mode based on the shape or color of the touch pen touching the operation screen. including the step of

In one aspect, the step of determining the plurality of operation modes is the step of determining whether each operation mode is the first operation mode or the second operation mode based on the shape of the object touching the operation screen. including.

In one aspect, the method includes deleting the first operation from the operation record if the first operation was performed by the second operation mode after the first operation was performed by the first operation mode. further includes

In one aspect, the method includes: when the first operation is performed according to the second operation mode, the second operation mode included in the operation record when the first operation is performed according to the first operation mode; replacing the first operation performed according to the first operational aspect prior to performance with the first operation performed according to the first operational aspect after performance under the second operational aspect.

In one aspect, in the step of deleting the first operation from the operation record, when the operation screen transitions due to the execution of the first operation, the operation on the operation screen after the transition due to the execution of the first operation is also deleted from the operation record. including the step of removing from

In one aspect, when the operation screen transitions due to the execution of the second operation, when the second operation is executed in the second operation mode, the operation screen after transition due to the execution of the second operation is displayed. excluding each operation performed in from the operation record, regardless of whether it was performed by the first operation aspect or the second operation aspect.

In one aspect, the plurality of operational modes further includes a third operational mode. The method further includes deleting from the operation record all operations performed after the third operation was performed when the third operation was performed according to the third operation mode.

According to another embodiment, there is provided a script generation device comprising a memory storing the above program and a processor for executing the program.

According to the present technology, even if there is an error in the operation being recorded, it is possible to correct the operation record without interrupting the recording of the operation.

The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the invention taken in conjunction with the accompanying drawings.

It is an example of an image forming apparatus for which an automatic execution script is generated by a test automation system. 1 illustrates an example of a test script automatic generation system according to one embodiment; FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device 200; FIG. FIG. 10 is a diagram illustrating an example of an overview of correction processing of evaluation operations based on correction operations; It is a figure which shows an example of operation|movement of a test-script automatic generation system when there is no correction operation. It is a figure which shows an example of operation|movement of a test-script automatic generation system at the time of correcting the operation just before. It is a figure which shows an example of operation|movement of a test-script automatic generation system at the time of correcting an incorrect operation after execution of another operation. It is a figure which shows an example of operation|movement of a test-script automatic generation system at the time of receiving evaluation operation and correction operation, respectively. It is a figure which shows an example of operation|movement of a test-script automatic generation system at the time of collectively correcting operation of the screen of a transition destination. It is a figure which shows an example of operation|movement of a test-script automatic generation system when a screen changes by correction operation. It is a figure which shows an example of operation|movement of a test-script automatic generation system at the time of collectively correcting operation of several setting. FIG. 7 is a diagram illustrating an example of an automatic execution script generation process in the information processing apparatus 200; FIG. 13 is a diagram showing an example of processing routine for a correction operation mode in step S1270; FIG. FIG. 13 is a diagram showing an example of processing of a record cancellation processing routine in step S1330; FIG. FIG. 15 is a diagram showing an example of processing routine processing after screen transition by the correcting operation in step S1470; FIG. FIG. 10 is a diagram showing an example of processing of a record cancellation processing routine by special operation;

Hereinafter, embodiments of the technical concept according to the present disclosure will be described with reference to the drawings. In the following description, the same parts are given the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

First, the outline of the test script automatic generation system according to the present embodiment will be described with reference to FIGS. 1 to 3. FIG. In the following description, processing contents of the test script automatic generation system will be described using an image forming apparatus as a test target, but the test target of the test script automatic generation system according to the present embodiment is not limited to the image forming apparatus. The test script automatic generation system according to this embodiment can be used for testing any device having input means such as a touch panel.

FIG. 1 shows an example of an image forming apparatus for which an automatic execution script is generated by a test automation system. Image forming apparatus 100 includes control unit 101, document reading unit 102, printout unit 103, FAX (facsimile) unit 104, storage device 105, wireless IF 106, network IF 107, user authentication unit 108, and an operation panel 109 . The control unit 101 includes a CPU (Central Processing Unit) 110 , a RAM (Random Access Memory) 111 , and a ROM (Read Only Memory) 112 . Operation panel 109 includes a display unit 113 and an operation unit 114 .

A control unit 101 controls the image forming apparatus 100 and executes various programs. The CPU 110 executes or refers to various programs and data read into the RAM 111 . In one aspect, CPU 110 may be an embedded CPU, an FPGA (Field-Programmable Gate Array), or a combination thereof. CPU 110 can execute programs for realizing various functions of image forming apparatus 100 .

RAM 111 stores programs executed by control unit 101 and data referred to by control unit 101 . In one aspect, DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory) may be used as RAM 111 .

ROM 112 is a non-volatile memory and may store programs executed by CPU 110 . In that case, the CPU 110 executes the program read from the ROM 112 to the RAM 111 . In some aspects, Erasable Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), or flash memory may be used as ROM 112 .

The document reading unit 102 can scan a document and convert the document into image data. CPU 110 can acquire image data via document reading unit 102 . In one aspect, document reading unit 102 can store acquired image data in storage device 105 . In another aspect, document reading unit 102 may acquire image data via network IF 107 and store the acquired image data in storage device 105 .

The print output unit 103 performs print processing of image data captured by the image forming apparatus 100 . In one aspect, a circuit that controls various actuators for printing functions, including an imaging unit, may be used as the print output section 103 .

A FAX unit 104 transmits and receives originals or image data by facsimile using a telephone line. In one aspect, a circuit with facsimile communication control functions and a telephone line communication port may be used as the FAX unit 104 .

The storage device 105 is a non-volatile memory, and can save data even when the power of the image forming apparatus 100 is turned off. Storage device 105 may store any programs and data that CPU 110 executes or references. In one aspect, a HDD (Hard Disk Drive) or SSD (Solid State Drive) may be used as the storage device 105 . The CPU 110 can read various programs from the storage device 105 to the RAM 111 as necessary and execute the read programs.

The wireless IF 106 is connected to wired network equipment. In some aspects, a Wi-Fi (registered trademark) module or the like may be used as the wireless IF 106 . In another aspect, wireless IF 106 may transmit and receive data using communication protocols such as TCP/IP (Transmission Control Protocol/Internet Protocol) and UDP (User Datagram Protocol).

The network IF 107 is connected to wired network equipment. In one aspect, a wired LAN (Local Area Network) port or the like may be used as the network IF 107 . In other aspects, network IF 107 may transmit and receive data using communication protocols such as TCP/IP and UDP.

User authentication unit 108 can authenticate a user based on a user ID (Identifier) and password input from operation panel 109 or the like. In a certain aspect, user authentication unit 108 may cooperate with an external authentication server, and may accept input of biometric authentication information such as a fingerprint instead of a user ID and password.

An operation panel 109 can receive various operations to the image forming apparatus 100 and display an operation menu and various information. The display unit 113 may be, for example, a liquid crystal monitor or an organic EL (Electro-Luminescence) monitor. The display unit 113 displays an operation menu and various information. The operation unit 114 is, for example, a touch sensor or button incorporated in the operation panel 109, and can receive operation input from the user.

FIG. 2 is a diagram showing an example of a test script automatic generation system according to this embodiment. A test script automatic generation system according to the present embodiment records a user's operation on a device under test and generates an automatic execution script for inputting the same operation to the device under test. The test script automatic generation system can accept correction of the operation when the user makes a mistake while recording the operation of the device under test. The configuration of the test script automatic generation system and the correction processing of the user's operation will be described below.

The test script automatic generation system includes an information processing device 200 and a camera 250 . The camera 250 records or captures, for example, the screen of the operation panel 109 of the image forming apparatus 100, which is the device under test, and the user's operation details. Information processing apparatus 200 can generate an automatic execution script for image forming apparatus 100 based on operation record 210 (video or photo) obtained from camera 250 .

The information processing apparatus 200 includes a screen analysis unit 201, an operation analysis unit 202, an operation determination unit 203, a correction unit 204, a script generation unit 205, a content DB (database) 206, a correction operation store 207, an evaluation It includes an operation store 208 and a script DB 209 .

The screen analysis unit 201 analyzes what screen is included in the operation record 210 based on the operation record 210 acquired from the camera 250 and the content DB 206 . As an example, the screen analysis unit 201 may determine that the screen included in the operation record 210 is a print setting screen, a scan setting screen, or the like.

The content DB 206 includes screen information of the operation screen of the device under test and information on the arrangement of operation components (input fields, pull-downs, buttons, etc.) included in each operation screen. The screen analysis unit 201 can compare the screen included in the operation record 210 and the operation screen included in the content DB 206 to detect on which screen the user is performing the operation input.

The operation analysis unit 202 can analyze the operation performed by the user based on the operation record 210 acquired from the camera 250 and the content DB 206 . As an example, the screen analysis unit 201 analyzes the screen included in the operation record 210 and identifies the position touched by the user's finger. Next, the screen analysis unit 201 refers to the content DB 206 and identifies the operation component arranged at the position touched by the user's finger, thereby detecting what operation input the user is performing.

The operation determination unit 203 can determine whether the user's operation is an evaluation operation or a correction operation based on the user's operation mode. The operation mode is how the user performed the operation input. As an example, the operation mode includes operation with the right hand, operation with the left hand, the number of fingers, the angle of the fingers as seen from the camera 250, whether or not a touch pen is used, and an object (finger, touch pen, etc.) that operates the operation panel 109. ) and the like. As an example, the operation determination unit 203 determines whether the user's operation is an evaluation operation or a correction operation based on whether the finger touching the operation panel 109 is the finger of the right hand or the finger of the left hand. can. As another example, the operation determination unit 203 can determine whether the user's operation is an evaluation operation or a correction operation, based on the shape or color of the touch pen that touches the operation panel 109 . As yet another example, based on the shape of anything touching the operation panel 109, it may be determined whether the user's operation is an evaluation operation or a correction operation. An evaluation operation is an operation that is included in an autorun script. That is, the evaluation operation is an operation for testing the operation of the device under test. A correction operation is an operation for deleting an incorrect operation from evaluation operations that have already been input.

In one aspect, the operation determination unit 203 determines that the evaluation operation is performed when the user operates the device under test with the right hand, based on the operation record 210, and corrects when the user operates the device under test with the left hand. It can be determined that an operation has been performed. In another aspect, the operation determination unit 203 determines, based on the operation record 210, the angle of the hand viewed from the camera 250, which finger was used to operate the device under test, or how many fingers were used to operate the device under test. Whether the user's operation is an evaluation operation or a corrective operation may be determined based on any method, such as whether or not the user's operation was performed.

In the following description, as an example, it is assumed that the operation determination unit 203 determines whether the user's operation is an evaluation operation or a correction operation based on which hand the user has operated the device under test. explain.

The modification operation store 207 stores modification operation temporary data that summarizes modification operations. Correction operation temporary data is a temporary collection of correction operations within a series of operations for a certain operation test. When the operation determination unit 203 determines that the user has operated the device under test with the left hand based on the operation record 210, the operation determination unit 203 can determine the user's operation input as a correction operation. A correction operation is included in the correction operation temporary data in association with the operation component actually operated by the correction operation. The corrective operation temporary data is stored in the corrective operation store 207 .

The evaluation operation store 208 stores evaluation operation temporary data that summarizes evaluation operations. Evaluation operation temporary data is a temporary collection of correction operations within a series of operations for a certain operation test. When the operation determination unit 203 determines that the user has operated the device under test with the right hand based on the operation record 210 , the operation input by the user is stored in the evaluation operation store 208 as an evaluation operation. The evaluation operation is included in the evaluation operation temporary data in association with the operation component actually operated by the evaluation operation. The evaluation operation temporary data is stored in the evaluation operation store 208 .

The correction unit 204 corrects the evaluation operation temporary data based on the correction operation temporary data. More specifically, the modifying unit 204 deletes some of the operations included in the evaluation operation temporary data based on the modifying operations included in the modifying operation temporary data.

The script generation unit 205 converts the corrected evaluation operation temporary data into an automatic execution script that can be read into the test target device, and stores the automatic execution script in the script DB 209 . In one aspect, the script generator 205 may output a list of operations that the user made wrong based on the corrective operation temporary data. In another aspect, the script generation unit 205 outputs information about each operation that the user made a mistake and the number of times each operation was made wrong, based on the past correction operation temporary data stored in the correction operation store 207. may Such information may be used to improve the user interface of the device under test.

In one aspect, the test script automatic generation system may include a configuration other than a camera to determine the user's operation content. As an example, the automatic test script generation system may include distance sensors on the upper right and upper left of the operation panel 109, and identify the finger position (operation position on the operation panel 109) from each of these distance sensors. As another example, a touch panel sheet covering the operation panel 109 may be provided, and the finger position (operation position on the operation panel 109) may be detected by the touch panel sheet. Furthermore, the touch panel sheet may have a fingerprint detection function. The automatic test script generation system can distinguish between right and left hand operation with the fingerprint detection function. The test script automatic generation system may analyze the information obtained from these various sensors as the operation record 210 .

As described above, the test script automatic generation system according to the present embodiment can determine whether the user's operation is an evaluation operation or a correction operation based on the user's operation mode. Additionally, the automatic test script generation system may modify the evaluation operations based on the modification operations. Therefore, even if the user makes a mistake during the test operation, the user can correct the operation on the spot without having to start over from scratch. As a result, the test script automatic generation system can greatly improve the efficiency of automatic execution script creation in a device under test having many functions.

Each functional unit of the information processing apparatus 200 in FIG. 2 may be realized as a program or data executed by hardware described in FIG. FIG. 3 is a diagram showing an example of the hardware configuration of the information processing device 200. As shown in FIG.

Information processing apparatus 200 includes CPU 301 , primary storage device 302 , secondary storage device 303 , external device interface 304 , input interface 305 , output interface 306 , and communication interface 307 .

The CPU 301 executes or refers to various programs and data read into the primary storage device 302 . In one aspect, CPU 301 may be an embedded CPU, an FPGA, or a combination thereof, or the like. The CPU 301 can execute programs for realizing various functional units in FIG.

Primary storage device 302 stores programs executed by CPU 301 and data referred to by CPU 301 . In some aspects, DRAM or SRAM may be used as primary storage 302 .

The secondary storage device 303 is a non-volatile memory, and can store data even when the power of the information processing device 200 is turned off. Secondary storage device 303 may store any programs and data that CPU 301 executes or references. In one aspect, an HDD or SSD may be used as secondary storage device 303 . The CPU 301 can read various programs from the secondary storage device 303 to the primary storage device 302 as necessary and execute the read programs.

The external device interface 304 can be connected to any external device such as printers, scanners and external HDDs. Information processing apparatus 200 may communicate with camera 250 via external device interface 304 . In one aspect, a USB (Universal Serial Bus) terminal or the like may be used as the external device interface 304 .

Input interface 305 may be connected to any input device such as a keyboard, mouse or gamepad. In some aspects, a USB port, a PS/2 port, a Bluetooth® module, and the like may be used as the input interface 305 .

The output interface 306 can be connected to any output device such as a cathode ray tube display, liquid crystal display or organic EL display. In some aspects, a USB terminal, a D-sub terminal, a DVI terminal, an HDMI (registered trademark) terminal, and the like may be used as the output interface 306 .

The communication interface 307 is connected to wired or wireless network equipment. Information processing apparatus 200 may communicate with camera 250 via communication interface 307 . In one aspect, a wired Local Area Network (LAN) port, a Wi-Fi module, or the like may be used as communication interface 307 . In other aspects, communication interface 307 may send and receive data using communication protocols such as TCP/IP, UDP, and the like.

FIG. 4 is a diagram illustrating an example of an overview of correction processing of evaluation operations based on correction operations. On screen 401, first, the user performs operation A and operation B with the right hand. The information processing apparatus 200 includes the operations A and B in the evaluation operation temporary data based on the operation record acquired from the camera 250 . Second, the user notices an operation error (operation B) and performs an operation to undo operation B with the left hand. The information processing apparatus 200 includes the correction operation of the operation B in the correction operation temporary data based on the operation record acquired from the camera 250 . Thirdly, the information processing apparatus 200 deletes the operation B from the evaluation operation temporary data based on the correction operation temporary data, and generates an automatic execution script from the evaluation operation temporary data after the correction.

A conventional test script automatic generation system does not have the function of correcting the evaluation operation based on such an operation mode. Therefore, the user cannot correct the evaluation operation in the middle, and if the user makes a mistake in inputting the evaluation operation even once, the user has to start over from the beginning.

For example, if a user uses a conventional test script auto-generation system to create an auto-execution script containing all 30 steps of operation, when the user fails the operation at the 29th step, the user has to redo all 30 steps. I needed it.

On the other hand, when the user uses the test script automatic generation system according to the present embodiment to create an automatic execution script including all 30 steps of operation, even if the user fails the operation at the 29th step, the correction The operation only needs to be performed once, and the user's burden of creating an auto-execution script can be greatly reduced.

Next, an operation example of the test script automatic generation system according to the present embodiment will be described with reference to FIGS. 5 to 11. FIG. It is assumed that the operation screens shown in FIGS. 5 to 11 are displayed on the operation panel 109. FIG.

FIG. 5 is a diagram showing an example of the operation of the test script automatic generation system when there is no correction operation. In the example of FIG. 5, the user performs all nine steps without making any mistakes. On the home menu screen 501, the user performs the evaluation operation (1) "select copy" with the right hand. The screen transitions to the copy setting screen 502 by the evaluation operation (1).

On the copy setting screen 502, the user performs evaluation operation (2) "set number of copies to 3", evaluation operation (3) "set color setting to color", and evaluation operation (4) "print type Set to double-sided” and evaluation operation (5) “select margin setting button”. The screen transitions to the margin setting screen 503 by the evaluation operation (5).

On the margin setting screen 503, the user performs the evaluation operation (6) "Set the vertical margin to 5 mm", the evaluation operation (7) "Set the horizontal margin to 5 mm", and the evaluation operation (8) with the right hand. "OK selection" is performed. The screen transitions to the copy setting screen 502 by the evaluation operation (8). On the copy setting screen 602, the user performs the evaluation operation (9) "execute printing" with the right hand.

Operation record 504 includes evaluation operations (1) through (9). Timeline 505 is a sequence of operations performed by the user. Internal processing of the information processing apparatus 200 based on the operation of FIG. 5 will be described below.

First, the information processing apparatus 200 generates evaluation operation temporary data 506 based on the operation record 504 acquired from the camera 250 . Evaluation operation temporary data 506 includes evaluation operations (1) to (9) as records. Since the user did not make an operation error, no corrective operation temporary data is generated. The information processing device 200 converts the evaluation operation temporary data 506 into an automatic execution script. The information processing apparatus 200 can save the generated automatic execution script in the secondary storage device 303 . For example, the user can cause the image forming apparatus 100 to reproduce the operation based on the operation in FIG. 5 by inputting an automatic execution script to the image forming apparatus 100 via a network.

FIG. 6 is a diagram showing an example of the operation of the test script automatic generation system when the immediately preceding operation is corrected. In the example of FIG. 6, the user makes an operation error once while performing all nine steps, and then performs a correction operation immediately after that.

On the home menu screen 601, the user performs the evaluation operation (1) "select copy" with the right hand. The screen transitions to the copy setting screen 602 by the evaluation operation (1). On the copy setting screen 602, the user performs evaluation operation (2) "set number of copies to 3" and evaluation operation (3) "set color setting to color" with the right hand.

Immediately after evaluation operation (3), the user realizes that he originally intended to set the color setting to monochrome. Therefore, the user performs the correction operation (A) "cancel color setting" with the left hand. Next, with the right hand, the evaluation operation (3') "Set color setting to monochrome" is performed.

Subsequently, on the copy setting screen 602, the user performs evaluation operation (4) "set print type to double-sided" and evaluation operation (5) "select margin setting button" with the right hand. The screen transitions to the margin setting screen 603 by the evaluation operation (5).

On the margin setting screen 603, the user performs the evaluation operation (6) "Set the vertical margin to 5 mm", the evaluation operation (7) "Set the horizontal margin to 5 mm", and the evaluation operation (8) with the right hand. "OK selection" is performed. The screen transitions to the copy setting screen 502 by the evaluation operation (8). On the copy setting screen 602, the user performs the evaluation operation (9) "execute printing" with the right hand.

Operation record 604 includes evaluation operations (1)-(9) and (3') and corrective operation (A). Timeline 605 is a chronological sequence of actions performed by the user. Internal processing of the information processing apparatus 200 based on the operation in FIG. 6 will be described below.

First, the information processing apparatus 200 generates evaluation operation temporary data 606 and correction operation temporary data 607 based on the operation record 604 acquired from the camera 250 . Evaluation operation temporary data 606 includes operation (3) and operation (3') for "color setting" as records. The correction operation temporary data 607 includes an operation (A) for correcting "color setting" as a record.

Second, the information processing device 200 corrects the evaluation operation temporary data 606 based on the correction operation temporary data 607 . More specifically, first, the information processing apparatus 200, based on the correction operation (A) for the "color setting", changes the "color setting" performed immediately before the correction operation (A) in the evaluation operation temporary data 606. Delete the evaluation operation (3) for Next, the information processing apparatus 200 determines the position of the evaluation operation (3) in the evaluation operation temporary data 606 based on the evaluation operation (3′) for “color setting” performed after the correction operation (A). insert the evaluation operation (3') into . Through this process, the information processing apparatus 200 finally replaces the evaluation operation (3) of the evaluation operation temporary data 606 with the evaluation operation (3').

Third, the information processing apparatus 200 converts the corrected evaluation operation temporary data 606 into an automatic execution script. The information processing apparatus 200 can save the generated automatic execution script in the secondary storage device 303 . For example, the user can cause the image forming apparatus 100 to reproduce the operation based on the operation in FIG. 6 by inputting an automatic execution script to the image forming apparatus 100 via a network.

As described above, the information processing apparatus 200 deletes the evaluation operation X based on detection of the correction operation Y for the operation component P after the evaluation operation X for the operation component P is performed. Alternatively, when the operation component P is an operation component for setting selection, the information processing apparatus 200 restores the setting of the operation component P to the default based on detection of the correction operation Y for the operation component P. FIG.

Further, after detecting the correction operation Y for the operation component P, the information processing apparatus 200 detects the evaluation operation Z for the operation component P again. , insert the evaluation operation Z. That is, the information processing apparatus 200 overwrites the evaluation operation X with the evaluation operation Z in the evaluation operation temporary data.

With these processes, even if the user makes a mistake in the middle of the operation, the user can obtain the desired automatic execution script simply by performing the correcting operation and the overwriting operation without having to restart the operation from 1.

FIG. 7 is a diagram showing an example of the operation of the test script automatic generation system when an erroneous operation is corrected after another operation is executed. In the example of FIG. 7, the user makes an operation error once while performing the operation of all nine steps, then performs another operation, and then executes the correction operation.

On the home menu screen 701, the user performs the evaluation operation (1) "select copy" with the right hand. The screen transitions to the copy setting screen 702 by the evaluation operation (1). On the copy setting screen 702, the user performs evaluation operation (2) "set number of copies to 3", evaluation operation (3) "set color setting to color", and evaluation operation (4) "set print type to 3" with the right hand. Set to double-sided” and evaluation operation (5) “select margin setting button”. The screen transitions to the margin setting screen 703 by the evaluation operation (5).

On the margin setting screen 703, the user performs evaluation operation (6) "set margin height to 5 mm", evaluation operation (7) "set margin width to 5 mm", and evaluation operation (8) with the right hand. "OK selection" is performed. The screen transitions to the copy setting screen 702 by the evaluation operation (8).

Immediately after the evaluation operation (8), the user realizes that he originally intended to set the color setting to monochrome. Therefore, the user performs the correction operation (A) "cancel color setting" with the left hand. Next, with the right hand, the evaluation operation (3') "Set color setting to monochrome" is performed. Subsequently, the user performs the evaluation operation (9) "print execution" with the right hand.

Operation record 704 includes evaluation operations (1)-(9) and (3') and corrective operation (A). Timeline 705 is a chronological sequence of actions performed by the user. The internal processing of the information processing apparatus 200 based on the operation shown in FIG. 7 will be described below.

First, the information processing apparatus 200 generates evaluation operation temporary data 706 and correction operation temporary data 707 based on the operation record 604 acquired from the camera 250 . Evaluation operation temporary data 706 includes operation (3) and operation (3') for "color setting" as records. Within the evaluation operation temporary data 706, operation (3') is located several steps after operation (3). The correction operation temporary data 707 includes an operation (A) for correcting "color setting" as a record.

Second, the information processing device 200 corrects the evaluation operation temporary data 706 based on the correction operation temporary data 707 . More specifically, first, the information processing apparatus 200 changes the “color setting” performed immediately before the correcting operation (A) in the evaluation operation temporary data 706 based on the correcting operation (A) for the “color setting”. Delete the evaluation operation (3) for The information processing apparatus 200 searches the correction operation temporary data 707 for an evaluation operation for the same “color setting” as the correction operation (A). Therefore, the modification operation (A) does not necessarily have to be performed immediately after the evaluation operation (3). Next, the information processing apparatus 200 determines the position of the evaluation operation (3) in the evaluation operation temporary data 706 based on the evaluation operation (3′) for “color setting” performed after the correction operation (A). insert the evaluation operation (3') into . Through this process, the information processing apparatus 200 finally replaces the evaluation operation (3) of the evaluation operation temporary data 706 with the evaluation operation (3').

Third, the information processing apparatus 200 converts the corrected evaluation operation temporary data 706 into an automatic execution script. The information processing apparatus 200 can save the generated automatic execution script in the secondary storage device 303 . For example, the user can cause the image forming apparatus 100 to reproduce the operation based on the operation in FIG. 7 by inputting an automatic execution script to the image forming apparatus 100 via a network.

As described above, the information processing apparatus 200 stores each operation in association with each operation component. Therefore, when an erroneous operation is accepted and another operation is performed, the information processing apparatus 200 can detect the erroneous operation to be corrected based on the correcting operation even when the correcting operation is accepted later. As a result, the user can correct the erroneous operation at any timing, not just immediately after the erroneous operation.

FIG. 8 is a diagram illustrating an example of the operation of the test script automatic generation system when an evaluation operation and a correction operation are respectively received. The test script automatic generation system determines whether the executed operation is an evaluation operation or a correction operation depending on the user's operation mode (for example, right hand operation or left hand operation). It is desirable that the test script automatic generation system does not include operations in the corrective operation mode in the automatic execution script in order to reduce user's operational errors.

The operation of the automatic test script generation system will be described below with reference to operation examples 1 and 2 of the automatic test script generation system when an evaluation operation and a correction operation are respectively received on the copy setting screen 801 . In both operational examples 1 and 2, it is assumed that the user wants to generate an auto-execution script that "sets color settings to monochrome."

In Operation Example 1, in step S811, the user performs the first operation "cancel color setting" with the left hand. In step S812, the second operation "set color setting to monochrome" is performed with the right hand. In step S813, the information processing apparatus 200 generates an automatic execution script 814 including the evaluation operation in step S812.

In Operation Example 2, in step S821, the user performs the first operation "cancel color setting" with the left hand. In step S822, the second operation "set color setting to monochrome" is performed with the left hand. In step S823, the information processing apparatus 200 generates an automatic execution script 824 that does not include the evaluation operation in step S822.

Comparing the above operation examples 1 and 2, when the information processing apparatus 200 detects an operation with the right hand after the first correction operation, the information processing apparatus 200 includes the operation with the right hand in the automatic execution script. On the other hand, when the information processing apparatus 200 detects an operation with the left hand after the first correction operation, the automatic execution script does not include the operation with the left hand. This is because if the operation in the modification operation mode has both functions of deleting and inputting the operation, the possibility of the user making a mistake in operation is high, and the program of the test script automatic generation system itself becomes complicated. It's for.

As described above, the information processing apparatus 200 includes evaluation mode operations in the automatic execution script and does not include correction mode operations in the automatic execution script. This processing clarifies, for example, that the operation with the right hand is the evaluation operation and the operation with the left hand is the correction operation. As a result, the test script automatic generation system can reduce user's operation errors and the program of the test script automatic generation system can be easily implemented.

FIG. 9 is a diagram showing an example of the operation of the test script automatic generation system when the operations of the transition destination screen are collectively corrected. In the example of FIG. 9, the user later corrects the settings on the margin setting screen 903 while performing all nine steps of operation.

On the home menu screen 901, the user performs the evaluation operation (1) "select copy" with the right hand. The screen transitions to the copy setting screen 902 by the evaluation operation (1). On the copy setting screen 902, the user performs evaluation operation (2) "set number of copies to 3", evaluation operation (3) "set color setting to color", and evaluation operation (4) "set print type to 3" with the right hand. Set to double-sided” and evaluation operation (5) “select margin setting button”. The screen transitions to the margin setting screen 903 by the evaluation operation (5).

On the margin setting screen 903, the user performs the evaluation operation (6) "Set the vertical margin to 5 mm", the evaluation operation (7) "Set the horizontal margin to 5 mm", and the evaluation operation (8) with the right hand. "OK selection" is performed. The screen transitions to the copy setting screen 902 by the evaluation operation (8).

Immediately after evaluation operation (8), the user notices that the "margin height" and "margin width" settings were originally intended to be 3 mm. Therefore, the user performs the correction operation (A) "cancel margin setting" with the left hand. At this time, the screen transitions to the margin setting screen 903 by a correction operation (A), which is a cancel operation. Therefore, the user changes the screen to the copy setting screen 902 by performing operation (B) “OK selection” with the left hand. By performing the evaluation operation (5′) “select margin setting button” again with the right hand, the screen changes to the margin setting screen 903 .

On the margin setting screen 903, the user performs the evaluation operation (6′) “Set the margin height to 3 mm”, the evaluation operation (7′) “Set the margin width to 3 mm”, and the evaluation operation ( 8') Execute "OK selection". The screen transitions to the copy setting screen 902 by the evaluation operation (8'). On the copy setting screen 902, the user performs the evaluation operation (9) "execute printing" with the right hand.

Operation record 904 includes evaluation operations (1)-(9) and (5')-(8') and corrective operation (A). The internal processing of the information processing apparatus 200 based on the operation shown in FIG. 7 will be described below.

First, the information processing apparatus 200 generates evaluation operation temporary data 906 and correction operation temporary data 907 based on the operation record 904 acquired from the camera 250 . The evaluation operation temporary data 906 includes operations (5) and (5′) for the “margin setting button”, operations (6) and (6′) for the “margin vertical setting”, and “margin horizontal setting”. and operations (8) and (8') for "OK selection" are included as records. Correction operation temporary data 907 includes correction operations (A) and (B) for "margin setting" as records.

Second, the information processing device 200 corrects the evaluation operation temporary data 906 based on the correction operation temporary data 907 . More specifically, first, the information processing apparatus 200 performs the evaluation operations (5) and (5) for the "margin setting" in the evaluation operation temporary data 906 based on the correction operations (A) and (B) for the "margin setting". , delete its detailed settings (lower items) (6) to (8). Next, the information processing apparatus 200 performs the evaluation operation temporary data 906 based on the evaluation operations (5′) to (8′) for “margin setting” performed after the correction operations (A) and (B). The evaluation operations (5′) to (8′) are inserted at the positions where the evaluation operations (5) to (8) of . Through this process, the information processing apparatus 200 finally replaces the evaluation operations (5) to (8) of the evaluation operation temporary data 906 with the evaluation operations (5') to (8').

Third, the information processing apparatus 200 converts the corrected evaluation operation temporary data 906 into an automatic execution script. The information processing apparatus 200 can save the generated automatic execution script in the secondary storage device 303 . For example, the user can cause the image forming apparatus 100 to reproduce the operation based on the operation in FIG. 9 by inputting an automatic execution script to the image forming apparatus 100 via a network.

As described above, when the information processing apparatus 200 receives a correction operation for canceling a certain setting, it can delete all the detailed settings of the transition destination screen (lower item) of the setting or restore the default settings. With this function, the automatic test script generation system can reduce the user's burden of correcting operational errors.

FIG. 10 is a diagram showing an example of the operation of the test script automatic generation system when the screen transitions due to the correction operation. In the example of FIG. 10, on the copy setting screen 1001, the user is performing the correction operation (1) "select margin setting button" with the left hand. The information processing apparatus 200 recognizes the correction operation (1) as a correction operation based on the operation record acquired from the camera 250 . On the other hand, the image forming apparatus 100 changes the copy setting screen 1001 to the margin setting screen 1002 in order to receive it as a normal operation.

Next, on the margin setting screen 1002, the user performs the evaluation operation (2) "Set the vertical margin to 3 mm" and the evaluation operation (3) "Set the horizontal margin to 3 mm" with the right hand. (4) Perform "OK selection".

When the above operations (1) to (4) are performed, the information processing apparatus 200 excludes the operations (1) to (4) from evaluation operations included in the automatic execution script 1003 . This is because the corrective operation (left hand operation) is originally for canceling the operation, and the automatic execution script 1003 should not include the transition in the corrective operation and the operation at the transition destination.

As described above, the information processing apparatus 200 does not include the operation at the transition destination in the automatic execution script when the screen transitions due to the operation in the modification mode. As a result, the automatic test script generation system can reduce user's operational errors.

FIG. 11 is a diagram illustrating an example of the operation of the test script automatic generation system when a plurality of setting operations are corrected collectively. In the example of FIG. 11, the user corrects "color setting", "print type setting", and "margin setting" while performing all nine steps of operation.

On the home menu screen 1101, the user performs the evaluation operation (1) "select copy" with the right hand. The screen transitions to the copy setting screen 1102 by the evaluation operation (1). On the copy setting screen 1102, the user performs evaluation operation (2) “set number of copies to 3”, evaluation operation (3) “set color setting to color”, and evaluation operation (4) “set print type” with the right hand. Set to double-sided” and evaluation operation (5) “select margin setting button”. The screen transitions to the margin setting screen 1103 by the evaluation operation (5).

On the margin setting screen 1103, the user performs evaluation operation (6) "Set margin length to 5 mm", evaluation operation (7) "Set margin width to 5 mm", and evaluation operation (8) with the right hand. "OK selection" is performed. The screen transitions to the copy setting screen 1102 by the evaluation operation (8).

Immediately after the evaluation operation (8), the user originally planned to set the "color setting" to monochrome, the "print setting" to single-sided, and the "margin height" and "margin width" to 3 mm. I realized that there was Therefore, the user performs the correction operation (A) "cancel color setting" three times with the left hand. The information processing apparatus 200 determines that all the operations after "color setting" are to be deleted by receiving three correction operations.

Next, with the right hand, the user performs evaluation operation (3′) “set color setting to monochrome”, evaluation operation (4′) “set print type to single-sided”, and evaluation operation (5′) “margin setting”. button selection". The screen transitions to the margin setting screen 1103 by the evaluation operation (5).

On the margin setting screen 1103, the user performs the evaluation operation (6′) “Set the margin height to 3 mm”, the evaluation operation (7′) “Set the margin width to 3 mm”, and the evaluation operation ( 8') Execute "OK selection". The screen transitions to the copy setting screen 1102 by the evaluation operation (8'). Subsequently, the user performs the evaluation operation (9) "print execution" with the right hand.

Operation record 1104 includes evaluation operations (1)-(9) and (3')-(8') and corrective operation (A). Internal processing of the information processing apparatus 200 based on the operation of FIG. 11 will be described below.

First, the information processing apparatus 200 generates evaluation operation temporary data 1106 and correction operation temporary data 1107 based on the operation record 1104 acquired from the camera 250 . The evaluation operation temporary data 1106 includes operations (3) and (3′) for “color setting”, operations (4) and (4′) for “print type setting”, and operation (5) for “margin setting button”. ), (5′), operations (6), (6′) for “set margin vertical”, operations (7), (7′) for “margin horizontal setting”, and “OK selection ” as a record. Correction operation temporary data 1107 includes as records three correction operations (A) for "color setting".

Second, the information processing device 200 corrects the evaluation operation temporary data 906 based on the correction operation temporary data 907 . More specifically, first, the information processing apparatus 200 performs evaluation operation (3) and subsequent operations for “color setting” in the evaluation operation temporary data 1106 based on three correction operations (A) for “color setting”. Remove all evaluation operations. In the present embodiment, the number of correction operations is three, but this is an example, and the number of input correction operations for collectively deleting settings by information processing apparatus 200 is not limited to this. In a certain aspect, the information processing apparatus 200 performs the evaluation operation after the evaluation operation of the operation component Q included in the evaluation operation temporary data based on the detection of any N correction operations of the operation component (setting) Q. can be deleted. In another aspect, the information processing apparatus 200 may determine any special operation, such as an operation using a touch pen, as a correction operation for batch deletion of settings.

Next, the information processing apparatus 200 performs the evaluation operations (3) to (8) in the evaluation operation temporary data 1106 based on the evaluation operations (3′) to (8′) performed after the correction operation (A). ) are inserted into the evaluation operations (3′) to (8′). Through this process, the information processing apparatus 200 finally replaces the evaluation operations (3) to (8) of the evaluation operation temporary data 1106 with the evaluation operations (3') to (8').

Third, the information processing apparatus 200 converts the corrected evaluation operation temporary data 1106 into an automatic execution script. The information processing apparatus 200 can save the generated automatic execution script in the secondary storage device 303 . For example, the user can cause the image forming apparatus 100 to reproduce the operation based on the operation in FIG. 11 by inputting an automatic execution script to the image forming apparatus 100 via a network.

Next, referring to FIG. 12 to FIG. 16, processing until the information processing apparatus 200 generates an automatic execution script will be described. In one aspect, CPU 301 may load a program for performing the processes of FIGS. 12 to 16 from secondary storage device 303 to primary storage device 302 and execute the program. In other aspects, part or all of the process may also be implemented as a combination of circuit elements configured to perform the process.

FIG. 12 is a diagram showing an example of an automatic execution script generation process in the information processing apparatus 200. As shown in FIG. In step S1210, CPU 301 accepts an operation mode setting from the user via input interface 305 or the like. The setting of the operation mode includes the operation mode of the evaluation operation and the operation mode of the correction operation.

The operation mode includes arbitrary operation modes such as operation with the right hand, operation with the left hand, the number of fingers, the angle of the fingers as seen from the camera 250, and whether or not a touch pen is used. As an example, a right-handed user may set the right hand operation to the evaluation operation and the left hand operation to the correction operation. Similarly, a left-handed user may set the left hand operation to the evaluation operation and the right hand operation to the correction operation. In addition, the user can set the operation with the finger as the evaluation operation and the operation with the stylus as the correction operation. The CPU 301 determines whether the user's operation is an evaluation operation or a correction operation, based on the operation mode set in that step.

In step S1220, CPU 301 determines whether recording of the user's operation by camera 250 has started. When CPU 301 determines that recording of the user's operation by camera 250 has started (YES in step S1220), CPU 301 moves the control to step S1230. Otherwise (NO in step S1220), CPU 301 shifts the control to step S1220.

In step S<b>1230 , CPU 301 acquires an operation record from camera 250 . The operation record may be a video or an image. The CPU 301 updates the evaluation operation temporary data based on the operation record.

In step S1240, CPU 301 determines whether or not recording of the operation record has ended. When CPU 301 determines that the recording of the operation record has ended (YES in step S1240), CPU 301 moves the control to step S1250. Otherwise (NO in step S1240), CPU 301 shifts the control to step S1260.

In step S1250, CPU 301 generates an automatic execution script based on the evaluation operation temporary data. In step S1260, CPU 301 determines whether or not the operation included in the operation record is a correction operation. As an example, CPU 301 can determine that the operation included in the operation record is a correction operation if the user's hand that moves to the image of the operation record is the left hand. When CPU 301 determines that the operation included in the operation record is a correction operation (YES in step S1260), CPU 301 moves the control to step S1270. Otherwise (NO in step S1260), CPU 301 shifts the control to step S1230. In step S1270, CPU 301 executes a processing routine for the modification operation mode. In one aspect, the CPU 301 may output the evaluation operation temporary data as is without generating an automatic execution script. In that case, the evaluation operation temporary data may be used, for example, by other auto-executing script generation programs or the like.

FIG. 13 is a diagram showing an example of the processing routine of the correction operation mode in step S1270. When CPU 301 determines in step S1260 in FIG. 12 that the operation included in the operation record is a correction operation, CPU 301 executes the subsequent processing.

In step S1310, CPU 301 determines that the operation included in the operation record is a corrective operation. In step S1320, CPU 301 determines whether or not the user has operated operation panel 109 based on the operation record. When CPU 301 determines that the user has operated operation panel 109 (YES in step S1320), CPU 301 moves the control to step S1330. Otherwise (NO in step S1320), CPU 301 terminates the operation mode processing routine.

In step S1330, CPU 301 executes a record cancellation processing routine. This step corresponds to the operation of deleting the evaluation operation included in the evaluation operation temporary data in FIGS.

FIG. 14 is a diagram showing an example of processing of the record cancellation processing routine of step S1330. When CPU 301 determines in step S1320 in FIG. 13 that the user has operated operation panel 109, CPU 301 executes the subsequent processes.

In step S1410, CPU 301 determines whether the number of records (correction operations) in the correction operation temporary data is one or more. When CPU 301 determines that the number of records (correction operations) in the correction operation temporary data is one or more (YES in step S1410), CPU 301 shifts the control to step S1420. Otherwise (NO in step S1410), CPU 301 terminates the record cancellation processing routine.

In step S1420, when there are N records in the correction operation temporary data, CPU 301 searches for the Nth record (latest data). In step S1430, CPU 301 determines whether a record (evaluation operation) corresponding to the record (correction operation) selected in step S1420 exists in the evaluation operation temporary data. As an example, when the CPU 301 acquires the record of the "color setting" correction operation in step S1420, the CPU 301 determines whether or not there is a record of the "color setting" evaluation operation in the evaluation operation temporary data. .

When CPU 301 determines that a record (evaluation operation) corresponding to the record (correction operation) selected in step S1420 exists in the evaluation operation temporary data (YES in step S430), CPU 301 shifts the control to step S1440. Otherwise (NO in step S1430), CPU 301 terminates the record cancellation processing routine.

In step S1440, CPU 301 determines whether or not the record of the evaluation operation found in step S1430 has sub-item settings. When CPU 301 determines that the record (evaluation operation) found in step S1430 has setting of lower items (YES in step S1440), CPU 301 shifts the control to step S1450. Otherwise (NO in step S1440), CPU 301 shifts the control to step S1480.

In step S1450, CPU 301 deletes the record (evaluation operation) found in step S1430 and the records of its lower items from the evaluation operation temporary data. The step corresponds to, for example, the processing in FIG.

In step S1460, CPU 301 determines whether or not the screen has transitioned due to a correction operation. When CPU 301 determines that the screen transition has occurred due to the correction operation (YES in step S1460), CPU 301 moves the control to step S1470. Otherwise (NO in step S1460), CPU 301 shifts the control to step S1410.

In step S1470, CPU 301 executes a processing routine after screen transition by the correction operation. Steps S1460 and S1470 correspond to the processing in FIG. 10, for example. In step S1480, CPU 301 deletes the record (evaluation operation) found in step S1440 from the evaluation operation temporary data. This step corresponds to, for example, the processing without deleting the sub-items in FIGS. 6 and 7 .

FIG. 15 is a diagram showing an example of processing routine processing after screen transition by the correction operation in step S1470. When CPU 301 determines in step S1460 in FIG. 14 that the screen has changed due to the correction operation, CPU 301 executes the subsequent processing.

In step S1510, CPU 301 determines whether the screen currently displayed on operation panel 109 has returned to the upper screen. When CPU 301 determines that the screen currently displayed on operation panel 109 has returned to the upper screen (YES in step S1510), CPU 301 ends the processing routine after screen transition in the correction mode. Otherwise (NO in step S1510), CPU 301 returns the control to step S1510. By this step, the CPU 301 does not include the user's operation in the correction operation temporary data until the original screen is returned.

FIG. 16 is a diagram showing an example of processing of a record cancellation processing routine by special operation. The processing in FIG. 16 corresponds to the processing in FIG. 11, for example. The process of Figure 16 may be performed in combination with Figures 12-15.

In step S1610, CPU 301 searches content DB 206 for the function item (or operation component) for which the special operation has been performed. In a certain aspect, the special operation may be any predetermined operation mode. As an example, the CPU 301 may determine that the same operation component is operated three times in the correction mode operation as a special operation, or may determine that an operation using a touch pen is a special operation.

In step S1620, CPU 301 determines whether or not there is a function item for which a special operation has been performed in the evaluation operation temporary data. When CPU 301 determines that there is a function item for which a special operation has been performed in evaluation operation temporary data (YES in step S1620), CPU 301 moves the control to step S1630. Otherwise (NO in step S1620), CPU 301 terminates the record cancellation processing routine by special operation. In step S1630, CPU 301 deletes the record of the function item for which the special operation was performed and all subsequent records from the evaluation operation temporary data.

As described above, the test script automatic generation system according to the present embodiment can determine whether the user's operation is an evaluation operation or a correction operation based on the user's operation mode. With this function, the test script automatic generation system can modify the operation details of the device under test at any time. As a result, it is possible to greatly reduce the user's burden when generating an automatic execution script for a device under test that has many operation items.

It should be considered that the embodiments disclosed this time are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the scope of the claims rather than the above description, and is intended to include all modifications within the meaning and scope equivalent to the scope of the claims.

100 image forming apparatus, 101 control unit, 102 document reading unit, 103 print output unit, 104 FAX unit, 105 storage device, 106 wireless IF, 107 network IF, 108 user authentication unit, 109 operation panel, 110 CPU, 111 RAM, 112 ROM, 113 display unit, 114 operation unit, 200 information processing device, 201 screen analysis unit, 202 operation analysis unit, 203 operation determination unit, 204 correction unit, 205 script generation unit, 206 content DB, 207 correction operation store, 208 evaluation operation store, 209 script DB, 210, 504, 604, 704, 904, 1104 operation record, 250 camera, 301 CPU, 302 primary storage device, 303 secondary storage device, 304 external device interface, 305 input interface, 306 output interface, 307 communication interface, 401 screen, 501, 601, 701, 901, 1101 home menu screen, 502, 602, 702, 801, 902, 1001, 1102 copy setting screen, 503, 603, 703, 903, 1002, 1103 margin setting screen, 505, 605, 705 time series, 506, 606, 706, 906, 1106 evaluation operation temporary data, 607, 707, 907, 1107 correction operation temporary data, 814, 824, 1003 automatic execution script.

Claims (20)

A program, said program comprising:
a step of detecting a user's operation on the operation screen of the target device;
determining a plurality of user operation modes in the operation; and
The plurality of operation modes include a first operation mode and a second operation mode,
The program causes the computer to:
The program further causes the execution of a step of generating an operation record in which the operations performed according to the second operation mode are excluded from the operations performed according to the first operation mode. In the step of determining the plurality of operation modes, each operation mode corresponds to the first operation mode or the second operation mode based on whether the finger touching the operation screen is the finger of the right hand or the finger of the left hand. 2. The program according to claim 1, comprising the step of determining which of the operating modes of The step of determining the plurality of operation modes determines whether each operation mode is the first operation mode or the second operation mode based on the shape or color of the touch pen touching the operation screen. 2. The program of claim 1, comprising steps. The step of determining the plurality of operation modes includes determining whether each operation mode is the first operation mode or the second operation mode based on the shape of an object touching the operation screen. 2. The program of claim 1, comprising: The program causes the computer to:
After the first operation is performed according to the first operation mode, if the first operation is performed according to the second operation mode, the step of excluding the first operation from recording is further performed. , the program according to any one of claims 1 to 4. The program causes the computer to:
when the first operation is executed in the second operation mode, and in the second operation mode included in the operation record when the first operation is executed in the first operation mode; replacing the first operation executed according to the first operation mode before execution of with the first operation executed according to the first operation mode after execution according to the second operation mode 6. The program according to claim 5, further executed. In the step of deleting the first operation from the operation record, when an operation screen transitions due to the execution of the first operation, the operation on the operation screen after the transition due to the execution of the first operation is also performed according to the operation. 7. A program according to claim 5 or 6, comprising the step of deleting from record. The program causes the computer to:
In the case where the operation screen transitions due to the execution of the second operation, when the second operation is executed in the second operation mode, the operation screen after the transition due to the execution of the second operation is executed. 8. The method according to any one of claims 1 to 7, further causing a step of excluding each operation from the operation record regardless of whether the operation was performed in the first operation mode or the second operation mode. program. The plurality of operation modes further includes a third operation mode,
Said program, on said computer,
Claims 1 to 8, wherein when a third operation is performed according to the third operation mode, deleting all operations performed after the execution of the third operation from the operation record is further performed. The program according to any one of Claims 1 to 3. The program causes the computer to:
10. The program according to any one of claims 1 to 9, further causing a step of outputting information for each operation executed in said second operation mode. A method for generating an operational record of a target device, comprising:
a step of detecting a user's operation on the operation screen of the target device;
and determining a plurality of user operation modes in the operation,
The plurality of operation modes include a first operation mode and a second operation mode,
The method includes
The method further comprising the step of generating an operation record in which, among the operations performed according to the first operation mode, operations performed according to the second operation mode are excluded. In the step of determining the plurality of operation modes, each operation mode corresponds to the first operation mode or the second operation mode based on whether the finger touching the operation screen is the finger of the right hand or the finger of the left hand. 12. The method of claim 11, comprising determining which of the operating modes of . The step of determining the plurality of operation modes determines whether each operation mode is the first operation mode or the second operation mode based on the shape or color of the touch pen touching the operation screen. 12. The method of claim 11, comprising steps. The step of determining the plurality of operation modes includes determining whether each operation mode is the first operation mode or the second operation mode based on the shape of an object touching the operation screen. 12. The method of claim 11, comprising: deleting the first operation from the operation record when the first operation is performed according to the second operation mode after the first operation is performed according to the first operation mode; A method according to any one of claims 11 to 14, comprising when the first operation is executed in the second operation mode, and in the second operation mode included in the operation record when the first operation is executed in the first operation mode; replacing the first operation executed according to the first operation mode before execution of with the first operation executed according to the first operation mode after execution according to the second operation mode 16. The method of claim 15, further comprising: In the step of deleting the first operation from the operation record, when an operation screen transitions due to the execution of the first operation, the operation on the operation screen after the transition due to the execution of the first operation is also performed according to the operation. 17. A method according to claim 15 or 16, comprising deleting from record. In the case where the operation screen transitions due to the execution of the second operation, when the second operation is executed in the second operation mode, the operation screen after the transition due to the execution of the second operation is executed. 18. The method of any one of claims 11 to 17, further comprising excluding each operation from the operation record regardless of whether it was performed by the first operation mode or the second operation mode. Method. The plurality of operation modes further includes a third operation mode,
4. The method further comprising, when a third operation is performed according to the third operation mode, deleting all operations performed after the execution of the third operation from the operation record. 19. The method according to any one of 11-18. a memory storing the program according to any one of claims 1 to 10;
and a processor for executing the program.

Images