JP5211951B2 - Inspection system, inspection device, display device, and program - Google Patents

Description

  The present invention relates to an inspection system, an inspection device, a display device, and a program.

In order to ensure the quality of electronic equipment, its operation is inspected at the manufacturing stage and pre-shipment stage. There exists what was disclosed by patent document 1 as an inspection system which automated the process which concerns on an inspection. In the inspection system disclosed in Patent Document 1, an apparatus to be inspected performs the same operation as when an operator image such as a soft button displayed by the apparatus is designated by the user, and the operator image A display screen corresponding to is displayed, and the inspection device inspects the display screen. When instructed by the inspection apparatus, the inspection target apparatus designates an operation element image based on position information indicating a predetermined position where the operation element image is arranged in the display area, and corresponds to the operation element image. Display the display screen.
JP 2007-328509 A

  The present invention has been made in view of the above-described circumstances, and an object thereof is to simplify an algorithm related to inspection of a display screen of a display device.

In order to solve the above-described problem, an inspection system according to claim 1 of the present invention includes a display device and an inspection device, and the display device designates a display means and a display screen to be displayed on the display means. An operator image display control means for displaying the operator image used in the display means, the operator image display control means for displaying the operator image to which identification information for identifying each operator image is added. , from said display screen displayed on the display means, corresponding to the designated operating image by the manipulator image display control means displays on the display means is controlled by been operating image sac Chi before Symbol inspection device and a display screen display switching means for switching the display screen, the test devices may get hands for acquiring the identification information added to the displayed operating image on a display unit of the display device When the instruction means for instructing said display screen the display switching means to display the display screen in which the identification information acquired by the acquiring unit corresponds to the added operating image on the display means, the instruction by said instruction means Accordingly, whether the display position of the operation element image on the display screen switched by the display screen display switching means matches the display position included in the identification information of the operation element image on the switched display screen. And inspection means for inspecting whether or not .

The inspection system according to claim 2 of the present invention is the inspection system according to claim 1, wherein the display screen display switching means of the display device displays a display screen including an image to which identification information for identifying each is added. The acquisition unit of the inspection apparatus acquires the identification information added to the image included in the display screen displayed on the display unit, and the inspection unit acquires the identification information acquired by the acquisition unit. A position in the display area of the display means of the image to which is added is specified, and the display screen is inspected based on the specified position.

  The inspection system according to a third aspect of the present invention is the inspection system according to the first aspect, wherein the display device transmits the identification information added to the operator image displayed on the display means to the inspection device. The inspection apparatus has a receiving means for receiving the identification information transmitted by the transmitting means of the display device, and the acquiring means receives the identification information received by the receiving means. It is characterized by acquiring.

  The inspection system according to a fourth aspect of the present invention is the inspection system according to the first aspect, wherein the inspection apparatus includes an imaging unit that performs imaging, and the acquisition unit is configured to display on the display device by the imaging unit. The displayed operator image is captured, and the identification information added to the operator image represented by the image information generated by the imaging is acquired.

Inspection apparatus according to claim 5 of the present invention, the display means and, an operating image display control means for the operating image for displaying on the display means used in the operation of specifying a display screen to be displayed on said display means The operation element image display control means for displaying the operation element image to which identification information for identifying each operation element image is added and the display screen displayed on the display means are controlled by the operation element image display control means. operator to the own inspection apparatus is displayed on the display means of the display device and a display screen display switching means for switching the display screen corresponding to the operating image has been specified among the displayed operating image on the display means Te An acquisition unit for acquiring the identification information added to the image, and a display screen corresponding to the operation element image to which the identification information acquired by the acquisition unit is added are displayed on the display unit. Said display screen and instruction means for instructing the switching means, in response to an instruction by said instructing means, said display screen and a display position of the operating element image in the display screen is switched by the switching means, the switched display screen as Inspection means for inspecting whether or not the display position included in the identification information of the operation element image matches .

According to a sixth aspect of the present invention, there is provided a display device, comprising: a display means; and an operator image display control means for causing the display means to display an operator image used for specifying a display screen to be displayed on the display means. , An operator image display control means for displaying the operator image to which identification information for identifying each operator image is added, and acquiring the identification information added to the operator image displayed on the display means It means, from a display screen displayed on the display unit, the identification information which the operating image display control means displays on the display means is controlled by been operating image sac Chi before Symbol acquisition means has acquired the additional the display position of the operating element image in the switched display screen by the display screen display switching means and the display screen switching means for switching the display screen corresponding to the operating image, which is, Having a test means and display position included in serial switched identification information of operating image display screen to check whether the match and said.

The computer program inspecting apparatus according to claim 7 of the present invention, the display means and, said operating element image used for operation of specifying a display screen to be displayed on the display means is displayed on the display unit operating image display control Means for displaying the operator image to which identification information for identifying each operator image is added, and the operator image display control unit from a display screen displayed on the display unit. displayed on the display unit of the display device and a display screen display switching means for switching the display screen corresponding to the operating element image the inspection apparatus instructs among the displayed operating image on the display means is controlled by means Acquisition means for acquiring the identification information added to the operated operator image, and a table corresponding to the operator image to which the identification information acquired by the acquisition means is added And instruction means for instructing said display screen the display switching means to display the screen on the display unit, in accordance with an instruction by said instructing means, the display position of the operating element image in the switched display screen by the display screen display switching means And an inspection means for inspecting whether or not the display position included in the identification information of the operator image on the switched display screen matches .

According to the first aspect of the present invention, an algorithm related to the inspection of the display screen of the display device can be simplified as compared with the case where the present configuration is not provided.
According to the invention which concerns on Claim 2, the identification information added to the image contained in a display screen is acquired, the position of an image is pinpointed, and a display screen can be test | inspected based on the pinpointed position.
According to the third aspect of the invention, the identification information added to the operator image displayed by the display device is transmitted to the inspection device, and the inspection device can receive the identification information and acquire it.
According to the invention which concerns on Claim 4, the identification information added to the operation element image which a display apparatus displays can be acquired from the operation element image imaged by the imaging means.
According to the invention which concerns on Claim 5, compared with the case where it does not have this structure, the algorithm which concerns on the test | inspection of the display screen of a display apparatus can be simplified.
According to the invention which concerns on Claim 6, compared with the case where it does not have this structure, the algorithm which concerns on the test | inspection of the display screen of a display apparatus can be simplified.
According to the invention which concerns on Claim 7, compared with the case where it does not have this structure, the algorithm which concerns on the test | inspection of the display screen of a display apparatus can be simplified.

DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, embodiments of the invention will be described with reference to the drawings.
(1) Configuration FIG. 1 is a diagram showing a configuration of an inspection system 1 according to an embodiment of the present invention. As shown in FIG. 1, the inspection system 1 includes an image forming apparatus 100 and an inspection apparatus 200. The image forming apparatus 100 includes a plurality of functions related to processing that can be executed by the apparatus, such as a “copy function”, a “facsimile function”, and a “scanner function”. It is a device before passing over. The image forming apparatus 100 includes a display unit that displays a display screen including various information, and is a kind of display device. The display screen displayed by the image forming apparatus 100 is displayed by, for example, a display screen on which one or a plurality of operation element images such as soft buttons operated by the user are arranged, or when an operation element image is designated by the user. There is a display screen. For example, on the display screen corresponding to the “copy function”, if an operator image for instructing the start of copy processing is specified, a message “copy in progress” and an image indicating the number of prints are included accordingly. Display screen is displayed. The “operator image” here is an image for realizing the same function as each operator such as a keyboard and a numeric keypad, and the image forming apparatus 100 uses a touch panel or the like to determine which operator image is a user. It recognizes whether it has been designated, and performs control according to the designated operator image.
The inspection apparatus 200 performs an inspection process to inspect whether the display screen of the image forming apparatus 100 is displayed in the expected display mode. The image forming apparatus 100 and the inspection apparatus 200 exchange information with each other by being communicably connected via a communication cable such as a USB (Universal Serial Bus) as necessary. When the image forming apparatus 100 is inspected by the inspection apparatus 200, both are connected by a communication cable by a worker or the like.

Next, the configuration of the image forming apparatus 100 will be described.
FIG. 2 is a block diagram illustrating a configuration of the image forming apparatus 100. As illustrated in FIG. 1, the image forming apparatus 100 includes a control unit 110, an image reading unit 120, a storage unit 130, a communication unit 140, a UI (User Interface) unit 150, and an image forming unit 160. .
The control unit 110 includes a CPU (Central Processing Unit) that controls the entire apparatus, a RAM (Random Access Memory) that provides a work area, and a ROM (Read Only Memory) that stores various control programs, and is described in the control program. The calculation process is performed according to the procedure. The control unit 110 is an example of an operator image display control unit and a display screen display control unit. The image reading unit 120 irradiates a document placed on a platen glass (not shown) with light, reads the reflected light, generates image information, and supplies the image information to the control unit 110. The control unit 110 causes the storage unit 130 to store image information supplied from the image reading unit 120 and image information received from an external device (not shown) by the communication unit 140.

The storage unit 130 includes a storage device such as an HDD (Hard Disk Drive), and defines the overall configuration of the display screen and processing conditions in addition to a program that defines operations when the inspection processing is performed by the inspection device 200. The display control program is defined by the definition of the contents and the definition of the display position in the processing condition designation column. This display control program is described in, for example, an XML (Extensible Markup Language) format. The processing conditions are conditions that determine the content of processing executed by the image forming apparatus 100. For example, in the “copy function”, the enlargement / reduction magnification, the paper size, duplex / single-sided printing, the number of prints, and the like. There are processing conditions. The designation field is a field that is displayed for each processing condition on the display screen in order to allow the user to specify each processing condition, and displays operation buttons such as radio buttons, up / down keys, and settings. Display window. The control unit 110 executes a display control program, generates image information according to the algorithm, and causes the UI unit 150 to display a display screen based on the image information. The storage unit 130 stores an “ID management table 131”, the details of which will be described later.
The communication unit 140 is an example of a transmission unit including an interface device for performing communication via a network. The communication unit 140 is an external device such as an inspection device 200 or a computer (not shown) via a network such as a LAN (Local Area Network). Sends and receives various information to and from.

The UI unit 150 includes a touch panel and various buttons, accepts an operation from the user, and notifies information by an image or sound. The UI unit 150 is an example of a display unit that has a rectangular display area, and displays an image in the display area according to image information in a bitmap format. Coordinates representing the respective positions are assigned to the display area of the UI unit 150 according to the orthogonal coordinate system, and each coordinate functions as position information representing the position of each pixel of the image information. Each of these coordinates is stored in the ROM. The UI unit 150 employs a so-called touch panel, and the control unit 110 refers to the position of the touch panel touched by the user according to an operation signal supplied from the UI unit 150, and determines the position. It is determined which manipulator image is designated based on the position information corresponding to.
The image forming unit 160 forms an image on a recording sheet based on the image information stored in the storage unit 130 under the control of the control unit 110. In the copy process, the control unit 110 supplies the image information supplied from the image reading unit 120 to the image forming unit 160 to form an image on a recording sheet.

FIG. 3 is a diagram illustrating an example of a display screen corresponding to the “copy function” displayed on the UI unit 150. The display screen P1 is a display screen that is displayed first when the user instructs to execute the copy function.
The display screen P1 shown in FIG. 3 is a display screen when the operator image A1 (tab) indicated as “basic” is designated by the user. As shown in the figure, in the display screen P1, designation columns for designating processing conditions relating to the copy function such as “magnification selection”, “paper selection”, “double-side / single-side selection”, and “number of copies” are arranged. Has been placed. For example, when the radio button type operator image A4 is designated by the user, the operator image A4 is displayed in a solid black color as shown in the figure, and “100%” is displayed in the display window. The By such a user operation, the image forming apparatus 100 is set to form an image with a “magnification” of 100% (equal magnification). Further, when the operator images A2 and A3 are selected by the user, the display screen is displayed when the “application 1” tab and the “application 2” tab are designated, respectively, and processing different from the case of the “basic” tab. A specification field for specifying a condition is displayed.
In addition, on the display screen P1 shown in FIG. 3, an operator image A21 marked “Start” is arranged at the lower right of the display area of the UI unit 150. When the operator image A21 is designated by the user, the control unit 110 displays a display screen corresponding to the designated operator image A21 and starts a copy process.

FIG. 4 is a diagram showing an example of the display screen P2 displayed when the operator image A21 is designated. In the figure, a display screen when the copy process is performed with the number of copies “14” is shown.
As shown in the figure, on the display screen P2, a “character image G1” representing a character string “copy in progress” is displayed near the center of the display area of the UI unit 150. On the right side of the character image G1, an “animation image G2” “1/14” indicating the execution state of the image forming apparatus 100 is displayed. This animation image G2 indicates that “1” of the number of copies “14” is being printed, and is changed one by one as the copy process proceeds. In addition, an “operator image G3” marked “stop” is arranged in the lower right part of the display area of the UI unit 150. When the operator image G3 is designated by the user, the control unit 110 stops the copy process.
As for the copy function, it is assumed that various display screens related to the copy function (for example, the display screen P2) are displayed starting from the display screen P1. As for other functions, various display screens related to the function are displayed starting from a certain display screen.

Next, FIG. 5 is a diagram illustrating an example of the ID management table 131.
As shown in the figure, in the ID management table 131, “operation element image ID” that is identification information for identifying each operation element image, and “display” that is identification information for identifying a display screen corresponding to the operation element image. Is associated with “screen ID”. That is, the ID management table 131 associates the operation element image to which each operation element image ID is assigned with the display screen displayed when the operation element image is designated. The operator images A1 to A21 in the ID management table 131 correspond to the operator images A1 to A21 shown in FIG. For example, as shown in the figure, the operator image ID “ID004” and the display screen ID “ID904” are assigned to the operator image A4 corresponding to “100%” of the processing condition “select magnification”. The operator image ID “ID021” and the display screen ID “ID921” are assigned to the operator image A21 marked “Start”. That is, when the operator image A4 to which the operator image ID “ID004” is assigned is designated by the user, the display screen to which the display screen ID “ID904” is assigned (here, the operator image A4 is solid black). Display screen), and when the operator image A21 assigned with the operator image ID “ID021” is designated by the user, the display screen P2 assigned with the display screen ID “ID921” is displayed. The As described above, when the operator image is specified by the user, a display screen to which the display screen ID associated with the operator image ID is assigned is displayed on the UI unit 150.

  The operator image ID assigned to the operator image is added by being embedded in the operator image displayed on the UI unit 150 as digital information using a digital watermark technique. The control unit 110 adds identification information for identifying each operation element image to the operation element image used for an operation for designating a display screen to be displayed on the UI unit 150 and causes the UI unit 150 to display the identification information. As the digital watermark technique here, for example, a known method such as a density pattern method or a dither method is used. In general, since a soft button is often expressed in a solid color, even if an operator image ID is added by being embedded in the operator image, it is difficult for the user to visually recognize the operator image ID. The displayed image does not impair the visibility and design of the display screen of the UI unit 150. The manipulator image ID is used for inspection processing performed by the inspection apparatus 200, which will be described in detail later.

Next, the configuration of the inspection apparatus 200 will be described.
FIG. 6 is a block diagram showing a configuration of the inspection apparatus 200. As shown in the figure, the inspection apparatus 200 includes a control unit 210, a storage unit 220, a communication unit 230, a display unit 240, and an operation unit 250. The control unit 210 is the same as the control unit 110 and includes a CPU, a RAM, and a ROM, and performs arithmetic processing according to a procedure described in the control program. The control unit 210 is an example of an inspection unit. The storage unit 220 includes a storage device such as an HDD, and stores various types of information such as an inspection program used for executing inspection processing. The storage unit 220 stores an “expected display mode management table 221”. The communication unit 230 is an interface device for communicating with the image forming apparatus 100, and is an example of a receiving unit that receives information such as image information from the image forming apparatus 100 and transmits information to the image forming apparatus 100. . The control unit 210 and the communication unit 230 are an example of an acquisition unit and an instruction unit. The display unit 240 includes a liquid crystal panel and the like, and displays a menu screen and the like for operating the inspection apparatus 200 under the control of the control unit 210. The operation unit 250 includes an input device such as a keyboard or an operator such as a button, receives an operation by a user, generates an operation signal representing the operation, and supplies the operation signal to the control unit 210.

FIG. 7 is a diagram showing an example of the expected display mode management table 221. As shown in FIG.
The expected display mode management table 221 is a display screen that is displayed on the UI unit 150 by the image forming apparatus 100 and is defined for each display screen that can be an inspection target in the inspection process (hereinafter, “ "Expected display mode") is described. This expected display mode is associated with the operator image ID (and display screen ID) in the expected display mode management table 221. FIG. 7 shows an expected display mode for the display screen P2.
As shown in the figure, in the expected display mode management table 221, the “display position” of the image in the display area is associated with “image ID” which is identification information for identifying each image included in the display screen P2. The “mode” in which the definition of the mode of the image is described, and the “response time” from when the display on the display screen P2 is instructed until the display is completed are described.

For example, the character image G1 shown in FIG. 4 represents a character string “copying in progress”, and an image ID “ID101” is assigned as shown in FIG. The image ID is the same as that of the operator image ID, and is added by embedding identification information in the image. For the character image G1, the contents of the “center” of the display area as the display position and the image type “character” and the image size “10 pixels” × “40 pixels” are defined as the display position. The image size here is the length (pixel) of each side of the circumscribed rectangle circumscribing the character string “Copying in progress”. The animation image G2 represents an animation image “1/14” on the display screen P2 shown in FIG. 4 and is assigned an image ID “ID102”. For the animation image G2, the display position “23 mm to the right of the character image G1” is defined, and the contents of the image type “animation” and the image size “20 pixels” × “20 pixels” are defined. The manipulator image G3 is an manipulator image labeled “stop” on the display screen P2 shown in FIG. 4 and is assigned an image ID “ID103”. For the operator image G3, the display position is “lower right area” of the display area, the image type is “soft button”, the image size is “20 pixels” × “30 pixels”, and the outer shape of the image is “wara shape”. Is defined. Also, “200 msec” is defined as the response time for the display screen P2.
When the inspection apparatus 200 inspects the image forming apparatus 100, the expected display form described in the expected display form management table 221 matches the display form of the display screen that is actually displayed on the image forming apparatus 100. The test result is obtained depending on whether or not it exists.

(2) Operation FIG. 8 is a flowchart showing an operation procedure of inspection processing executed in the inspection system 1.
First, before the start of the inspection process, the inspection worker connects the communication unit 140 of the inspection apparatus 200 and the communication unit 230 of the image forming apparatus 100 with a communication cable. Then, when the operator operates the operation unit 250 of the inspection apparatus 200 to instruct to start the inspection process, the inspection system 1 starts the inspection process.

When the control unit 210 of the inspection apparatus 200 determines that the inspection process is to be started (step S1; YES), the control unit 210 transmits a notification to that effect to the image forming apparatus 100 by the communication unit 230 (step S2). When receiving the start notification by the communication unit 140, the control unit 110 of the image forming apparatus 100 causes the UI unit 150 to display a predetermined display screen based on the start notification and the display control program (step S3). Here, the control unit 110 displays the display screen P1 corresponding to the copy function on the UI unit 150, and the inspection apparatus 200 inspects the display screen displayed starting from the displayed display screen P1.
And the control part 110 transmits the image information of the bitmap format showing the displayed display screen P1 to the test | inspection apparatus 200 by the communication part 140 (step S4). That is, the control unit 110 transmits the operation element image ID added to the operation element image displayed on the UI unit 150 to the inspection apparatus 200.

When the control unit 210 of the inspection apparatus 200 receives the image information representing the display screen P1 through the communication unit 230 (step S5), the manipulator image ID added to the manipulator image included in the display screen P1 represented by the image information. Is detected and acquired (step S6). At this time, the control unit 210 scans image information representing the display screen P1 in units of pixels, detects an image pattern indicating the operator image ID described in the expected display mode management table 221, and controls the operator Get the image ID. And the control part 210 performs the process step demonstrated below in order to test | inspect the display screen corresponding to this operation element image ID.
In the following, an operation when the control unit 210 acquires the operation element image ID “ID021” added to the operation element image A21 will be described.

The control unit 210 determines the position of the display area in the UI unit 150 where the operation element image to which the operation element image ID “ID021” is added is arranged based on the acquired image pattern indicating the operation element image ID. Specify (step S7). Specifically, the control unit 210 determines the center coordinates of the image pattern based on the size of the display area of the UI unit 150, the image information representing the display screen, the position where the image pattern indicating the operator image ID is included, and the like. Is specified as the position of the operator image. Then, the control unit 210 generates the center coordinates as position information representing the position of the operator image.
Based on the position information indicating the position of the operator image, the control unit 210 transmits an instruction (designation instruction) for designating the position in the display area represented by the position information to the image forming apparatus 100 by the communication unit 230. (Step S8).

When the designation instruction is received by the communication unit 140, the control unit 110 of the image forming apparatus 100 generates the same operation signal as when the position in the display area included in the designation instruction is designated by the user, and follows the operation signal. The operation is executed to switch from the display screen P1 to the display screen P2 and display it on the UI unit 150 (step S9). That is, the control unit 110 causes the UI unit 150 to display a display screen corresponding to the operator image instructed by the inspection apparatus 200 among the operator images displayed on the UI unit 150. As a result, a display screen P2 as shown in FIG.
Note that the image forming apparatus 100 recognizes that the inspection process is being performed based on the start notification transmitted in step S <b> 2, and performs the copy process by the image forming unit 160 only by displaying the display screen on the UI unit 150. Make it not exist.

  Then, the control unit 110 of the image forming apparatus 100 transmits bitmap format image information representing the display screen P2 to the inspection apparatus 200 via the communication unit 140 (step S10). When the control unit 210 of the inspection apparatus 200 acquires the image information representing the display screen P2 through the communication unit 230, the control unit 210 displays the display screen based on the screen information and the contents described in the expected display mode management table 221. P2 is inspected (step S11). That is, the control unit 210 inspects the display screen displayed on the UI unit 150 by the control unit 110 in accordance with an instruction from the own device.

  In step S11, the controller 210 checks whether the display screen P2 is displayed as indicated by the expected display mode described in the expected display mode management table 221 shown in FIG. For example, for the character image G1, as shown in FIG. 7, it is defined that the character image G1 is “center” of the display area of the UI unit 150 and the image size is “10 pixels” × “40 pixels”. Therefore, the control unit 210 analyzes the image information and inspects whether the image is displayed in the expected display manner on the display screen P2. Specifically, the control unit 110 analyzes the display position and mode of each image described in the expected display mode management table 221, the display area size and image information of the UI unit 150, and inspects the display screen. As for the response time, the image forming apparatus 100 measures the response time using a timer such as a timer and transmits the response time to the inspection apparatus 200. The control unit 210 of the inspection apparatus 200 may inspect the response time based on the measurement result. The control unit 110 inspects the animation image G2 and the operation element image G3 by the same method. That is, the control unit 210 specifies the position in the display area of the UI unit 150 of the image to which the image ID is added based on the image ID included in the display screen, and inspects the display screen based on the specified position.

Then, as a result of the inspection, for example, if the control unit 210 determines that the position of the character image G1 is different from the expected display mode, a message ““ Copying in progress ”is not displayed in the center” is displayed on the display unit. The information is displayed on 240 and notified to the worker, and the fact is described in a predetermined log file and stored in the storage unit 130. On the other hand, when the control unit 210 determines that the display screen P2 matches the expected display mode, the control unit 210 displays a message such as “Normally displayed” on the display unit 240 or that the display screen P2 has no abnormality. To the log file.
The display mode of the inspection result is not limited to this. For example, an error code may be displayed and the error content may be displayed on the display unit 240 or written in a log file. Further, as an output mode of the inspection result, the control unit 210 may cause the printer or the like to print out, and any mode may be used.

  Subsequently, as shown in FIG. 3, since the display screen P1 includes the operation element images A1, A2, A3, etc., the control unit 110 executes the above processing steps S2 to S11 and starts the display screen P1. Are sequentially inspected based on the inspection program. Then, when the inspection of all display screens that can be displayed starting from the display screen P1 is completed, the control unit 110 ends the inspection of the display screen related to the “copy function”. Then, the inspection apparatus 200 performs an inspection of the display screen of the image forming apparatus 100 for another function such as a facsimile function or a scanner function.

In the embodiment described above, the image forming apparatus 100 displays each operator image and identification information for identifying each operation image and an image included in the display screen. On the other hand, when the inspection apparatus 200 acquires image information representing the display screen displayed by the image forming apparatus 100, detects the identification information from the image information, and acquires the identification information, the inspection apparatus 200 adds the identification information to the operator image. A corresponding display screen is displayed on the image forming apparatus 100. Then, the inspection apparatus 200 inspects the display mode of the display screen of the image forming apparatus 100 by specifying the position of each image by the image ID. With this configuration, the controller 210 identifies the position of the operator image based on the operator image ID even when the arrangement position of the operator image is changed, added or reduced, and the operator image at the specified position. By designating, the display screen to be inspected is displayed and inspected.
In the manufacturing stage and the pre-shipment stage, there is a case where the position of the operation element image is changed due to the design change, and a new operation element image is added or reduced. Conventionally, when the position information of the operator image is changed, the inspection program needs to be updated. For example, if the type or position of the operator image differs depending on the type or version of the device, the inspection program is increasingly inspected. The algorithm concerned was complicated. In addition, a huge amount of human work such as correction of a program accompanying a design change may be required. On the other hand, according to the inspection system 1, the inspection apparatus 200 instructs to display a display screen corresponding to the operation element image displayed by the image forming apparatus 100 based on the operation element image ID, and displays the display screen. The algorithm related to the above inspection is simplified.

(3) Modifications The above embodiment may be modified as follows. Specifically, the following modifications are mentioned, for example. These modifications can be combined with each other.
(3-1) Modification 1
In the above-described embodiment, the inspection apparatus 200 acquires bitmap-format image information representing a display screen from the image forming apparatus 100 by communication. On the other hand, the inspection apparatus includes an imaging unit that performs imaging, and the inspection apparatus 200 captures the display area of the UI unit 150 of the image forming apparatus 100 by the imaging unit and represents image information that represents a display screen generated by the imaging. The operator image ID may be acquired from The imaging means here is means capable of imaging such as a digital camera. In this case, the inspection apparatus 200 images the display area of the UI unit 150 by the imaging unit, and the control unit 210 detects and acquires the operator image ID from the image information representing the captured image, which is the same as the embodiment. Perform the inspection process according to the procedure. Also in this case, the control unit 210 transmits a designation instruction for displaying each display screen by the communication unit 230.

(3-2) Modification 2
In the above-described embodiment, the control unit 210 of the inspection apparatus 200 causes the image forming apparatus 100 to execute an operation when the operator image is specified by transmitting a designation instruction for designating the position represented by the position information. It was. In response to this, the control unit 210 transmits the acquired operator image ID itself to the image forming apparatus 100, and the image forming apparatus 100 specifies the position of the operator image to which the operator image ID is added. A child image may be designated. Further, for example, a display screen ID is stored so that the inspection apparatus 200 stores a table equivalent to the ID management table 131 as shown in FIG. 4 and displays a display screen corresponding to the detected operator image ID. The display screen itself to be displayed on the image forming apparatus 100 may be instructed by, for example, transmitting.

(3-3) Modification 3
In the above-described embodiment, the image forming apparatus 100 generates an operation signal when each operation element image is designated in accordance with an instruction from the inspection apparatus 200, executes the operation, and switches the display screen. . On the other hand, the inspection apparatus 200 may be configured to specify an operator image using mechanical means such as a robot. The identification information added to the operator image is not limited to the digital watermark, and may be, for example, a barcode, a two-dimensional code, or MIG (MIcro Gradation).
In addition, the inspection process executed by the inspection apparatus 200 is not limited to using the definition of the expected display mode as in the embodiment, but stores image information in a bitmap format that represents a display screen that represents the expected display mode. The degree of coincidence between the image represented by the image information and the display screen represented by the image information acquired from the image forming apparatus 100 is obtained using pattern matching, and the inspection apparatus 200 performs inspection based on the degree of coincidence. May be performed. That is, the inspection processing algorithm is not limited to that described in the embodiment.

(3-4) Modification 4
In the above-described embodiment, the display screen ID is described in association with the operator image ID in the ID management table 131. However, instead of the display screen ID, the processing conditions included in the image forming apparatus are associated. Also good. Since the control unit 110 displays a display screen corresponding to each processing condition based on the display control program, the display screen corresponding to the operator image may be specified by the processing condition.
Further, the display screen mode and the expected display mode in the above-described embodiment are merely examples, and the contents thereof are determined by a designer or the like, and the functions provided in the display device that is the image forming apparatus 100 are those of the embodiment. It is not limited to. The inspection apparatus 200 displays a display screen to be inspected starting from a predetermined display screen for each function provided in the image forming apparatus 100. However, the inspection apparatus 200 operates based on the operator image ID. As long as a child image is designated and a display screen corresponding to the child image is displayed on the image forming apparatus 100, any algorithm related to the display order or the like may be used.
Further, although the image forming apparatus 100 has transmitted the image information representing the display screen in step S4, the image forming apparatus ID may be sequentially detected by the own apparatus and transmitted to the inspection apparatus 200. In this case, the inspection apparatus 200 acquires only the operator image ID transmitted by the image forming apparatus 100.
If the inspection apparatus 200 determines that there is a display screen that does not match the expected display mode in the inspection process, the inspection process may be terminated at that time, or may continue until the end. It is good also as a structure which makes a worker etc. designate whether it is set as operation | movement.
In the embodiment, the image forming apparatus 100 transmits image data in bitmap format to the inspection apparatus 200. However, the image forming apparatus 100 may be image data described in a page description language (PDL), for example. It suffices to realize a configuration in which the device 200 detects and acquires the operator image ID from the image data representing the display screen.

  The image ID may include information representing the expected display mode itself such as the shape and size of the operator image. For example, when the size and shape of an image are determined and the display position is an inspection target, for example, the inspection apparatus may determine the expected display mode of the size and shape based on the image ID. These conditions may not be defined in the management table 221. In the embodiment, the inspection apparatus 200 inspects the display screen displayed starting from the display screen P1, but the display screen P1 may be inspected. As long as the expected display mode is defined.

(3-5) Modification 5
Further, the image forming apparatus may have a function as the inspection apparatus 200. In this case, the image forming apparatus 100 may perform an inspection process on the self apparatus at a time after the image forming apparatus reaches the end user. That is, the image forming apparatus 100 has the function that the inspection apparatus 200 has in the embodiment. Perform inspection processing. In this case, the image forming apparatus is a manipulator image display control unit that causes the display unit and a manipulator image used for an operation to specify a display screen to be displayed on the display unit to be displayed on the display unit. An operator image display control means for displaying an operator image to which identification information for identifying a child image is added; an acquisition means for acquiring the identification information added to the operator image displayed on the display means; Of the operation element images controlled by the operation element image display control means and displayed on the display means, a display screen corresponding to the operation element image to which the identification information acquired by the acquisition means is added is displayed on the display means. A display device comprising: a display screen display control means to be displayed on the display; and an inspection means for inspecting a display mode of the display screen displayed on the display means by the display screen display control means Functions in.

(3-6) Modification 6
In the above-described embodiment, the display device to be inspected is the image forming apparatus 100. However, the inspection target device performed by the inspection system is not limited to this. For example, a personal computer, a mobile phone, a PDA (Personal Digital) Assistant) or the like, and any processing executed by the apparatus may be used. In other words, any operation element image displayed on the display unit may be designated and function as a display device that displays a display screen corresponding to the designated operation element image. Further, unlike the UI unit 150, it does not have a touch panel function, and a user operates a pointing device such as a cross key, a mouse, or a pen device provided separately, and designates an operator image by a pointer on the display screen. The display device may be configured as follows.

(3-7) Modification 7
In the above-described embodiment, part of the control executed by the control unit 110 and the control unit 210 may be performed in cooperation with other hardware, or performed by one or a plurality of other hardware. You may do it. In addition, when the control unit 110 or the control unit 210 executes a program to execute each function, the program includes a magnetic recording medium (magnetic tape, magnetic disk, etc.) and an optical recording medium (optical disk (CD, DVD), etc.). It can be provided in a state of being recorded on a computer-readable recording medium such as a magneto-optical recording medium or a semiconductor memory. It is also possible to download via a network such as the Internet.

It is a figure showing the composition of the inspection system concerning one embodiment of the present invention. FIG. 2 is a block diagram illustrating a configuration of an image forming apparatus according to the embodiment. It is the figure which showed an example of the display screen displayed on a UI part. It is the figure which showed an example of the display screen displayed on a UI part. It is the figure which showed an example of the ID management table which concerns on the same embodiment. It is a block diagram which shows the structure of the inspection apparatus which concerns on the same embodiment. It is the figure which showed an example of the expectation display mode management table which concerns on the same embodiment. It is the flowchart which showed the operation | movement procedure of the test | inspection process performed in the test | inspection system which concerns on the embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Inspection system, 100 ... Image forming apparatus, 110 ... Control part, 120 ... Image reading part, 130 ... Memory | storage part, 131 ... ID management table, 140 ... Communication part, 150 ... UI part, 160 ... Image forming part, 200 DESCRIPTION OF SYMBOLS ... Inspection apparatus 210 ... Control part 220 ... Memory | storage part 230 ... Communication part 240 ... Display part 250 ... Operation part.

Claims (7)

A display device and an inspection device;
The display device
Display means;
An operator image display control means for displaying an operator image used for an operation for designating a display screen to be displayed on the display means on the display means, wherein the operation is added with identification information for identifying each operator image. An operator image display control means for displaying a child image;
Display from said display screen displayed on the display means, corresponding to the designated operating image by the manipulator image display control means displays on the display means is controlled by been operating image sac Chi before Symbol inspection device and a display screen display switching means for switching the screen,
The inspection device includes:
Obtaining means for obtaining the identification information added to the operator image displayed on the display means of the display device;
Instruction means for instructing the display screen display switching means to display on the display means a display screen corresponding to the operation element image to which the identification information acquired by the acquisition means is added;
The display position of the operator image on the display screen switched by the display screen display switching means according to the instruction by the instruction means, and the display position included in the identification information of the operator image on the switched display screen And an inspection means for inspecting whether or not the two coincide with each other . The display screen display switching means of the display device causes the display means to display a display screen including an image to which identification information for identifying each is added,
The acquisition unit of the inspection apparatus acquires the identification information added to the image included in the display screen displayed on the display unit,
The inspection means specifies a position in a display area of the display means of an image to which the identification information acquired by the acquisition means is added, and inspects the display screen based on the specified position. Item 4. The inspection system according to Item 1. The display device
Transmission means for transmitting the identification information added to the operator image displayed on the display means to the inspection device;
The inspection device includes:
Receiving means for receiving the identification information transmitted by the transmitting means of the display device;
The inspection system according to claim 1, wherein the acquisition unit acquires the identification information received by the reception unit. The inspection device includes:
Comprising imaging means for imaging,
The acquisition means includes
The operation unit image displayed on the display unit of the display device is imaged by the imaging unit, and the identification information added to the operation unit image represented by the image information generated by the imaging is acquired. The inspection system according to claim 1, wherein A display unit and a control unit image display control unit for displaying a control unit image used for an operation for designating a display screen to be displayed on the display unit on the display unit, and identifying information for identifying each control unit image The operator image display control means for displaying the added operator image, and the operator image displayed on the display means controlled by the operator image display control means from the display screen displayed on the display means. of acquisition for acquiring the identification information added to the displayed operating image on the display unit of the display device and a display screen display switching means for switching the display screen corresponding to the operating element image itself inspecting apparatus instructs Means,
Instruction means for instructing the display screen display switching means to display on the display means a display screen corresponding to the operation element image to which the identification information acquired by the acquisition means is added;
The display position of the operator image on the display screen switched by the display screen display switching means according to the instruction by the instruction means, and the display position included in the identification information of the operator image on the switched display screen And an inspection means for inspecting whether or not the two coincide with each other . Display means;
An operator image display control means for displaying an operator image used for an operation for designating a display screen to be displayed on the display means on the display means, wherein the operation is added with identification information for identifying each operator image. An operator image display control means for displaying a child image;
Obtaining means for obtaining the identification information added to the operator image displayed on the display means;
Wherein the display screen displayed on the display unit, the identification information displayed operating image sac Chi before Symbol acquisition means has acquired the display means being controlled by said operating image display control means is added the Display screen display switching means for switching to a display screen corresponding to the operator image, display position of the operator image on the display screen switched by the display screen display switching means, and identification information of the operator image on the switched display screen A display device comprising: inspection means for inspecting whether or not a display position included in the screen matches . The computer of the inspection device ,
A display unit and a control unit image display control unit for displaying a control unit image used for an operation for designating a display screen to be displayed on the display unit on the display unit, and identifying information for identifying each control unit image The operator image display control means for displaying the added operator image, and the operator image displayed on the display means controlled by the operator image display control means from the display screen displayed on the display means. acquisition of which the inspection apparatus acquires the identification information added to the displayed operating image on the display unit of the display device and a display screen display switching means for switching the display screen corresponding to the operating image has been specified Means,
Instruction means for instructing the display screen display switching means to display on the display means a display screen corresponding to the operation element image to which the identification information acquired by the acquisition means is added;
The display position of the operator image on the display screen switched by the display screen display switching means according to the instruction by the instruction means, and the display position included in the identification information of the operator image on the switched display screen Is a program that functions as an inspection means for inspecting whether or not the two match .

Images