JP2008158643A - Display inspection method, display inspection apparatus and display inspection program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To inspect whether or not display character strings to be inspected are displayed properly without recognizing the display character strings converted into a character code. <P>SOLUTION: A capture image of a display character string displayed in an item display area 21 is acquired, and a correct comparative character string image is generated according to character codes in a resource file embedded in software (printer driver) to be inspected, correspondence information, attribute information and the like. The capture image is compared with the comparative character string image to inspect character loss and the like. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for inspecting a display character string displayed on a display screen of a display device.

Patent Document 1 discloses an appearance inspection apparatus that inspects whether or not a predetermined character string (for example, a model number) is accurately displayed on an electronic component such as a substrate. Specifically, the image reading device acquires an image on the electronic component and analyzes it to extract and recognize a character string displayed on the electronic component. The recognized character string is compared with an expected character string prepared in advance and expected as a recognition result.
JP 2002-183712 A

  However, in the appearance inspection apparatus of Patent Document 1 and the like, what uses so-called OCR (Optical Character Reader) software is generally used to recognize a character string displayed on an electronic component. This OCR software recognizes a character string displayed on an electronic component by converting a character string image extracted from an image captured by an image reading device into a character code by matching with a conversion pattern stored in advance. To do.

  Therefore, in the above-described appearance inspection apparatus, the conversion accuracy by the OCR software greatly affects the inspection accuracy. In addition, when plural kinds of languages can be used for the character string displayed on the electronic component, there is a problem that conversion pattern data corresponding to each language must be prepared in advance.

  The present invention has been completed based on the above-described circumstances, and the purpose thereof is to convert the display character string to be inspected into a character code and recognize the normal display character string. The present invention provides a display inspection method, a display inspection apparatus, and a display inspection program capable of inspecting whether or not the image is displayed.

  As means for achieving the above object, a display inspection method according to a first invention is a display inspection method for inspecting a display character string displayed on a display screen of a display device, wherein the display screen includes Obtaining image data of the displayed display character string; generating image data of a comparison character string based on a setting data character code prepared in advance; image data of the display character string; Comparing image data with image data of a comparison character string, and determining whether or not the display character string is normally displayed based on the result of the image comparison.

  According to a second aspect of the present invention, there is provided a display inspection apparatus comprising: obtaining means for obtaining image data of a display character string displayed on a display screen of the display device; and image data of a comparison character string based on a setting data character code prepared in advance. Generation means for generating the image, comparison means for comparing the image data of the display character string and the image data of the comparison character string, and whether the display character string is normally displayed based on the result of the image comparison Determining means for determining whether or not.

  A display inspection program of a third invention is a display inspection program for inspecting a display character string displayed on a display screen of a display device, and is an image of the display character string displayed on the display screen on a computer. An acquisition process for acquiring data; a generation process for generating image data of a comparison character string based on a setting data character code prepared in advance; image data of the display character string; and image data of the comparison character string; And a determination process for determining whether or not the display character string is normally displayed based on the result of the image comparison.

  A fourth invention is the third display inspection program, wherein the setting data character code is data for generating image data of the display character string.

  A fifth invention is the third display inspection program, wherein the setting data character code is test data prepared separately from data for generating image data of the display character string.

<First to third inventions>
According to the present invention, a display character string displayed on a display screen of a display device is acquired as image data (image data), while a correct answer is obtained based on a data character code (for example, character code) prepared in advance. The image data (image data) of the comparison character string is generated. Then, the display character string is compared with the comparison character string to determine whether or not the display character string is normally displayed. Therefore, for example, conversion by OCR software or the like becomes unnecessary, and it is not necessary to prepare pattern data for conversion for each language used.

<Fourth Invention>
According to the present invention, the setting data character code for generating the image data of the comparison character string is data for generating the image data of the display character string. At this time, the characters in the comparison character string and the display character string do not match each other. However, for example, a program for generating a display character string and a display layout program for displaying the display character string are inconsistent, so that a part of the display character string is not displayed in the display area in the display layout, and so When a character defect occurs, the character defect can be inspected with high accuracy.

<Fifth invention>
According to the present invention, it is possible to inspect not only the above-mentioned character missing but also the match / mismatch of each character.

<Embodiment 1>
(About display inspection program)
Embodiment 1 of the present invention will be described with reference to FIGS.
For example, in software executed by a general-purpose computer, a display image (for example, a user interface or a dialog box) including a predetermined display character string is displayed on a display screen of a display device connected to the computer. There are many things to do. Some of these display images are generated from layout data in which an item display area is designated and a character code (an example of “setting data”) related to a display character string to be displayed in the item display area. is there. Specifically, the layout data includes attribute information (position (font type such as italic / Mincho), character spacing, etc.), character size, screen resolution, etc. , Number of colors, etc.). Then, a layout image is generated based on the layout data, and a display character string image (image data) is generated for each item display area based on a character code and attribute information.

  When a display image is generated by such a method, for example, each character of the attribute information is different from a design value due to a defect in the development of the software, so that some characters are not displayed. Malfunctions can occur.

  The display inspection program of the present invention is for causing a development computer to execute a process for inspecting whether or not the above-mentioned character missing or the like has occurred in the development stage of the software.

(Development computer)
FIG. 1 shows a development computer 10. At the time of the display inspection, software under development (a printer driver in this embodiment) and the display inspection program are installed in the computer 10. The computer 10 in which the display inspection program is installed is an example of the “display inspection apparatus”.

  FIG. 2 is a block diagram showing the internal configuration of the computer 10. The computer 10 includes an operation unit 11 that receives an input operation from the outside (for example, an input unit such as a keyboard or a mouse), a display unit 12 (an example of a “display device”), a ROM 13, a RAM 14, and a CPU 15 (“computer”). 1), a hard disk 16, a USB interface 17, and a CDROM drive 18.

  In addition to various application software, the hard disk 16 stores a printer driver and a display inspection program that are being developed, and the CPU 15 performs application software, a display inspection program, and the like based on a start command from the operation unit 11. Is read from the hard disk 16 and activated. The display inspection program is recorded on, for example, a recording medium (CDROM in this embodiment), read by the CDROM drive 18 and stored in the hard disk 16.

(About data related to dialog boxes)
FIG. 3 shows a dialog box 20 as an example of a display image displayed on the display unit 12 by executing the printer driver. This dialog box 20 is displayed, for example, for allowing the user to make print settings when a print request is made to the printer. In this dialog box 20, display character strings such as “Paper Size” and “Multiple Page” are displayed in each item display area (static text control) 21. Each item display area 21 is assigned an individual identification number (control ID).

  FIG. 4 is a schematic diagram showing a data structure of correspondence information between each control ID and a character code. With this correspondence information, for each control ID, a resource ID, attribute information, and a character code related to a character string to be displayed are associated. The correspondence information is included in the printer driver as an inspection target, and is stored in the hard disk 16 after the printer driver is installed. Here, the resource ID and the character code are resource files that constitute a part of the printer driver as described above, and this is an example of “data for generating image data of a display character string”.

  The layout data constituting the dialog box 20 of FIG. 3 is also included in the printer driver, and is stored in the hard disk 16 after the printer driver is installed.

(Control contents executed by display inspection program)
For example, the printer driver is activated, and the display inspection program is activated in a state where the dialog box 20 is displayed on the display screen 12A of the display unit 12. Then, the CPU 15 executes the control shown in FIGS. First, in S1, environmental information (screen resolution, number of colors, etc.) is acquired from the operating system of the computer 10. Next, in S2, it is determined whether or not a character string that has not yet been examined remains among the character strings displayed in each item display area 21 on the dialog box 20. Specifically, as described above, each item display area 21 is assigned with each control ID, and the CPU 15 sequentially designates the item display area 21 as an inspection target by sequentially designating the control ID. Go. Then, when all control IDs have been specified, there is no character string to be inspected (S2: No), and the display inspection for the dialog box 20 ends.

  If there remains a character string to be checked in S2 (S2: Yes), one control ID that has not been specified in S3 is specified, and a resource ID corresponding to the specified control ID is determined based on the correspondence information. Determine (S4). In S5, the character code corresponding to the resource ID is acquired from the resource file, and the attribute information associated with the control ID is acquired in S6. In step S7, a comparison character string image is generated as image data from the character code, environment information, and attribute information. At this time, the CPU 15 executes “generation processing” and functions as an example of “generation means”.

  Subsequently, in S8, on the dialog box 20 displayed on the display screen 12A of the display unit 12, the display character string image (hereinafter, referred to as the item display area 21 corresponding to the control ID currently designated) is displayed. "Captured image") is acquired as image data. At this time, the CPU 15 executes “acquisition processing” and functions as an example of “acquisition means”. Here, since the comparison character string image is an image before being incorporated into the layout image, even if the item display area 21 is smaller than the design value, an image in which all character strings based on the character code are reflected. In other words, the correct image is obtained. On the other hand, since the capture image is an image after being incorporated into the layout image, if the item display area 21 is smaller than the design value, or more than the number of characters that can be displayed in the item display area 21. When a character string is set, an image in which a part of the character string based on the character code is missing can be obtained.

  Then, the process proceeds to the comparison / determination process of S9, and the control contents shown in FIG. 6 are executed. At this time, the CPU 15 executes “comparison processing / determination processing” and functions as an example of “comparison means, determination means”. In S21, both left ends of the captured image and the comparison character string image are set as comparison start points. In S22, it is determined whether or not the right end of the comparison character string image has been reached. If it has not reached the right end (S22: No), in S23, one line currently being processed (in the captured image and the comparison character string image, An image comparison between the captured image and the comparison character string image is performed for each direction (a line L having a width corresponding to a predetermined number of unit pixels) extending in the direction (direction orthogonal to the character string arrangement direction) (see FIG. 7).

  Then, in S24, it is determined whether or not the pixels of the captured image and the comparison character string image match for one line currently being processed. If they match (S24: Yes), the line to be processed in S25. Is moved from the currently processed line L to the line L one line to the right. In S26, it is determined whether the processing target line has reached the right end of the captured image. If the right end has not been reached (S26: No), the process returns to S22.

  On the other hand, when the right end is reached (S26: Yes), for the captured image, the line to be processed is moved down by one character and to the left end line L, and the image is displayed on this line L. It is determined whether or not there is (S28). If there is an image (S28: Yes), the process returns to S22. Note that “Yes” in S28 is when the captured image is an image in which the character string is broken in the middle as shown in FIG.

  For example, as shown in FIG. 7, when the captured image is a normal image displaying all the character string images to be displayed “is displayed correctly?”, The captured image and the comparison character string image are all lines. At L, the pixels match (the images match) (S24: Yes). As a result, when the line to be processed reaches the right end of the comparison character string image (S22: Yes), it is determined in S29 that there is no missing character.

  Next, as shown in FIG. 9, when the width of the item display area 21 on the layout image is narrower than the width of the character string image “Is it displayed correctly?” Thus, the pixels of the captured image and the character string image for comparison do not match (S24: No). As a result, it is determined in S30 that there are missing characters.

  Furthermore, as shown in FIG. 8, when the captured image is a line break in the middle of the character string, when the processing target line reaches the right end of the captured image (S26: Yes), the processing target line is changed. The image comparison is continued according to the left end of the lower row. If there is no character image in the second line one level below (S28: No), it is determined that there is no character missing because the captured image does not match the comparison character string image at that time (S31). On the other hand, since there is a character image on the second line in the image of FIG. 8 (S28: Yes), the process returns to S22, but “??” on the second line is missing a character. Accordingly, it is determined that there is a missing character (S30), assuming that the pixels do not match in S24 (S24: No).

  If the character image on the second line is not missing, “Yes” is determined in S22, and it is determined that no character is missing (S29). That is, in the present embodiment, it is determined that there is no missing character if all the character strings to be displayed are displayed in the item display area 21 regardless of the presence or absence of a line feed. It should be noted that the configuration may be such that when a line break is detected (S28: Yes), it is determined that there is a missing character, or the configuration is such that the presence or absence of a line break is inspected even if it is not determined that the character is missing. Note that “OK” and “NG” are recorded in the inspection result corresponding to each item display area (control ID), and for example, recorded in the hard disk 16 in the SVC format.

(Effect of this embodiment)
According to the present embodiment, the captured image of the display character string displayed in the item display area 21 is acquired, while the character code in the resource file incorporated in the software to be inspected (printer driver) and the correspondence relationship A comparison character string image that is a correct answer is generated based on the information and the attribute information. Then, the captured image and the comparison character string image are compared with each other to inspect for the presence or absence of missing characters. Therefore, it is not necessary to convert the captured image into character data using character conversion software as in the prior art, and inspection can be performed if only the character code used in the software to be inspected is prepared.

<Embodiment 2>
10 and 11 show the second embodiment. In the first embodiment, the configuration uses the character code in the resource file that is finally incorporated in the software to be inspected (printer driver) as the setting data. However, in the second embodiment, the resource file is In this configuration, a test character code prepared separately is used as setting data. The other points are the same as in the first embodiment. Therefore, the same reference numerals as those in the first embodiment are given and the redundant description is omitted, and only different points will be described next.

  The setting data character code of the present embodiment is an example of “test data”. Specifically, the character code in a design database (for example, a database of translation materials) used when developing software to be inspected. It is. This is created, for example, in the software design stage and has extremely high reliability.

  Here, as shown in FIG. 10, the design database includes character string data groups in a plurality of languages having the same meaning with respect to the contents to be displayed in each item display area 21 of the dialog box 20. Character string data groups in a plurality of languages are associated with a common control ID and resource ID.

  FIG. 11 is a flowchart showing the generation / acquisition processing in the second embodiment. The same processes as those in the flowchart of FIG. 5 of the first embodiment are denoted by the same reference numerals, and description thereof is omitted. First, in S1 ′, in addition to environment information (screen resolution, number of colors, etc.) from the operating system of the computer 10, the language used by the operating system (hereinafter referred to as “OS language”) is acquired. In S5 ′, a character code associated with the resource ID and in the same language as the OS language is acquired. Then, a comparison character string image is generated based on the character code and attribute information of the same language as the OS language.

  With such a configuration, in addition to missing characters, it is possible to check for errors in the incorporation of resource files by collating with the design database (checking each character of the display character string). Moreover, since character strings in the same language as the OS language of the computer 10 are automatically taken from the design database and a comparison character string image is generated, even a person who is not familiar with multiple languages can inspect. it can.

<Embodiment 3>
12 to 14 show the third embodiment. In the first and second embodiments, the display image (dialog box 20) to be inspected is assigned to each item display area 21 on the program, and assigned to each item based on each control ID. The image in the item display area 21 could be acquired as a captured image. However, among the display images to be inspected, there is no such thing as the control ID on the program as in the dialog box 30 shown in FIG. 12, and the position, size, and range of each item display area 31 are not present. And information such as the display format (hereinafter referred to as “item display area attribute information”) cannot be obtained from the program. One example is a display screen when the software (printer driver) to be inspected is created by Macromedia Flash8 (trademark), for example.

  The third embodiment relates to a display inspection program that can inspect for missing characters on a display screen that does not have a control ID or the like. First, even if the attribute information of each item display area 31 cannot be acquired on the program, the developer should have recognized it at the development stage of the software. There should have been information.

Therefore, in the present embodiment, a database is prepared in which item IDs instead of the control IDs are assigned to the attribute information of each item display area 31 based on the development material. Further, as shown in FIG. 13, correspondence information between each item ID and a character code is prepared. With this correspondence information, for each item ID, a character string ID, attribute information, and a character code related to a character string to be displayed are associated. FIG. 14 is a flowchart showing the generation / acquisition process. Basically, the process shown in FIGS. 5 and 11 of the first and second embodiments is different in that the item ID is used instead of the control ID, and the other points are almost the same.
With such a configuration, it is possible to check for missing characters on a display screen that does not have a control ID or the like.

<Other embodiments>
The present invention is not limited to the embodiments described with reference to the above description and drawings. For example, the following embodiments are also included in the technical scope of the present invention.
(1) In the above-described embodiment, the general-purpose personal computer 10 in which the display inspection program is installed has been described as an example of the display inspection apparatus. A dedicated device may be used.

  (2) In the above embodiment, the printer driver is exemplified as the software to be inspected. However, the present invention is not limited to this, and any software that displays a display screen in which a character string is described on the display screen of the display device may be used. Can be an inspection target. For example, a form printing software that prints various forms such as insurance design documents has a function to display a form image in which a character string is displayed in each item display area on the display screen of the display device. obtain.

  (3) In the above-described embodiment, whether the start position of the left end of the captured image and the comparison character string image is the same may be the inspection target. Furthermore, whether or not the color of the characters matches may be an inspection target.

  (4) In the above embodiment, for example, based on the size information of each item display area, a configuration in which only a portion where occurrence of character loss is predicted in advance, such as the right end and the lower end of each item display area, is compared or predicted The image comparison may be performed preferentially over other parts.

The perspective view of the computer for development concerning Embodiment 1 of the present invention. Block diagram showing the internal configuration of the development computer Schematic diagram showing the dialog box Schematic diagram showing the data structure of correspondence information between each control ID and character code Flow chart showing generation / acquisition processing Flow chart showing comparison / judgment processing Schematic diagram for explaining image comparison of captured image and comparison character string image (normal) Schematic diagram for explaining image comparison of captured image and comparison character string image (when there is a line break and character missing) Schematic diagram for explaining the image comparison between the captured image and the comparison character string image (with missing characters) Schematic diagram showing the data structure of the design database of the second embodiment Flow chart showing generation / acquisition processing Schematic diagram showing the dialog box of the third embodiment Schematic diagram showing the data structure of correspondence information between each item ID and character code Flow chart showing generation / acquisition processing

Explanation of symbols

10. Computer (display inspection device)
12. Display unit (display device)
12A ... display screen 15 ... CPU (computer, acquisition means, generation means, comparison means, determination means)

Claims (5)

A display inspection method for inspecting a display character string displayed on a display screen of a display device,
Obtaining image data of the display character string displayed on the display screen;
Generating image data of a comparison character string based on a setting data character code prepared in advance;
Comparing the image data of the display character string and the image data of the comparison character string;
A display inspection method including a step of determining whether or not the display character string is normally displayed based on a result of the image comparison. Acquisition means for acquiring image data of a display character string displayed on the display screen of the display device;
Generating means for generating image data of a comparison character string based on a setting data character code prepared in advance;
Comparison means for comparing the image data of the display character string and the image data of the comparison character string;
And a determination unit that determines whether or not the display character string is normally displayed based on a result of the image comparison. A display inspection program for inspecting a display character string displayed on a display screen of a display device,
On the computer,
An acquisition process for acquiring image data of the display character string displayed on the display screen;
Generation processing for generating image data of a comparison character string based on a setting data character code prepared in advance;
A comparison process for comparing the image data of the display character string and the image data of the comparison character string;
A display inspection program for executing determination processing for determining whether or not the display character string is normally displayed based on the result of the image comparison. A display inspection program according to claim 3,
The setting data character code is data for generating image data of the display character string. A display inspection program according to claim 3,
The setting data character code is test data prepared separately from data for generating image data of the display character string.

Images