JP6010318B2 - Image inspection apparatus, image inspection method, and program - Google Patents

Description

  The present invention relates to an image inspection apparatus for inspecting a difference between two images.

  As a conventional image inspection apparatus, an image pickup unit that generates a reference image by capturing a printed material that is a reference for calculating a color tone variation amount, and that generates a comparison image by capturing a printed material that is a comparison target of colorimetric values; Integration means for generating a reference color value and a comparison color value by executing integration in a predetermined area corresponding to each of the image and the comparison image, and a color variation amount based on the reference color value and the comparison color value There has been known one having a comparison means for calculating (see, for example, Patent Document 1).

JP 2003-266646 A (first page, FIG. 1 etc.)

  However, in the conventional image inspection apparatus, for example, when the images to be compared are overlapped, the difference between the images is detected by comparing the overlapping portions. There was a problem that it was not possible.

  For example, if the layout of elements such as characters in the images to be compared has changed, even if the content described in the entire image is substantially the same, the overlapping portions will be different images, so There was a problem that a difference was detected in many areas. For example, as a result of processing an image of a document for creating a printed material due to various circumstances and restrictions during printing, characters between the document and the print data actually used for printing are displayed. May be changed. In such a case, even if the contents are substantially the same between the document and the print data, there has been a problem that the character portion whose arrangement has been changed is detected as a difference portion. It was.

  The image inspection apparatus of the present invention compares the first image and the second image with the first image storage unit that stores the first image, the second image storage unit that stores the second image, A detection unit that detects the location of the character, and an output unit that outputs information indicating the location of the difference detected by the detection unit. The detection unit acquires a character image from each of the first image and the second image. The image acquisition means compares one or more first character images, which are character images acquired from the first image, with one or more second character images, which are character images acquired from the second image. It is an image inspection apparatus provided with the character image comparison means which detects a location.

  With this configuration, it is possible to appropriately detect a difference in content between images.

  In the image inspection apparatus according to the present invention, in the image inspection apparatus, the detection unit is included in one or more first character data that is character data corresponding to an image of a character included in the first image, and the second image. Character data acquisition means for acquiring one or more second character data that is character data corresponding to a character image, the first character data acquired by the character data acquisition means, and the second character data An image inspection apparatus further comprising character data comparison means for detecting a location.

  With this configuration, it is possible to detect a difference in content between images by different processes, and to appropriately detect a difference in content between images from different viewpoints.

  In the image inspection apparatus of the present invention, in the image inspection apparatus, the detection unit further includes a replacement detection unit that detects a place where the character of the second character image is replaced with respect to the first character image, and the output unit includes This is an image inspection apparatus that outputs information indicating a place where the character detected by the replacement detection means is replaced.

  With this configuration, it is possible to detect a place where characters are replaced.

  In the image inspection apparatus of the present invention, in the image inspection apparatus, the detection unit further includes an additional detection unit that detects a location where one or more characters of the second character image are added to the first character image. The output unit is an image inspection apparatus that outputs information indicating a place where the character detected by the addition detection unit is added.

  With this configuration, it is possible to detect a place where a character is added.

  In the image inspection apparatus of the present invention, in the image inspection apparatus, the detection unit further includes a deletion detection unit that detects a position where one or more characters of the second character image are deleted from the first character image. The output unit is an image inspection apparatus that outputs information indicating a location where the character detected by the deletion detection unit is deleted.

  With this configuration, it is possible to detect a location where a character is deleted.

  In the image inspection apparatus of the present invention, in the image inspection apparatus, the detection unit further includes a morpheme analysis unit that performs morpheme analysis on each of the first character data and the second character data acquired by the character data acquisition unit. The comparison means is an image inspection device that compares morphemes or words respectively obtained for the first character data and the second character data by morphological analysis and detects a difference portion.

  With this configuration, it is possible to detect differences in characters in the image in units of morpheme or words, so it is possible to grasp what kind of difference has occurred compared to the case where differences in characters are detected in units of characters. Cheap. Also, unlike the case where a difference is detected in units of one character, it is easy to estimate the location where the difference has occurred, so the location of the difference can be shown without overlapping the original image and the location where the difference was detected. .

  In the image inspection apparatus of the present invention, in the image inspection apparatus, the detection unit performs layout recognition for each of the first image and the second image, and detects one or more areas, respectively. The layout recognition means further comprises correspondence determination means for determining a correspondence relationship between one or more areas recognized for the first image and one or more areas recognized for the second image, and the character image acquisition means includes correspondence determination means. The first character image and the second character image are obtained for each set of regions determined to correspond to each other, and the character image comparison means compares the first character image and the second character image for each region, It is an image inspection apparatus which detects the location.

  With such a configuration, even if the area where the characters are arranged is greatly changed, the characters can be compared between the corresponding areas, so that the characters can be compared correctly.

  In the image inspection apparatus of the present invention, in the image inspection apparatus, the character image comparison unit compares at least a difference in luminance value between pixels constituting each of the first image and the second image with a threshold value. The detection unit is an image inspection apparatus that includes a threshold setting unit that sets a threshold according to the appearance of a difference in luminance value of pixels that form the first image and the second image, respectively. is there.

  With this configuration, pixel matching can be determined using a threshold value set in accordance with the character image to be compared, so that character images can be compared with high accuracy.

  In the image inspection apparatus of the present invention, the image inspection apparatus further includes a threshold reception unit that receives an input of a threshold, and the character image comparison unit includes brightness of pixels that respectively constitute at least the first image and the second image. It is an image inspection apparatus that detects a difference portion by comparing a difference between values with a threshold received by a threshold receiving unit.

  With such a configuration, the user can appropriately set the degree of detecting the difference in character images.

  In the image inspection apparatus of the present invention, in the image inspection apparatus, the character image comparison unit compares at least a difference in luminance values of pixels constituting the first character image and the second character image with a threshold value, A part for detecting a location, an evaluation result receiving unit that receives a user's evaluation result for one or more points of difference output by the output unit, and a portion of a difference corresponding to each evaluation result received by the evaluation result receiving unit Learning a combination of a luminance value acquisition unit that acquires information about luminance values of the first character image and the second character image, a combination of an evaluation result, information on a luminance value of a portion corresponding to the evaluation result, and a threshold value The image inspection apparatus further includes a first character image to be compared and a threshold value acquisition unit that acquires a threshold value corresponding to the second character image.

  With this configuration, it is possible to compare appropriate character images using a threshold value set based on past determination results.

  According to the image inspection apparatus or the like according to the present invention, it is possible to appropriately detect a difference in content between images.

Block diagram of an image inspection apparatus according to an embodiment of the present invention Flow chart explaining the operation Flow chart explaining the operation Flow chart explaining the operation Flow chart explaining the operation The figure which shows an example of the first image The figure which shows an example of the extraction character data management information The figure which shows an example of the second image The figure which shows the detection process of the same color area The figure which shows the detection process of the recognition area The figure which shows the first area management information The figure which shows the detection process of the recognition area The figure which shows the second area management information Figure showing the second character image The figure which shows the second character image management information The figure which shows the second character data management information Figure showing the comparison result information The figure which shows the same area correspondence management information The figure which shows an example of the same morpheme comparison result information The figure which shows an example of the same comparison result information The figure which shows an example of the same comparison result information Figure showing the display example Figure showing the display example Figure showing the display example Figure showing the display example Figure showing the display example The schematic diagram which shows an example of the external appearance of the computer which implement | achieves the image inspection apparatus by this Embodiment Diagram showing the internal configuration of the computer

  Hereinafter, embodiments of an image inspection apparatus and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

(Embodiment)
FIG. 1 is a block diagram of an image inspection apparatus 1 in the present embodiment.

  The image inspection apparatus 1 includes a first image storage unit 11, a second image storage unit 12, a detection unit 13, a threshold reception unit 14, an evaluation result reception unit 15, a luminance value acquisition unit 16, a threshold acquisition unit 17, and an output unit 18. Prepare.

  The detection unit 13 includes a layout recognition unit 131, a character image acquisition unit 132, a character data acquisition unit 133, a character data comparison unit 134, a morpheme analysis unit 135, a correspondence determination unit 136, a character image comparison unit 137, a threshold storage unit 138, a threshold value Setting means 139, replacement detection means 140, addition detection means 141, and deletion detection means 142 are provided.

  The first image storage unit 11 stores a first image. The first image is an image serving as a reference for comparison. The image described in the present embodiment is, for example, a still image such as color, gray scale, and black and white. The image is, for example, a character image, a mascot character, an icon, a pattern, an illustration, or the like. The image of the present embodiment is an image including an image of one or more characters, for example. The image of the present embodiment may be bitmap data or vector data. The image may have one or more character data (text data). The character data is information for outputting a character image, for example, and is, for example, a character code. The file format of the first image does not matter. The first image is, for example, a document for printing. For example, image data used for actual printing is created based on the first image. The first image is, for example, a manuscript from which print data is printed on the surface of a container or the like. When the first image has character data, the first image is obtained by extracting one or more character data extracted from each area where the character data in the first image and character data are extracted. Information for designating each area may be further included. Each area where character data is arranged is, for example, an area called a text block or a text frame. The storage here may be temporary storage. For example, a first image received by an input receiving unit or the like (not shown) may be temporarily stored in the first image storage unit 11. The first image storage unit 11 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

  The second image storage unit 12 stores a second image. The second image is the same image as the first image. The second image is an image that is compared with the first image. The second image is an image created based on the first image, for example. For example, it is an image of print data used for actual printing created using a first image as a document. The second image is print data printed on the surface of a container or the like, for example. The file format and size of the first image and the second image may be the same or different. The storage here may be temporary storage. For example, a second image received by an input receiving unit or the like (not shown) may be temporarily stored in the second image storage unit 12. The second image storage unit 12 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

  The detection unit 13 compares the first image and the second image to detect a difference portion. The difference portion is a portion where images do not match. In the present embodiment, the detection unit 13 includes one or more first character images that are character images acquired from the first image, and one or more second character images that are character images acquired from the second image. An example in the case of including processing for detecting a difference portion will be described.

  The layout recognizing unit 131 recognizes layouts for the first image and the second image, respectively, and detects one or more regions. Recognizing a layout means, for example, recognizing one or more areas (text blocks, text frames, etc.) in which characters are arranged, an area including text, and the like in an image. For example, the layout recognizing unit 131 recognizes areas where characters are arranged for the first image and the second image, respectively. Recognizing an area means, for example, acquiring information defining the area. Obtaining information defining a region means obtaining information that can define the contour of the region, for example. Acquiring information that can define a contour means, for example, a combination of region position information (for example, coordinates) and size information (for example, vertical and horizontal length information), or coordinates of vertexes of the contour ( A case where the contour is a polygon) and a coordinate group indicating the contour of the region. Further, obtaining information defining a region may mean obtaining coordinate information of all pixels in the region. Hereinafter, an example of processing in which the layout recognition unit 131 recognizes a layout will be described. Here, each area that the layout recognition unit 131 finally recognizes from the first image and the second image is referred to as a recognition area.

  FIG. 2 is a flowchart for explaining an example of layout recognition processing performed by the layout recognition unit 131 for the first image and the second image. Here, as an example, the case of performing region detection for the first image will be described, but the same processing may be performed for the second image.

  (Step S <b> 101) The layout recognition unit 131 performs processing for recognizing the same background color region for the first image. Details of this processing will be described later.

  (Step S102) The layout recognition unit 131 recognizes a region surrounded by a frame for each of the regions recognized in Step S101. Details of this processing will be described later.

  (Step S103) The layout recognizing means 131 detects a registered marker for each area recognized in Step S101 and Step S102. A registered marker is a marker (a mark, a sign, etc.) registered in advance, and specifically an image. And the area | region where the detected marker is arrange | positioned is excluded from each area | region. Details of this processing will be described later.

  (Step S104) The layout recognizing unit 131 acquires information defining the remaining area in the first image. As a result, the area surrounded by the frame in the area of the same background color (however, the area from which the area matching the registered marker is removed) and the remaining area in the first image are recognized. The area recognized in this way is the recognition area. For the remaining area, a minimum rectangular area including pixels having different colors and brightness from other areas may be recognized. Since the process of recognizing the minimum rectangular area is a known technique, detailed description thereof is omitted here. Then, the process ends.

  FIG. 3 is a flowchart for explaining the details of the process of recognizing the same background color area from the first image by the layout recognizing unit 131 described in step S101.

  (Step S <b> 201) The layout recognition unit 131 performs region labeling processing using the color values (for example, CMYK values, RGB values, etc.) of each pixel constituting the first image. The area labeling process is a process for detecting an area by connecting adjacent pixels of the same color. For example, a pixel having a color value difference (for example, a hue difference) equal to or less than a predetermined threshold value may be determined as a pixel having the same color. One or more areas of the same color are detected from the first image by the area labeling process. A specific color may be excluded from the labeling target.

  (Step S202) The layout recognizing unit 131 detects a region having a certain area or more as a candidate region among the labeled regions.

  (Step S203) The layout recognizing unit 131 selects the candidate area having the smallest area among the currently detected candidate areas (remaining without being deleted). The candidate area currently detected is a candidate area that remains without being deleted.

  (Step S204) The layout recognition unit 131 fills the inside of the candidate area selected in Step S203 (for example, including the area surrounded by the candidate area) with the same color.

  (Step S205) The layout recognizing unit 131 acquires information defining an area filled with the same color as information defining an area of the same background color, and accumulates it in a storage medium (not shown). Then, this area is excluded from the candidate areas. In addition, each area | region of the same background color recognized here is called a color area | region here.

  (Step S206) The layout recognition unit 131 determines whether there is another candidate area. If there is, the process returns to step S203, and if not, the process ends.

  In the above processing, when the image to be recognized for layout is vector data, each pixel when vector data is rendered at a predetermined resolution as a pixel to be subjected to labeling processing, etc., or a preview of vector data Each pixel of the image may be used. The same applies to other processes described below. Or you may recognize the area | region where the adjacent vector figure which has the same color is arrange | positioned as an area | region of the same background color.

  FIG. 4 is a flowchart for explaining a process in which the layout recognition unit 131 recognizes a region surrounded by a frame in each of the color regions recognized in step S101 described in step S102.

  (Step S301) The layout recognizing means 131 selects one color area recognized in step S101.

  (Step S302) The layout recognition unit 131 extracts pixel edges by comparing pixel values (for example, luminance values) between adjacent pixels for each pixel in the color region selected in Step S301. For example, if the difference between pixel values is equal to or greater than a predetermined threshold value, an edge is detected. Note that the threshold value for edge detection difference or the like may be specified in advance or may be set dynamically.

  (Step S303) The layout recognizing means 131 determines whether or not a closed curve surrounded by an edge exists in one color area from the edge extracted in Step S302. When it exists, it progresses to step S304, and when it does not exist, it progresses to step S308.

  (Step S304) The layout recognizing unit 131 selects one closed curve and determines whether or not the length of the circumference of the closed curve is equal to or longer than a predetermined length. If it is longer than the length specified in advance, the process proceeds to step S305, and if it is less than the length, the process proceeds to step S307.

  (Step S305) The layout recognition unit 131 determines that the inside of the one closed curve selected in Step S304 is an area surrounded by one frame, acquires information defining this area, and identifies this area Are stored in a storage medium (not shown) in association with the area identification information.

  (Step S306) The layout recognition unit 131 excludes the area defined in Step S305 from the inside of the one color area selected in Step S301.

  (Step S307) The layout recognition unit 131 determines whether there is another closed curve in the color region. If there is, the process returns to step S304, and if not, the process proceeds to step S308.

  (Step S308) The layout recognizing unit 131 determines whether another color area detected in step S101 exists. If it exists, the process returns to step S302, and if it does not exist, the process returns to the upper process.

  Through the above processing, an area surrounded by a frame in the color area can be recognized.

  In the above processing, when the image to be recognized for layout is vector data, an area surrounded by straight lines in the color area may be recognized as an area surrounded by a frame.

  Next, a process for detecting a registered marker for each area recognized in step S102 described in step S103 will be briefly described. The layout recognition unit 131 reads an image of a registered marker registered in advance from a storage medium (not shown). And the area | region which corresponds with a registration marker is each detected in the area | region recognized by step S101 and the area | region recognized by step S102. Then, the area where the detected marker is arranged is excluded from each area described above, information defining the area excluding the marker area is acquired, and stored in a storage medium (not shown) or the like. For example, the detection of the area that matches the registered marker is performed by changing the position (coordinates) and size (scale) of the image of the registered marker as parameters, and the image in the area recognized in step S101 of the first image, Matching with the image in the region recognized in step S102 is performed, and when an evaluation value equal to or higher than a predesignated value is obtained, it is determined that a registered marker has been found. Note that the processing for performing matching between images is a known technique, and thus detailed description thereof is omitted here.

  Note that the layout recognition unit 131 may perform all of the above processing, or may perform only a part thereof. For example, only the area surrounded by the frame may be detected without performing the process of recognizing the color area. Further, the layout recognition unit 131 may perform other layout recognition processing.

  The character image acquisition unit 132 acquires character images from the first image and the second image, respectively. The image of one character acquired by the character image acquisition unit 132 is preferably a character image in units of one character. Here, the character image acquired from the first image is referred to as a first character image, and the character image acquired from the second image is referred to as a second character image. The character image acquisition unit 132 acquires, for example, each character image included in each recognition area recognized by the layout recognition unit 131. In addition, the character image acquisition unit 132 may perform a process of detecting each character image in the first image and the second image in order to acquire a character image, and further define an image of each detected character. Information (for example, information defining the contour) may be acquired. Note that the process of detecting and acquiring a character image in units of one character from one region in the image is a known technique as a so-called OCR (Optical Character Recognition) technique, and thus detailed description thereof is omitted here. . If the first image or the second image is an image including character data, outline data of each character, bitmap data of each character, bitmap data obtained by rasterizing each character, and the like are acquired. You may do it. Moreover, the character image acquisition means 132 should just acquire the image of the character contained in a 1st image and a 2nd image as a result, and the acquisition method, means, etc. are not ask | required. For example, the character data included in the first image may be once extracted from the first image, and then the first character image may be acquired by rasterizing the extracted character data for each character. You may think that the character image of the image was acquired.

  Further, when the correspondence determination unit 136 described later detects a correspondence relationship between the recognition areas recognized above in the first image and the second image, the character image acquisition unit 132 determines that the first image and the second image The character image for each character may be acquired for each set of recognition regions that are determined to be corresponding recognition regions by the later-described correspondence determination unit 136 among the recognition regions that are recognized in FIG. . In this case, the character arrangement order, the acquisition direction, and the like when acquiring the character image are usually the same arrangement order and acquisition direction in the first image and the second image.

  The character data acquisition unit 133 includes one or more first character data that is character data corresponding to a character image included in the first image and one or more character data corresponding to a character image included in the second image. Get second character data. The character data is, for example, a character code. Alternatively, the character data may be a hash or the like corresponding to the character code. Specifically, the character data acquisition unit 133 acquires character data corresponding to one or more first character images acquired by the character image acquisition unit 132 for each recognition area included in the first image. Further, the character data acquisition unit 133 acquires character data corresponding to one or more second character images acquired by the character image acquisition unit 132 for each recognition area included in the second image. Thereby, the first character data and the second character data corresponding to the first character image and the second character image are acquired. Information indicating the correspondence between the character image and the character data (for example, information in which the character image identification information is associated with the character code) is stored in, for example, a storage medium (not shown). To do. Note that the process of acquiring character data corresponding to a character image such as a first character image or a second character image is a known technique as a technique such as OCR, and a detailed description thereof will be omitted here.

  The character data comparison unit 134 compares the first character data acquired by the character data acquisition unit 133 with the second character data and detects a difference portion. Here, the character data comparison unit 134 compares the first character data and the second character data between the recognition regions recognized by the layout recognition unit 131 from the first image and the second image, and detects a difference portion. To do. The character data comparing means 134 converts the first character data and the second character data for each character in the same arrangement order from one recognition area of the first image and one recognition area of the second image. By acquiring and comparing, it is sequentially determined whether or not the character data match. For example, character data is read character by character from left to right from the upper line to the lower line from the character data acquired for each recognition area, and the read character data (for example, character code or character By comparing (hash), the character match / mismatch is judged. If the compared characters do not match, the search is performed until a matching character appears, so as to detect, for example, a part where there is a difference in character data caused by line breaks, character addition, deletion, etc. Also good. A place where a mismatched character is detected is one of the places having a difference. In addition, when the compared characters do not match, if the character used for comparing the first character data matches the character next to the character used for comparing the second character data, A portion immediately before the unmatched portion of the image is a portion where a difference is detected because a character is missing. The process of comparing two character strings composed of character data and detecting a difference portion is a well-known technique as a technique such as document proofing or document comparison, so a detailed description will be given here. Omitted. For example, the character data comparing unit 134 accumulates information indicating a portion determined not to match by comparison, that is, a portion determined to have a difference, in a storage medium (not illustrated). Information indicating the location determined to have a difference is, for example, identification information of a character image corresponding to character data determined to have a difference, information indicating a position of a character image determined to have a difference, or the like. is there. Similarly, information indicating a location determined to match, that is, a location determined to have no difference, may be similarly stored in a storage medium (not shown) or the like. The same applies to the information on the part determined not to be different.

  The character data comparison unit 134 performs comparison between corresponding recognition areas. Between the corresponding recognition areas is an area considered to have the same content (same information). However, if the corresponding recognition area is not known, the character data comparison unit 134, for example, also serves to detect the corresponding recognition area, and in the second image, the one or more recognition areas recognized in the first image. For the combination with one or more recognized recognition areas, the first character data and the second character data between the recognition areas are compared to detect a difference. The process for detecting the difference portion here may be considered as a process for detecting a matching portion. The character data comparison unit 134 may compare any combination, but a combination that can determine the correspondence between one recognition area of the first image and one recognition area of the second image. It is preferable to make a comparison. Character data comparison is performed for combinations of recognition areas in this way because the area where the characters are arranged may be switched between the first image and the second image. This is because the correspondence determination unit 136 (to be described later) can determine which recognition area of the second image corresponds to the area.

  For example, the character data comparison unit 134 compares the first character data and the second character data for combinations of all the recognition areas of the first image recognized by the layout recognition unit 131 and all the recognition areas of the second image. I do.

  Further, for example, the character data comparison unit 134 performs the same process by considering that the recognition area detected by the layout recognition unit 131 is the same as the recognition area in which the same content information is described. You may make it compare the character data only between the recognized recognition areas.

  In addition, when the comparison between character data of one recognition area of the first image and one recognition area of the second image is performed, the matching rate of the character data of both is equal to or higher than a predetermined ratio In addition, for each recognition area constituting the set of recognition areas, comparison between character data with other recognition areas may not be performed.

  In addition, in the first image and the second image, when there is no significant change in the recognition area where the characters are arranged, the recognition area in the first image and the recognition area in the second image The character data may be compared only for the recognition regions that overlap each other in the image.

  In addition, the character data comparison unit 134 may compare the morpheme or the word obtained for the first character data and the second character data by the morphological analysis, and detect the difference portion. For example, the character data comparison unit 134 is acquired by a morpheme analysis unit 135 (to be described later) performing morphological analysis on the character data acquired by the character data acquisition unit 133 for each recognition area of the first image and the second image. The morphemes are compared to determine whether the morphemes match. By performing the comparison for each morpheme, it is possible to detect a difference in characters in morpheme units in the second image. For example, the character data comparison unit 134 may perform comparison by morpheme or word as described above instead of or in addition to the character data comparison as described above. . The character data comparison unit 134 accumulates, for example, information indicating a portion determined to have a different morpheme or word, that is, information indicating a portion determined to have a difference in a storage medium (not illustrated). The information indicating the portion determined to have a difference is, for example, identification information of a character image corresponding to one or more character data determined to have a difference, information indicating the position of the character image, or the like.

  The morpheme analysis unit 135 performs morpheme analysis on the first character data and the second character data acquired by the character data acquisition unit 133, respectively. Since morphological analysis is a known technique, a detailed description thereof is omitted here. As a morphological analysis system, for example, ““ Mecab ”(searched on February 28, 2012), Internet (URL: http://mecab.googlecode.com/svn/trunk/mecab/ doc / index.html), "" ChaSen ", [Search February 28, 2012], Internet (URL: http://chasen-legacy.sourceforge.jp/) It is. The processing for acquiring words from a character string is a known technique as a technique for a Japanese input system used in a computer or the like.

  The morphological analysis unit 135 can be usually realized by an MPU, a memory, or the like. The processing procedure of the morphological analysis unit 135 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The correspondence determining unit 136 determines a correspondence relationship between one or more recognition regions recognized by the layout recognition unit 131 for the first image and one or more recognition regions recognized for the second image. For example, the correspondence determining unit 136 performs each recognition performed by the character data comparing unit 134 on a combination of one or more recognition regions recognized in the first image and one or more recognition regions recognized in the second image. Correspondence is determined according to the detection result of the process of comparing the first character data and the second character data between the regions to detect the difference. For example, in the detection process of the difference part between the first character data and the second character data performed for the combination of the recognition area of the first image and the recognition area of the second image, the number or ratio of the detected difference parts is It is determined that the combination of the recognition area of the first image and the recognition area of the second image that is the smallest is the corresponding recognition area. Alternatively, in the comparison of morphemes or words, a combination of recognition areas having the smallest number or ratio of detected morphemes or words that are not matched may be determined as the corresponding recognition areas.

  Further, the correspondence determining unit 136 acquires the similarity of one or more character data between the recognition areas of the first image and the second image, and determines that the recognition area having the highest similarity is the corresponding recognition area. You may make it do. Similarity determination can be made, for example, by calculating similarity by performing character string similarity calculation by dynamic programming. For the calculation of the similarity between character strings, refer to the following document “Appendix A Code String Comparison and Matching”, [February 29, 2012 Search], Internet (URL: http: // sslab.nuee.nagoya-u.ac.jp/~sato/lec/kisoron2/string.pdf) ". In addition to the above method, the similarity between regions may be determined using an N-gram. Regarding N-grams, refer to the following documents: "" Text analysis using N-gram model ", [February 29, 2012 search], Internet (URL: http://www.shuiren.org) /chuden/teach/n-gram/index-j.html) ".

  In addition, the correspondence determining unit 136 compares the numbers of the first character image and the second character image (or the number of character codes) between the recognition regions of the first image and the second image, and recognizes that the numbers are the same. You may make it judge the recognition area corresponding between the area | regions or the recognition area | regions with the nearest number.

  Note that, as a result, the correspondence determining unit 136 may cause all of the recognition regions detected in the first image to correspond to all of the recognition regions detected in the second image on a one-to-one basis. preferable.

  The character image comparison unit 137 compares one or more first character images, which are character images acquired from the first image, with one or more second character images, which are character images acquired from the second image, Detect differences. Specifically, the character image comparison unit 137 detects a difference portion by comparing a difference in luminance value between pixels constituting the first image and the second image with a threshold value. As this threshold value, for example, a threshold value stored in threshold value storage means 138 described later is used. This threshold will be described later. For example, the character image comparison unit 137 includes the first character data and the second character data acquired by the character data acquisition unit 133 from the recognition area of the first image determined to be supported by the correspondence determination unit 136 and the recognition area of the second image. And detect the difference. Here, for example, since the character data comparison unit 134 compares character data corresponding to character images included between the two recognition areas, the matching positions of the character data in each recognition area are known in advance. ing. For this reason, character images corresponding to the character data are character-by-character character-by-character from each recognition area with reference to a position where it is determined that the character data matches or does not match between the first image and the second image. The character image comparison unit 137 detects the difference portion by comparing the acquired character images.

  The detection of the difference portion corresponds to, for example, the case where the first character image and the second character image are aligned by changing the size or shape of the first character image and the aligned first character image and the second character image. This is done by comparing pixel values (for example, luminance values) between (overlapping) pixels. For example, a portion where the difference in pixel values is greater than or equal to the threshold is determined to be a difference portion, and a portion where the difference is less than the threshold is determined to be a portion having no difference. Further, it is determined whether or not the first character image and the second character image match based on the number and ratio of the pixels or pixel groups having the difference. That is, it is determined whether or not the compared first character image and second character image are a difference between the first image and the second image. For example, when the number of pixels having a difference or the ratio of pixels having a difference to the pixels of the entire character image exceeds a predetermined threshold value, the comparison between the first character image and the second character image is performed in pixel units. For example, the comparison may be performed in units of blocks including a plurality of pixels. In this case, for example, a value obtained by statistically processing a plurality of pixel values such as an average value of a plurality of pixel values may be compared.

The character image comparison unit 137 accumulates information indicating the second character image determined not to match or the first character image (for example, coordinates and identification information of these images) in a storage medium (not shown). Further, the character image comparison unit 137 may accumulate information (for example, coordinates) indicating a detected difference portion in a storage medium (not shown) or the like. Similarly, information indicating the second character image and the first character image determined to match (for example, the identification information of each of the first character image and the second character image determined to match, the respective coordinates, etc. ) May be stored in a storage medium (not shown). Further, the character image comparison unit 137 may store information indicating a portion having no difference in a storage medium (not shown) or the like.
The comparison by the character data comparison unit 134 performed as described above is a comparison of character data acquired by OCR from the first image or the second image. The OCR selects one candidate from a plurality of character data candidates. Since it is presented, it does not necessarily indicate the accuracy of the characters in the image. Therefore, after comparing with the character data comparing means 134, comparing with the character image comparing means 137 allows more accurate comparison. It can be carried out.

  The threshold storage unit 138 stores a threshold used by the character image comparison unit 137 to determine whether or not the pixel of the first character image matches the pixel of the second character image. The threshold is, for example, a pixel value such as a luminance value or a CMYK value. For example, a threshold value designated in advance by default or the like is stored. Further, the threshold value set by the threshold value setting means 139 described later may be accumulated. Further, threshold values received from a user or the like by a threshold receiving unit 14 described later may be accumulated. The storage here is a concept including temporary storage. The threshold storage means 138 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

  The threshold value setting unit 139 determines whether the pixel of the first character image and the pixel of the second character image match according to the appearance situation of the difference in luminance value of the pixels constituting the first image and the second image, respectively. A threshold value used to determine whether is set. Then, the set threshold value is accumulated in the threshold value storage means 138. The threshold value setting means 139 sets a value obtained by statistical processing as a threshold value. For example, an average value or a median value of differences in luminance values of a plurality of pixels constituting the first image and the second image is set as a threshold value. In addition, for example, an average value, a median value, or the like of a difference between luminance values of pixels constituting each image in a corresponding recognition area between the first image and the second image may be set as a threshold value. In addition, the average value or the median value of the difference in luminance values of the pixels in the first character image and the second character image to be compared, or the pixels in the area including the area of a predetermined size around the first character image and the second character image. You may use for a threshold. In addition, when the difference in luminance value between pixels contained in the first image and the second image (or in the corresponding area thereof) is divided into two classes with a certain threshold, the interclass variance is the largest. Such a threshold value may be acquired and set. Further, instead of using the average value of the luminance value differences between the pixels, the average value difference of the luminance values of the pixels of each image may be used as a threshold value or the like.

  The replacement detection unit 140 detects a place where the character of the second character image is replaced with respect to the first character image. For example, the replacement detection means 140 is a place where two or more consecutive non-matching first character images or non-matching second character images are continuously arranged among the places detected by the character image comparison means 137 as differences. Are detected, and two or more second character images in which the detected portions are continuously arranged are arranged in a different order to generate a permutation. Then, the generated permutation and the two or more first character images that have not been rearranged are compared for each character, and it is determined whether or not they match as described above. Then, when it is determined that each of the two or more second character images constituting one of the generated permutations matches each of the two or more first character images that have not been rearranged, A portion where the plurality of first character images and the second character image do not match is detected as a portion where the character string of the second image is replaced with respect to the first image. Then, information indicating the detected location (for example, identification information of two or more detected second character images and information on positions or areas where the two or more second character images are arranged) is not shown. Accumulate in etc. In addition, although the case where the permutation which rearranged the 2nd character image was used was demonstrated here, the permutation which arranged the 1st character image may be used. Moreover, you may detect the location where the character string was replaced by another method. Moreover, when detecting the location where the 1st character image or 2nd character image which does not correspond is detected, you may determine that the character image is located in a line also when inserting a new line.

  Here, the place where the character string is replaced is detected using the comparison result of the character image comparing means 137, but the character string is replaced using the comparison result of the character data comparing means 134. You may make it detect a location. The same applies to the following addition detection means 141 and deletion detection means 142.

  The addition detection unit 141 detects a place where one or more characters of the second character image are added to the first character image. Here, the portion where the second character image is added to the first character image may be considered as the portion where the first character image is deleted from the second character image. For example, the additional detection unit 141 detects a place where the second character image in which the corresponding first character image does not exist among the places detected by the character image comparison unit 137 as a difference. And the addition detection means 141 acquires the information (for example, identification information of a 2nd character image, a coordinate, etc.) which shows this 2nd character image as information which shows the location where the character was added.

  The deletion detection unit 142 detects a location where one or more characters of the second character image are deleted from the first character image. Here, the portion where the second character image is deleted from the first character image may be considered as the portion where the first character image is deleted from the second character image. For example, the deletion detection unit 142 detects a first character image in which no corresponding second character image exists. Then, the deletion detection unit 142 acquires information indicating the first character image (for example, identification information and coordinates of the first character image) as information indicating a location where the character is deleted.

  The threshold reception unit 14 receives a threshold used by the character image comparison unit 137 to determine whether or not the first character image pixel matches the second character image pixel. For example, a threshold value input by a user or the like via an input unit (not shown) or the like is received. The threshold reception unit 14 accumulates the received threshold in the threshold storage unit 138. The threshold value input means may be anything such as a numeric keypad, a keyboard, a mouse, or a menu screen. The threshold reception unit 14 can be realized by a device driver for input means such as a numeric keypad or a keyboard, control software for a menu screen, or the like.

  The evaluation result receiving unit 15 receives a user evaluation result for one or more differences output from the output unit 18 described later. Specifically, the evaluation result receiving unit 15 receives an evaluation result indicating whether or not the outputted difference portion or a portion that is not a difference is actually a difference portion, for each difference portion. For example, when the output unit 18 indicates a portion where the first character image and the second character image do not coincide with each other as a portion that is not a difference portion, for this portion, using a mouse, a touch panel, or the like, When designating that it is a difference part, the evaluation result reception part 15 receives the evaluation result that it is a difference part about this part. In addition, when the output unit 18 indicates a part where the first character image and the second character image match as a part of the difference, the part of the difference is used with respect to this part using a mouse, a touch panel, or the like. When it is specified that the difference is not, the evaluation result receiving unit 15 receives the evaluation result that the specified difference portion is not the difference portion. Moreover, you may make it receive the evaluation result which shows that it is a difference location about the location correctly shown as a difference location.

  The input means may be anything such as a numeric keypad, keyboard, mouse or menu screen. The evaluation result receiving unit 15 can be realized by a device driver for input means such as a numeric keypad or a keyboard, control software for a menu screen, or the like.

  The luminance value acquisition unit 16 acquires information related to the luminance values of the first character image and the second character image for the difference portion corresponding to each evaluation result received by the evaluation result receiving unit 15. The information about the brightness value acquired here is, for example, a set of differences in brightness values of corresponding pixels between the first character image and the second character image located at the location where the evaluation result is received, or the difference between these brightness values. (For example, an average value, a median value, a maximum value, a minimum value, a value having the highest appearance frequency, etc.). Further, the luminance value of each pixel of the first character image and the second character image located at the location where the evaluation result is received, and the luminance value obtained for one or more constituting each character image are the same as described above. This is a value obtained by statistical processing. The value relating to the luminance value may be a plurality of types of values. Further, the value relating to the luminance value may be a value relating to a difference in luminance value between corresponding pixels of the first character image and the second character image located at the location where the evaluation result is received. The value relating to the luminance value acquired here differs depending on, for example, what kind of machine learning the threshold acquisition unit 17 described later performs and how to set the threshold.

  The luminance value acquisition unit 16 can usually be realized by an MPU, a memory, or the like. The processing procedure of the luminance value acquisition unit 16 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The threshold value acquisition unit 17 acquires the evaluation result received by the evaluation result reception unit 15 and the location corresponding to the evaluation result (specifically, the first character image and the second character image) by the luminance value acquisition unit 16. Learning is performed by using, as learning data, a combination of information relating to the luminance value and a threshold value used by the character image comparison unit 137 when comparing the first character image and the second character image. For learning using learning data, a learning model such as SVR can be used as a known technique. In this case, the learning data may be considered as teacher data. However, learning may be performed in any way. For example, when an evaluation result indicating an error is obtained, the threshold value may be learned by changing the threshold value used for the evaluation to a value in a direction in which a correct evaluation can be obtained. . For example, when an evaluation result indicating that the evaluation is incorrect is received between characters determined to be different characters when evaluated with one luminance value, the threshold value may be decreased by a predetermined value. The optimum threshold value may be learned by increasing it. The threshold acquisition unit 17 accumulates the threshold acquired by learning in the threshold storage unit 138.

  A case where the threshold is used by using the learning result of the threshold acquisition unit 17 will be described below. Immediately before the character image comparison unit 137 compares the first character image and the second character image in one set, the luminance value acquisition unit 16 described above obtains the same luminance as described above from the first character image and the second character image. Get information about values. Then, information regarding the luminance value is input to the threshold value acquisition unit 17. The threshold acquisition unit 17 uses the information on the brightness value to evaluate what evaluation result (that is, the character image comparison unit 137 does when the character image comparison unit 137 performs evaluation using each of the plurality of threshold values. Whether the determination result is a correct value) is obtained using the learning result. Then, one of the threshold values (for example, the lowest threshold value) at which the determination result of the character image comparison unit 137 is correct is acquired and accumulated in the threshold value storage unit 138. Then, the character image comparison means 137 uses the threshold value to compare the first character image and the second character image of the one set.

  The threshold acquisition unit 17 can usually be realized by an MPU, a memory, or the like. The processing procedure of the threshold acquisition unit 17 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The output unit 18 outputs information indicating the location of the difference detected by the detection unit 13. Outputting the information indicating the location of the difference is, for example, outputting the location where the difference is detected in a different output mode from the location where the difference was not detected for the first image, the second image, or a part thereof. It is to be. For example, the first character image or the second character image in which the difference is detected is output in an output mode different from other character images. When the output mode is different, the background color of the output character image is different, the image of the other color is combined with the output character image (for example, by overlaying), the output and the character image are output, It is to output by surrounding with a line of a pattern designated in advance. For example, a first character image or a second character image in which a difference is detected is combined with an image having a different color and output. Further, outputting the information indicating the difference portion may mean outputting information specifying the difference portion with coordinates or the like. When the output is a display, the output mode may be considered as a display mode. Note that the output unit 18 may output a recognition area including a different portion of the recognition areas recognized by the layout recognition unit 131 of the second image or the first image in an output mode different from the others.

  For example, the output unit 18 outputs information indicating a part detected by the character image comparison unit 137 as a difference part. Moreover, the output part 18 outputs the information which shows the location where the character string detected by the replacement detection means 140 is replaced, for example. Further, the output unit 18 outputs information indicating a location where the character detected by the addition detection unit 141 is added. Further, the output unit 18 outputs information indicating a location where the character detected by the deletion detection unit 142 is deleted. The output unit 18 may output information indicating the location of the difference detected by the character data comparison unit 134 for the character code. Further, the output unit 18 may output information indicating a portion where there is a difference between morphemes or words detected by the character data comparison unit 134 using morpheme analysis. Note that the output unit 18 may output portions where different detections are performed in different output modes. For example, the part where the character string detected by the replacement detection unit 140 is switched and the part where the character detected by the addition detection unit 141 is added may be output in different output modes. A different output mode means output in an output mode (for example, a display mode) that can be visually identified. For example, different colors and patterns are overlaid or surrounded by lines of different colors and patterns. Or display underline of different colors or patterns.

  The output described here means display on a display, projection using a projector, printing on a printer, transmission to an external device, storage on a recording medium, processing result to other processing devices or other programs, etc. It is a concept that includes delivery. The output unit 18 may or may not include an output device such as a display or a speaker. The output unit 18 may be implemented by output device driver software, or output device driver software and an output device.

  Next, the operation of the image inspection apparatus 1 will be described using the flowchart of FIG. Here, it is assumed that the first image storage unit 11 and the second image storage unit 12 store a first image and a second image to be compared in advance. Further, as an example here, the first image is image data including character data, and the first image and the first character data extracted from the first image correspond to the first image storage unit 11. It is assumed that it is stored with the attached. Further, it is assumed that the extracted character data is stored in the first image storage unit 11 in association with identification information such as an area (for example, a text box) in which each of the first images is stored.

  (Step S <b> 401) The layout recognition unit 131 performs layout recognition processing on the first image stored in the first image storage unit 11. The layout recognition process performed here is, for example, the same process as the process shown in FIG. The layout recognizing unit 131 associates information defining one or more recognized recognition areas with identification information (for example, a file name) of the first image and identification information of the recognition area, and a storage medium (not shown). To accumulate.

  (Step S <b> 402) The layout recognition unit 131 performs layout recognition processing on the second image stored in the second image storage unit 12. The layout recognition process performed here is, for example, the same process as the process shown in FIG. The layout recognizing unit 131 stores information defining one or more recognized recognition areas in a storage medium or the like (not shown) in association with the identification information of the second image and the identification information of the recognition area.

  (Step S403) The character image acquisition means 132 detects and acquires one or more second character images, which are character images for each character, for each recognition area of the second image. Information indicating the area where each detected second character image is arranged (for example, information indicating the coordinates of the corners of each second character image) is added with identification information of the second character image and is not shown in the figure. Accumulate on media.

  (Step S404) The character image acquisition unit 132 corresponds to each second character image by using one or more second character images acquired by the character image acquisition unit 132 from each recognition region for each recognition region of the second image. The second character data to be acquired is acquired. The acquired second character data is stored in a storage medium (not shown) in association with the identification information of the corresponding second character image.

  (Step S405) The character data comparison means 134 substitutes 1 for the value of the counter m.

  (Step S406) The character data comparing means 134 substitutes 1 for the value of the counter n.

  (Step S407) The character data comparison unit 134 associates the second character data acquired from the m-th recognition area of the second image, of the second character data acquired in Step S404, with the first image. The first character data extracted from the nth recognition area of the first image stored in the first image storage unit 11 is compared. For example, the second character data and the first character data are read and compared one by one in the order designated in advance, and it is determined whether or not they match. The comparison here is, for example, a comparison of character codes or a comparison of hashes obtained for characters. Then, information indicating whether or not they match is stored in a storage medium (not shown) in association with the identification information of the second character image corresponding to the second character data used for the comparison, for example. Further, information for identifying the compared first character data may be further stored in association with each other. If the first character data and the second character data do not match, the character may be skipped and the comparison or the like may be performed again.

  (Step S408) The character data comparison unit 134 uses, for example, information indicating a comparison result obtained by the comparison in step S407 and indicating whether the first character data and the second character data match each other. The identification information of the second character image corresponding to the second character data and the identification information of the compared recognition area are associated with each other and stored in a storage medium (not shown). Further, information for identifying the compared first character data may be further stored in association with each other.

  (Step S409) The character data comparison unit 134 increments the value of the counter n by 1.

  (Step S410) The character data comparison unit 134 determines whether or not there is an nth recognition area among the recognition areas recognized by the layout recognition unit 131 for the first image. If there is, the process returns to step S407, and if not, the process proceeds to step S411.

  (Step S411) The character data comparison unit 134 increments the value of the counter m by 1.

  (Step S412) The character data comparing means 134 determines whether or not the mth recognition area is present in the recognition areas recognized by the layout recognition means 131 for the second image. If there is, the process returns to step S406, and if not, the process proceeds to step S413.

  (Step S413) The correspondence determining unit 136 corresponds to the recognition region of the first image and the recognition region of the second image according to the information indicating the comparison result for each recognition region acquired by the character data comparison unit 134. Determine the recognition area. For example, using the comparison result accumulated in step S408, the ratio (or number) of the first character data that matches the second character data included in this recognition area is the highest for one recognition area of the second image. A high recognition area of the first image is detected, and the detected recognition area and the one recognition area are determined as corresponding recognition areas. Then, information having identification information of each recognition area is stored in a storage medium or the like (not shown) as information indicating the correspondence between the recognition areas.

  (Step S414) The morpheme analysis unit 135 sequentially performs morpheme analysis on the first character data and the second character data corresponding to the recognition areas of the first image and the second image associated by the correspondence determination unit 136. I do.

  (Step S415) The character data comparison means 134 performs comparison between morphemes obtained by morpheme analysis between corresponding recognition regions in order, and accumulates the comparison results in a storage medium (not shown). For example, it is determined whether or not they match in morpheme units, and morpheme identification information is associated with information indicating the presence or absence of the matching morpheme and stored in a storage medium (not shown).

  (Step S416) The character image comparison unit 137 determines whether or not to use the threshold acquired by the threshold setting unit 139 using the learning result as the threshold used for comparing the character images. If so, the process proceeds to step S420. If not, the process proceeds to step S417.

  (Step S417) The threshold reception unit 14 determines whether a threshold is received. If accepted, the process proceeds to step S418. If not accepted, the process proceeds to step S419.

  (Step S418) The threshold receiving unit 14 sets the threshold received in step S417. Specifically, it accumulates in the threshold storage means 138. Then, the process proceeds to step S420.

  (Step S419) The threshold value setting unit 139 obtains a threshold value for each recognition area set by using the appearance status of the luminance value of the pixel in each recognition area for each recognition area set associated in step S413. To do. Then, the acquired threshold value is stored in the threshold value storage means 138 in association with each recognition area group.

  (Step S420) The character image comparison means 137 substitutes 1 for the value of the counter p.

  (Step S421) The character image obtaining unit 132 obtains the first character from the p-th recognition area among the recognition area pairs associated with the first image and the second image in Step S413. An image and a second character image are acquired for each character in accordance with an acquisition order designated in advance. For example, the character image acquisition unit 132 extracts character data associated with the recognition area in the p-th recognition area from the character data extracted from the first image (ie, extracted from the p-th recognition area). The first character image is obtained by rasterizing the character data) for each character. However, rasterization may be performed collectively.

  (Step S422) The character image comparison means 137 compares the first character image and the second character image acquired in step S421 in units of one character, and determines whether or not they are the same character. The determination result is shown in correspondence with the identification information of the first character image used for comparison, the identification information of the first character code corresponding to the first character image used for comparison, and the identification information of the second character image. Do not store in storage media. When comparing, the first character image and the second character image respectively corresponding to the first character data and the second character data associated with the result of comparison by the character data comparison unit 134 are compared. Is preferred. By doing so, there is an effect that it is possible to perform a lean comparison using the comparison result of the character data comparison unit 134. Moreover, the effect that the comparison result of the character data comparison means 134 can be confirmed is acquired. In addition, when it is specified in step S416 that the threshold acquired by the threshold acquiring unit 17 is acquired, for example, when comparing one first character image and a second character image, the luminance value acquiring unit 16 The information about the luminance value is acquired for each pixel of the first character image and the second character image, and the threshold value acquisition unit 17 uses the learning result to use the information about the luminance value to evaluate the evaluation result reception unit 15. Obtains a threshold value that is considered to be an evaluation indicating that the determination result of the character image comparison unit 137 is correct, temporarily stores the threshold value in the threshold value storage unit 138, and uses this threshold value as one first character image The second character image is compared.

  (Step S423) The character image comparison means 137 increments the value of the counter p by 1.

  (Step S424) The character image comparing means 137 determines whether or not the p-th set is present in the recognition area set associated in step S413. If there is, the process returns to step S421, and if not, the process proceeds to step S425.

  (Step S425) The replacement detection unit 140 uses the comparison result of the character image comparison unit 137 or the comparison result of the character data comparison unit 134 to replace the character of the second character image with respect to the first character image. Detect the location. And the information which shows the detected location is accumulate | stored in the storage medium etc. which are not shown in figure.

  (Step S426) The addition detection unit 141 uses the comparison result of the character image comparison unit 137 or the comparison result of the character data comparison unit 134 to detect one or more characters of the second character image with respect to the first character image. Detect the added part. And the information which shows the detected location is accumulate | stored in the storage medium etc. which are not shown in figure.

  (Step S427) The deletion detection unit 142 uses the comparison result of the character image comparison unit 137 or the comparison result of the character data comparison unit 134 to detect one or more characters of the second character image with respect to the first character image. Detect deleted parts. And the information which shows the deleted location is accumulate | stored in the storage medium etc. which are not shown in figure.

  (Step S428) The output unit 18 outputs information indicating a portion where there is a difference between the second image and the first image detected by the detection unit 13. For example, the information which shows the location (namely, location where the 1st character image and the 2nd character image do not correspond) which the character image comparison means 137 detected as a difference location is output. Moreover, the output part 18 outputs the information which shows the location where the character string detected by the replacement detection means 140 is replaced, for example. Further, the output unit 18 outputs information indicating a location where the character detected by the addition detection unit 141 is added. Further, the output unit 18 outputs information indicating a location where the character detected by the deletion detection unit 142 is deleted. The output unit 18 may output information indicating the location of the difference detected by the character data comparison unit 134 for the character code. Further, the output unit 18 may output information indicating a portion where there is a difference between morphemes or words detected by the character data comparison unit 134 using morpheme analysis. Further, these pieces of information may be appropriately switched and output by a user operation received by a reception unit (not shown) or the like. Moreover, you may output the recognition area with a difference among the recognition areas which the layout recognition means 131 recognized about the 2nd image and the 1st image in the output mode different from another recognition area.

  (Step S429) The evaluation result receiving unit 15 determines whether or not the user's evaluation result for one or more differences output by the output unit 18 has been received. If accepted, the process proceeds to step S430. If not accepted, the process proceeds to step S432.

  (Step S430) The luminance value acquisition unit 16 acquires a value related to the luminance value for each location (for example, a character image) for which the evaluation result has been received in Step S429.

  (Step S431) The threshold acquisition unit 17 receives the evaluation result received in Step S429, the value related to the luminance value acquired in Step S430 corresponding to the evaluation result, and the character image comparison unit 137 that is the object of evaluation. Learning is performed using a set of threshold values (for example, a threshold value set by the user in step S418) used by the character image comparison unit 137 in the comparison process as learning data.

  (Step S432) The output unit 18 determines whether or not to end the output. For example, it is determined that the output is to be ended when an instruction to end the output or the like is received via a receiving unit (not shown) or the like. When the output is to be ended, the output is ended and the processing is ended. If the output is not terminated, the process returns to step S428. If there is no need to update the output, the process may return to step S429.

  Note that step S419 and the like may be omitted, and the character image comparison unit 137 may use the threshold value stored in the threshold value storage unit 138 as a default for character image comparison.

  In the flowchart of FIG. 5, the process ends when the power is turned off or the process ends.

  Hereinafter, a specific operation of the image inspection apparatus 1 in the present embodiment will be described.

  FIG. 6 is a diagram illustrating an example of the first image stored in the first image storage unit 11. Here, it is assumed that the first image 221 is an image including character data. In the figure, text boxes 61 to 64 are areas where character data is arranged. As shown in the drawing, it is assumed that “T1061” to “T1064” are assigned to the text boxes 61 to 64 as the identification information of the text box. In the present embodiment, the first image 221 is, for example, an image of a document that is printed on the side of a container such as a can such as a beverage submitted by a design company or a requester of printing. To do. Specifically, it is a document image to be printed on the surface of a cylindrical can, and the file name of the first image 221 is “plate data document”. Here, the case where the container to be printed is a cylindrical can as an example will be described. However, the container to be printed has any shape (rectangular, spherical, etc.) and any material. It may be a container of (metal, plastic, etc.).

  FIG. 7 is a diagram illustrating an example of extracted character data management information for managing extracted character data. The extracted character data management information has items of “file name”, “extracted character data”, and “box ID”. “File name” is the file name of the first image 221 from which character data is extracted. “Extracted character data” is character data extracted from each text box of the first image 221. “Box ID” is identification information of a text box from which character data is extracted. It is assumed that the extracted character data is stored in the first image storage unit 11. Here, each line is considered a record.

  FIG. 8 is a diagram illustrating an example of the second image 222 stored in the second image storage unit 12. The second image 222 is assumed to be the second image 222 corresponding to the first image 221. It is assumed that all the character data in this image data is outlined and does not have character data (character code) information. In the present embodiment, the second image 222 is print data used for printing on an actual can created by a container manufacturer, a printing company that prints on the container, and the like. It is assumed that the print data is originally created. It is assumed that the file name of the second image 222 is “print data”. In the second image 222, it is assumed that the layout or the like has been changed with respect to the first image 221.

  First, the user receives a first image 221 stored in the first image storage unit 11 of the second image 222 “print data” stored in the second image storage unit 12 with respect to a reception unit (not shown) or the like. It is assumed that an instruction for inspecting a difference with respect to the “plate data original” is given.

  The layout recognition unit 131 performs processing for recognizing the layout of the first image 221. First, an area of the same color (color area) is detected for the first image 221 shown in FIG. Next, the layout recognizing unit 131 detects a region having a certain area or more in the detected figure as a candidate region. Next, the layout recognizing unit 131 selects a candidate area having the smallest area among the currently detected candidate areas, acquires information defining the area including the interior as information defining the color area, Accumulate (register) in a storage medium (not shown). Here, it is assumed that the information defining the region is coordinates such as an inflection point around the region. Further, the area identification information is given according to a rule designated in advance. The accumulated area is excluded from the candidate areas. Similarly, for other candidate areas, information defining the color area is sequentially acquired and accumulated.

  For example, for the first image 221, a color area 91 and a color area 92 as shown in FIG. 9 are detected.

  Next, the layout recognition unit 131 detects an area surrounded by a frame for each of the color areas 91 and 92 detected above. First, the layout recognizing unit 131 selects one color area recognized in step S101, compares pixel values (for example, luminance values) between adjacent pixels for each pixel in the selected color area, Extract edges. For example, if the difference between pixel values is equal to or greater than a predetermined threshold value, an edge is detected. From the extracted edge, the layout recognition unit 131 determines whether or not a closed curve surrounded by the edge exists within one color region. Here, since there is a closed curve, one closed curve is selected, and it is determined whether or not the circumference of the closed curve is equal to or longer than a predetermined length. Here, since the length is longer than a predetermined length, the inside of the selected one closed curve is determined as an area surrounded by one frame, information defining this area is acquired, and this area is identified. The information is stored in a storage medium or the like (not shown) in association with the first area identification information that is information. The information defining the area is, for example, coordinates such as an inflection point around the area. Further, another closed curve is detected in the same color area, and if detected, information defining the area inside the detected closed curve and the area identification information are associated with each other and stored in a storage medium (not shown). To do. Similar processing is sequentially performed for other color regions.

  Further, in each detected area, an image matching a registered marker image stored in a storage medium (not shown) or the like is detected in advance, and the detected area is excluded from each area. Here, it is assumed that an image that matches the registered marker image is not detected.

  Furthermore, the layout recognizing unit 131 detects pixels (for example, character pixels) having different colors and brightness from other regions in the remaining region other than the color region from both ends and the vertical direction of the first image 221. Detect the pixel at the highest position, the pixel at the lowest position, the pixel at the leftmost position, and the pixel at the rightmost position, respectively, and leave a rectangular area with sides passing through these points. And the information defining this area and the first area identification information are associated with each other and stored in a storage medium (not shown).

  Accordingly, four recognition regions 1001 to 1004 as shown in FIG. 10 are detected for the first image 221 shown in FIG. It is assumed that the first area identification information associated with each of the four recognition areas 1001 to 1004 is “R01” to “R04”.

  FIG. 11 shows first area management information for managing information defining a recognition area recognized by the layout recognition unit 131 for the first image 221. The first area management information has items of “file name”, “area ID”, and “area definition information”. “File name” is the file name of the first image 221. The “first area ID” is first area identification information. “Region definition information” is information that defines a region. Here, it is assumed that each area is a rectangular area, and the information defining the area is the coordinates of the four corners of the area. In the present embodiment, xr and ys (r and s are integers of 1 or more) are specific numerical values, respectively.

  Similarly, the layout recognition unit 131 also detects one or more recognition regions for the second image 222 shown in FIG. 8 as in the case of the first image 221 described above.

  It is assumed that four recognition areas 1201 to 1204 as shown in FIG. 12 are detected for the second image 222 shown in FIG. 8 by this processing. It is assumed that the second area identification information associated with each of the four recognition areas 1201 to 1204 is “R21” to “R24”.

  FIG. 13 shows second area management information for managing information defining a recognition area recognized by the layout recognition unit 131 for the second image 222. The second area management information has items of “file name”, “second area ID”, and “area definition information”. “File name” is the file name of the second image 222. “Second area ID” is area identification information. “Region definition information” is information that defines a region. Here, it is assumed that each area is a rectangular area, and the information defining the area is the coordinates of the four corners of the area.

  Next, the character image acquisition unit 132 detects one or more second character images for each recognition area recognized by the layout recognition unit 131 for the second image 222. First, the second character image is detected for each character in the recognition region whose “second region ID” is “R21”. For example, one second character image is a minimum rectangular image in which one character fits. Such processing for detecting the second character image is a technique known in the art such as OCR. Then, as information for managing the detected second character image, information that defines the area of each detected second character image, and second character image identification information that is identification information of the detected second character image area In association with the second area ID “R21”, it is stored in a storage medium (not shown).

  FIG. 14 is a diagram illustrating a second character image detected by the character image acquisition unit 132 for the recognition region 1201 whose “second region ID” is “R21”. In the figure, it is assumed that each rectangular image indicated by diagonal lines is each second character image detected by the character image acquisition means 132.

  Similarly, the character image acquisition unit 132 detects the second character image for each recognition region whose “second region ID” is “R22” to “R24”, and manages each detected second character image. Information to be stored in a storage medium (not shown).

  FIG. 15 is a diagram showing second character image management information for managing the second character image detected for each recognition area of the second image 222. The second character image management information includes “second region ID” that is identification information of the recognition region in which the second character image is detected, “second character image ID” that is second character image identification information, and second character It has “character area definition” which is information defining the area of the image.

  Next, the character data acquisition unit 133 acquires character data corresponding to each second character image detected from the second image 222 by using the OCR technique. Here, for the second character image arranged in each recognition area, character data is sequentially acquired from the left to the right from the upper row to the lower row. And the character data acquired about each 2nd character image is made into the 2nd character image identification information which is the identification information of a corresponding 2nd character image, and the 2nd area | region identification information which shows the recognition area in the 2nd image 222. Correspondingly, it is stored in a storage medium (not shown).

  FIG. 16 shows second character data management information for managing the second character data acquired by the character data acquisition means 133. The second character data management information has items of “second region ID”, “second character image ID”, and “second character” as second character data. The “second area ID” corresponds to the “second area ID” in FIG. Here, an actual character is shown as the value of “second character”, but the value of “second character” may be a character code.

  Next, the character data comparison unit 134 compares the one or more second character data acquired for each recognition area of the second image 222 with the first character data included in each recognition area of the first image 221. Then, from the comparison result, the correspondence determining unit 136 determines the correspondence between the recognition areas. Here, as the first character data included in each recognition area of the first image 221, the first character data extracted from the text box included (or overlapped) in each recognition area detected for the first image 221 by the layout recognition unit 131 is used. Get single character data. For example, as shown in FIGS. 6 and 8, the recognition area of the first image 221 whose “first area ID” is “R01” includes a text box 61 whose identification information is “T1061”. Therefore, the character data extracted from the text box 61 is used as the first character data in the recognition region of the first image 221 whose “first region ID” is “R01”. Specifically, as the first character data corresponding to the recognition region of the first image 221 whose “first region ID” is “R01”, the “box ID” is “1” from the extracted character data management information shown in FIG. The value of “extracted character data” of the record “T1061” is used.

  When character data extracted in advance from the first image 221 is not prepared, or when character data cannot be extracted from the first image 221, the character image acquisition unit 132 is similar to the case of the second image 222. The second character data may be acquired by detecting one or more first character images for each recognition area of the first image 221.

  First, the character data comparison unit 134 determines that the second character data acquired from the recognition area where the “second area ID” of the second image 222 is “R21” and the “first area ID” of the first image 221 are “ The first character data acquired from the recognition area “R01” is compared for each character. As the second character data acquired from the recognition region where the “second region ID” of the second image 222 is “R21”, the record of which the “second region ID” of the second character data management information is “R21”. The value of “second character” is acquired and used in order from the top. As the first character data acquired from the recognition region whose “first region ID” is “R01”, the value of “extracted character data” of the record whose “box ID” of the extracted character data management information is “T1061” is used. . As a result of the comparison, a matching character and a non-matching character are detected. In addition, when there is no matching character, a character loss is detected by performing a comparison by shifting the character. The character comparison result is stored in a storage medium (not shown) in association with the first area ID “R01” of the first image 221 and the second area ID “R21” of the second image 222. The character comparison result includes each second character data compared, identification information of the second character image corresponding to each second character data, the first character data compared, and information indicating a match or mismatch. Information.

  FIG. 17 shows comparison result information indicating the comparison results accumulated in a storage medium (not shown). The comparison result information indicates “first area ID” that is first area identification information of the first image 221, “second area ID” that is second area identification information of the second image 222, and a second character. Match or mismatch when character data is compared with “second character”, “second character image ID” which is identification information of the second character image, and “first character” which is the first character data compared. The comparison result, the second character image indicated by the “second character image”, and the first character image corresponding to the first character data indicated by the “first character” are matched or mismatched. The information “image comparison result”, the information “replacement” indicating that the character is replaced, the information “addition” indicating that the character is added, and the character being deleted It has an item “Delete” which is information indicating There. When there is no corresponding first character code or second character code due to missing character data, the values of “second character”, “second character image ID”, and “first character” are blank (for example, NULL). It becomes. Also, here, the comparison between the first character image and the second character image and the detection of the replacement, addition, deletion, etc. of the characters are not performed, so “image comparison result”, “replacement”, “addition”, “deletion” "Is blank (for example, NULL).

  Similarly, the character data comparison unit 134 determines that the second character data acquired from the recognition area where the “second area ID” of the second image 222 is “R21” and the “first area ID” of the first image 221 are the same. Comparison result information is obtained by comparing the first character data respectively obtained from the recognition areas “R02” to “R04” for each character.

  Further, the character data comparison unit 134 repeats the same processing for each recognition area whose “second area ID” of the second image 222 is “R22” to “R24”.

  Next, in the comparison result information stored in a storage medium (not shown), the correspondence determination unit 136 indicates that the value of the “comparison result” is “match” among the records whose “second area ID” is “R21”. The number of records is counted for each value of “first area ID”, and the recognition area of the first image 221 indicated by the “first area ID” having the highest total value is “R21 ID”. Is a recognition area corresponding to the recognition area of the second image 222. Similarly, the recognition area of the corresponding first image 221 is also determined for the recognition areas whose “second area ID” is “R22” to “R24”. Then, the area correspondence management information, which is information having “first area ID” and “second area ID” of the corresponding recognition area, is stored in a storage medium (not shown).

  FIG. 18 shows area correspondence management information indicating the correspondence between recognition areas. The area correspondence management information has a “second area ID” and a “first area ID”.

  Thus, even if there is a large variation in the recognition area where the same sentence string is arranged in the first image 221 and the second image 222 by acquiring the correspondence relationship between the recognition areas, Can be compared.

  Note that the record of the comparison result information acquired between the recognition areas that are not compatible may be deleted.

  Next, the morpheme analysis unit 135 performs morpheme analysis on the first character data and the second character data in the recognition area determined to correspond by the correspondence determination unit 136 of the first image 221 and the second image 222, respectively. I do.

  Then, the character data comparison unit 134 compares the first character data and the second character data of each recognition area determined to correspond to each morpheme obtained by the morpheme analysis according to the character arrangement order. . Then, the character data comparison unit 134 uses the identification information of the recognition area of the compared first image 221 and the recognition area of the compared second image 222 as morpheme comparison result information that is information indicating the comparison result of the morpheme. A storage medium (not shown) including identification information, a morpheme of the first image 221 used for comparison, a morpheme of the second image 222 used for comparison, and information indicating matching or mismatching as a comparison result To accumulate.

  FIG. 19 is a diagram showing an example of morpheme comparison result information stored in a storage medium (not shown) by the character data comparison unit 134. The morpheme comparison result information includes items of “first region ID”, “second region ID”, “first morpheme”, “second morpheme”, and “comparison result”. The “first morpheme” is a morpheme of the first image 221. The “second morpheme” is a morpheme of the second image 222. The “comparison result” is a comparison result between the corresponding “first morpheme” and “second morpheme”, the value “match” indicates that the morphemes match, and “mismatch” indicates that they do not match. .

  Next, the threshold setting unit 139 determines whether or not to use the threshold acquired by the threshold acquisition unit 17 for comparison between the first character image and the second character image. Here, for example, if not used is set in advance by initial setting or the like, the threshold reception unit 14 displays an input interface or the like for receiving a threshold from the user on a monitor or the like. Here, if the user inputs an instruction not to input a threshold value, the threshold setting unit 139 selects that the threshold value set by the threshold setting unit 139 is used for comparison between the first character image and the second character image. The threshold value used for comparison is acquired. Here, the threshold value setting unit 139 acquires and sets a threshold value for each pair of the recognition area of the first image 221 and the recognition area of the second image 222 that are associated with the area correspondence management information illustrated in FIG. To do. For example, the threshold value setting unit 139 includes a recognition area whose “first area ID” is “R04” and a recognition area whose “second area ID” is “R21” associated with each other in the area correspondence management information. The brightness value of each pixel is acquired, and the average of the brightness values of each recognition area is calculated. Then, the difference between the average value of the luminance values of the first area and the average value of the luminance values of the second area is set to the first character image in the recognition area whose “first area ID” is “R04”, and “second area” The threshold is used for comparison with the second character image in the recognition area whose ID is “R21”. Here, as an example, a value obtained by subtracting the average value of the luminance values of the first region from the average value of the luminance values of the second region is set as the threshold value. The threshold value setting unit 139 sets threshold values for other corresponding recognition regions as well as the average value of the differences obtained by subtracting the luminance values of the first region.

  In addition, what is necessary is just to use the threshold value which the user input, when a user inputs a threshold value. When the user does not input a threshold value, a dialog or the like is issued to confirm whether or not the threshold value set by the threshold value setting unit 139 is used, and the threshold value set by the threshold value setting unit 139 is determined by the user. When it is instructed not to use, a threshold set by default or the like may be used.

  Next, the character image acquisition unit 132 sets the first character image and the combination of the recognition region of the first image 221 and the recognition region of the second image 222 associated with the region correspondence management information shown in FIG. Acquire the second character image one character at a time. Here, the character image acquisition means 132 acquires the 2nd character image detected above for every character as a 2nd character image of one recognition area. Further, the character image acquisition unit 132 rasterizes (ie, bitmaps) the character data extracted from the text box in one recognition area of the first image 221 for each character, so that the first character image in one recognition area. To get. Here, the combination of the second character image indicated by the “second character image ID” included in each record of the comparison result information shown in FIG. 17 and the first character image obtained from the first character data indicated by the “first character”. In order from the information record.

  The character image comparison unit 137 calculates a difference in luminance value of corresponding pixels between the first character image and the second character image for each character acquired by the character image acquisition unit 132. When the first character image and the second character image to be compared have different sizes (vertical and horizontal pixel numbers), the size of one or both of the first character image and the second character image is changed so that the pixel numbers match. And the pixel of the same coordinate of each character image is judged as a corresponding pixel. Here, as an example, a value obtained by subtracting the luminance value of the pixel of the first character image from the luminance value of the pixel of the second character image is set as the difference in the luminance value of the pixel. Then, the threshold value setting means 139 compares the threshold value set above with the calculated luminance value difference for the set of recognition areas. If the luminance value difference is equal to or larger than the threshold value, the compared pixels are not matched. to decide. The determination result is temporarily stored in, for example, a storage medium (not shown). Then, after the same comparison is performed for the corresponding pixels of the first character image and the second character image in one set, the character image comparison unit 137 compares the number of pixels determined to be inconsistent. When the calculated value is equal to or greater than a predetermined threshold value, it is determined that the compared first character image and the second character image do not match, and when the calculated value is less than the threshold value, they are determined to match. To do. If they match, the value of “image comparison result” of the corresponding record is set to “match”, which is a value indicating matching, similar to the comparison result information shown in FIG. Sets the value to “mismatch”, which is a value indicating that they do not match.

  In this example, when the character image comparison unit 137 acquires a character image corresponding to each record of the comparison result information, there is no one of “second character image ID” or “first character” in one record. In this case, neither the first character image nor the second character image corresponding to this record may be read out, or one of them may be read out, but the comparison between the character images may not be performed. In this case, the value of “image comparison result” of this record is set to “mismatch”.

  The same processing is performed for the first character image and the second character image of the associated recognition area of one set, and then the same processing is performed for the other recognition areas.

  FIG. 20 is a diagram illustrating an example of comparison result information obtained as a result of the character image comparison unit 137 comparing the first character image and the second character image.

  The replacement detection unit 140 detects a place where the character of the second image 222 is replaced with respect to the first image 221 using the comparison result information as shown in FIG. Specifically, the replacement detection unit 140 acquires the value of the “image comparison result” of the comparison result information illustrated in FIG. 20 in order from the upper record to the lower record, and displays the “image comparison result” value. A plurality of consecutive records whose values are “mismatch” are detected. Then, the arrangement order of the “second character” values of the detected plurality of records is changed, and a plurality of character strings are acquired. Then, compare each acquired character string with the character string in which the "first character" of the detected multiple records is extracted from the top record toward the bottom record, and whether there is a match. Judge whether or not. If there is a match, it is determined that the characters have been replaced. Then, “1”, which is a value indicating that the character has been replaced, is input as the “replacement” value of the plurality of consecutive records. If there is no match, it is determined that the character has not been replaced, and the value of “replacement” is left blank (for example, NULL).

  The addition detection unit 141 detects a portion where the character of the second image 222 has been added to the first image 221 using the comparison result information as shown in FIG. Specifically, the addition detection unit 141 sequentially sets the value of “first character” and the value of “second character” of the comparison result information shown in FIG. 20 from the upper record to the lower record. Acquired records are detected in which the value of “first character” is blank (NULL) and the value of “second character” is not blank. Then, “1”, which is a value indicating that a character has been added, is input as the “addition” value of the plurality of detected records.

  The deletion detection unit 142 uses the comparison result information as illustrated in FIG. 20 to detect a portion where the character of the second image 222 has been deleted from the first image 221. Specifically, the deletion detection unit 142 sequentially sets the value of “first character” and the value of “second character” in the comparison result information shown in FIG. 20 from the upper record to the lower record. Acquired records are detected in which the value of “second character” is blank (NULL) and the value of “first character” is not blank. Then, “1”, which is a value indicating that the character has been deleted, is input as the “delete” value of the plurality of detected records.

  FIG. 21 is a diagram illustrating comparison result information obtained as a result of character replacement, addition, or deletion detection.

  The output unit 18 uses the comparison result information shown in FIG. 21 and the second character image management information shown in FIG. 15 as a result of comparison between the first image 221 and the second image 222. Obtain and output information indicating the determined location. For example, a display image indicating a location determined to have a difference is acquired and output to a monitor (not shown) or the like.

  For example, the output unit 18 reads the second image 222 stored in the second image storage unit 12, and stores the second image 222 on each region indicated by “character region definition” of each record of the second character image management information illustrated in FIG. 15. In other words, an image of a color designated in advance is overlaid on the area of each second character image. At this time, in the comparison result information shown in FIG. 21, the second character image ID having the “second character image ID” that matches the value of the “second character image ID” of the record whose “image comparison result” value is “mismatch”. An image of a predesignated color different from the color of the other image to be overlaid is overlaid on the area indicated by “character area definition” of the record of the two-character image management information. Similarly, the first image 221 stored in the first image storage unit 11 is read out, and an image of a color designated in advance is placed on the minimum rectangular area including each character of the character data arranged in the first image 221. 21 is overlaid on other characters on the character indicated by the first character data corresponding to the record whose “image comparison result” value is “mismatch” in the comparison result information shown in FIG. Overlay an image with a color different from the image color. The character indicated by the first character data corresponding to the one record whose “image comparison result” value is “mismatch” is the one record in the record having the same “first region ID” as the one record. Text box included in the recognition area on the first image 221 (or overlapping the recognition area) indicated by the order of appearance (array order counted from the top of the record) and the “first area ID” of this one record Can be identified by detecting. For example, among the one or more character data arranged in the detected text box, the character whose order from the top is the order indicated by the appearance order detected above is the value of “image comparison result”. Is a character indicated by the first character data corresponding to one record in which “mismatch” is present. A text box included in the recognition area on the first image 221 indicated by one “first area ID” (or overlapping with the recognition area) is a recognition area or first image 221 recognized in the first image 221. It can be obtained using information such as coordinates that define the area in which the text box is placed. When character data is extracted from the first image 221 in advance, information indicating the position (or region) in the first image 221 where the character is arranged is acquired for each character data. Thus, information indicating this position (or region) is associated with each character data and managed in the extracted character data management information, so that the value of “image comparison result” is “ The position (or region) of the character indicated by the first character data corresponding to one record that is “mismatch” may be detected in the first image 221.

  Then, the first image 221 and the second image 222 obtained by overlaying the colored images in this way are displayed on the monitor, whereby the first image 221 and the second image showing the difference are displayed. An image 222 can be displayed.

  In this example, an image of a color different from the other color is overlaid on the character whose “image comparison result” value is “mismatched”, but the character whose “comparison result” value is “mismatch”. In addition, an image of a color different from the others may be displayed in an overlay manner.

  Further, the output unit 18 designates the entire recognition area of the first image 221 and the second image 222 in advance when a difference is detected in one recognition area recognized by the layout recognition unit 131. It is also possible to display an overlay with an image of the selected color.

  FIG. 22 is a diagram illustrating a display example of the first image 221 and the second image 222 showing a portion having a difference. In the figure, an area 220 is a portion where a difference is displayed as an overlay. In the figure, a first image 221 and a second image 222 are displayed side by side. Thereby, a user can be notified of a part with a difference.

  Further, it is assumed that the user gives an instruction to display the place where the character is replaced to the image inspection apparatus 1 via a reception unit (not shown) or the like. In the display showing the difference portion shown in FIG. 22, the output unit 18 instead of overlaying an image of a color different from that of other characters on the character whose “image comparison result” value is “mismatch”. 21, the second character image management having “second character image ID” that matches the value of “second character image ID” of the record whose “replacement” value is “1”. An image of a color different from the image overlaid on another character is overlaid on the region indicated by the “character region definition” of the information record. The same applies to the first image 221.

  FIG. 23 is a diagram showing a display example of the first image 221 and the second image 222 showing the places where characters have been replaced. In the figure, a region 230 is a portion where a difference is displayed in an overlay manner. Thereby, it is possible to notify the user of the place where the character is changed.

  Further, it is assumed that the user gives an instruction to display the location where the character is added to the image inspection apparatus 1 via a reception unit (not shown) or the like. In the display showing the difference portion shown in FIG. 22, the output unit 18 instead of overlaying an image of a color different from that of other characters on the character whose “image comparison result” value is “mismatch”. 21, the second character image management having “second character image ID” that matches the value of “second character image ID” of the record whose “addition” value is “1” in the comparison result information shown in FIG. An image of a color different from the image overlaid on another character is overlaid on the region indicated by the “character region definition” of the information record. The same applies to the first image 221.

  FIG. 24 is a diagram showing a display example of the first image 221 and the second image 222 showing the places where characters have been added. In the figure, a region 240 is a portion where a difference is displayed in an overlay manner. As a result, the user can be notified of the location where the character has been added.

  Further, it is assumed that the user gives an instruction to display the location where the character has been deleted to the image inspection apparatus 1 via a reception unit (not shown). In the display showing the difference portion shown in FIG. 22, the output unit 18 instead of overlaying an image of a color different from that of other characters on the character whose “image comparison result” value is “mismatch”. In the comparison result information shown in FIG. 21, the second character image management having “second character image ID” that matches the value of “second character image ID” of the record whose “delete” value is “1”. An image of a color different from the image overlaid on another character is overlaid on the region indicated by the “character region definition” of the information record. The same applies to the first image 221.

  FIG. 25 is a diagram showing a display example of the first image 221 and the second image 222 showing the locations where characters have been deleted. In the figure, a region 250 is a portion where a difference is displayed as an overlay. As a result, the user can be notified of the location where the character has been deleted.

  In addition, it is assumed that the user gives an instruction to display the difference portion detected using morphological analysis to the image inspection apparatus 1 via a reception unit (not shown). The output unit 18 acquires the values of the “first morpheme” and the “second morpheme” in order from the upper record to the lower record of the morpheme analysis result information shown in FIG. Acquire images of character strings arranged in accordance with the acquisition order, and each of the regions on the morpheme where the value of the “comparison result” is “match” and the region on the morpheme that is “mismatch” , Overlay images of different colors specified in advance. The output unit 18 displays the image acquired in this manner on a monitor or the like (not shown).

  FIG. 26 is a display example showing a detection result of a difference portion by morphological analysis. In the figure, an area 261 indicates the location of the detected difference. Although only the characters are displayed here, as shown in FIG. 22, the areas where the characters corresponding to the morphemes are arranged on the first image 221 and the second image 222 are detected and detected. An image of a color corresponding to the value of the morpheme comparison result may be overlaid on the area.

  Here, for example, it is assumed that a location where a difference is detected as shown in FIG. For example, a corresponding one first character image and one second character image are the same character, but are detected as different portions, or corresponding one first character image and one second character image. It is assumed that the user has found a portion detected as a portion having no difference even though the character image is a different character. If the user designates a character positioned at such a location on the second image 222 as a location that the image inspection apparatus 1 misidentifies by clicking with a mouse or the like, the evaluation result receiving unit 15 The character image comparison means 137 accepts an evaluation result indicating that the second character image at the clicked position is erroneous. The evaluation result receiving unit 15 receives an evaluation result indicating that the determination is correct for the second character image at the position where the click has not been made.

  The luminance value acquisition unit 16 acquires, for each evaluation result received by the evaluation result reception unit 15, a second character image that has received the evaluation result and a first character image that has been compared with each of the second character images. For each evaluation result, information on the brightness value is acquired from the first character image and the second character image. It is preferable that there are many types of information regarding luminance values. For example, as the information about the brightness value, the average value, median value, maximum value, and minimum value of the brightness of the first character image, and the average value, median value, maximum value, and minimum value of the brightness of the second character image are acquired. Further, the luminance and the like having the highest number of appearances in the first character image and the second character image are also acquired. In addition, the luminance value acquisition unit 16 determines, for each evaluation result, the difference between the luminance values used by the character image comparison unit 137 to determine whether or not the pixels of the first character image and the second character image match. Get the threshold.

  For each evaluation result acquired above, the threshold acquisition unit 17 learns a set of the evaluation result, information about the luminance value, and the threshold as teacher data.

  Then, the character image comparison unit 137 uses a threshold value set by the threshold value setting unit 139 as a threshold value for determining whether or not the pixels of the first character image and the second character image coincide with each other via a reception unit (not shown). Assume that the user has set the image inspection apparatus 1.

  In this case, immediately before the character image comparison means 137 compares the corresponding pixels of one first character image and one second character image, the luminance value acquisition unit 16 makes each first character to be compared. From the image and the second character image, information related to the brightness value of the item similar to the above is acquired. The threshold acquisition unit 17 receives a combination of information on the luminance value and each of a plurality of thresholds prepared in advance as an input, and outputs an evaluation result for each threshold. The result of evaluation for each threshold value that is output is that the evaluation result is correct when the difference between corresponding pixels of the first character image to be compared and the second image 222 is evaluated using one threshold value. It is a value indicating whether or not there is an error. For this reason, the threshold value acquisition unit 17 acquires one threshold value (here, the threshold value with the smallest value) from the threshold values at which the evaluation result is correct, and temporarily stores it in the threshold value storage means 138. Then, the character image comparing means 137 uses this threshold value to determine whether or not the pixels of the first character image and the second character image match. Thereby, the pixel comparison by the threshold value using the learning result becomes possible.

  In this specific example, the case where character data is extracted in advance from the first image 221 has been described. However, the present invention is applicable even if character data is not extracted in advance, for example, If character data has not been extracted in advance, the same processing as that for acquiring the second character data or the second character image from the second image 222 may be performed on the first image 221 as well. .

  As described above, according to the present embodiment, the difference between the first image 221 and the second image 222 is detected by comparing the character images acquired from the first image 221 and the second image 222. Therefore, the difference in content between images can be detected by the presence or absence of substantial information regardless of the difference in layout or the difference in character size.

  In addition, for the recognition areas detected by performing the layout recognition in each of the first image 221 and the second image 222, the corresponding recognition area is determined, and the characters are compared between the corresponding recognition areas. Even if the recognition area in which the characters are arranged has been greatly changed, the characters can be compared correctly.

  In the present embodiment, the layout recognition unit 131 performs layout recognition on the first image 221 and the second image 222 to detect a recognition region, and the correspondence determination unit 136 detects a corresponding recognition region. The character image comparison means compares the character images between the corresponding recognition regions. If there is no significant difference in the character layout between the first image 221 and the second image 222, If there is only one image, the layout recognition process may not be performed. In this case, the layout recognition unit 131 and the like can be omitted. Here, the character data acquired from the character image is used together, but the comparison of the characters using the character data is omitted, and the characters arranged in the first image 221 and the second image 222 depend on the character image. Only the comparison may be performed.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  Further, although cases have been described with the above embodiments where the image inspection apparatus is a stand-alone, the image inspection apparatus may be a stand-alone apparatus or a server apparatus in a server / client system. In the latter case, the output unit or the reception unit receives an input or outputs a screen via a communication line.

  In each of the above embodiments, each component such as the detection unit 13, the luminance value acquisition unit 16, and the threshold acquisition unit 17 may be configured by dedicated hardware, or a component that can be realized by software. May be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. At the time of execution, the program execution unit may execute the program while accessing a storage unit (for example, a recording medium such as a hard disk or a memory).

  The software that realizes the image inspection apparatus in each of the above embodiments is the following program. In other words, the program sets up a computer that can access the first image storage unit in which the first image is stored and the second image storage unit in which the second image is stored. Comparing, a program for functioning as a detection unit that detects a location of a difference and an output unit that outputs information indicating the location of the difference detected by the detection unit. A character image acquisition means for acquiring a character image from each of the images; one or more first character images that are character images acquired from the first image; and one or more first characters that are character images acquired from the second image. This is a program comprising character image comparing means for comparing two character images and detecting a difference portion.

  In the above program, in a transmission step for transmitting information, a reception step for receiving information, etc., processing performed by hardware, for example, processing performed by a modem or an interface card in the transmission step (only performed by hardware). Not included) is not included.

  In the program, the functions realized by the program do not include functions that can be realized only by hardware. For example, a function that can be realized only by hardware such as a modem or an interface card in an acquisition unit that acquires information or an output unit that outputs information is not included in the function realized by the program.

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 27 is a schematic diagram illustrating an example of an external appearance of a computer that executes the program and realizes the image inspection apparatus according to the embodiment. The above-described embodiment can be realized by computer hardware and a computer program executed on the computer hardware.

  27, a computer system 900 includes a computer 901 including a CD-ROM (Compact Disk Read Only Memory) drive 905 and an FD (Floppy (registered trademark) Disk) drive 906, a keyboard 902, a mouse 903, a monitor 904, and the like. Is provided.

  FIG. 28 is a diagram showing an internal configuration of the computer system 900. 28, in addition to the CD-ROM drive 905 and the FD drive 906, a computer 901 is connected to an MPU (Micro Processing Unit) 911, a ROM 912 for storing a program such as a bootup program, and the MPU 911. A RAM (Random Access Memory) 913 that temporarily stores program instructions and provides a temporary storage space, a hard disk 914 that stores application programs, system programs, and data, and an MPU 911 and a ROM 912 are interconnected. And a bus 915. The computer 901 may include a network card (not shown) that provides connection to the LAN.

  A program that causes the computer system 900 to execute the functions of the image inspection apparatus and the like according to the above-described embodiment is stored in the CD-ROM 921 or FD 922, inserted into the CD-ROM drive 905 or FD drive 906, and stored in the hard disk 914. May be forwarded. Instead, the program may be transmitted to the computer 901 via a network (not shown) and stored in the hard disk 914. The program is loaded into the RAM 913 when executed. The program may be loaded directly from the CD-ROM 921, the FD 922, or the network.

  The program does not necessarily include an operating system (OS) or a third party program that causes the computer 901 to execute the functions of the image inspection apparatus according to the above embodiment. The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 900 operates is well known and will not be described in detail.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the image inspection apparatus according to the present invention is suitable as an apparatus for inspecting a difference between images, and particularly useful as an apparatus for inspecting a difference between characters between images. is there.

DESCRIPTION OF SYMBOLS 1 Image inspection apparatus 11 1st image storage part 12 2nd image storage part 13 Detection part 14 Threshold value reception part 15 Evaluation result reception part 16 Luminance value acquisition part 17 Threshold value acquisition part 18 Output part 131 Layout recognition means 132 Character image acquisition means 133 Character data acquisition means 134 Character data comparison means 135 Morphological analysis means 136 Correspondence determination means 137 Character image comparison means 138 Threshold storage means 139 Threshold setting means 140 Replacement detection means 141 Additional detection means 142 Deletion detection means

Claims (11)

A first image storage unit storing a first image including two or more characters;
A second image storage unit that stores a second image including two or more characters;
A detector for comparing the first image with the second image to detect a difference portion;
An output unit that outputs information indicating the location of the difference detected by the detection unit;
The detector is
A character image acquisition means for acquiring a character image from each of the first image and the second image;
Two or more first character data that is character data corresponding to a character image included in the first image; and two or more second character data that are character data corresponding to a character image included in the second image; Character data acquisition means for acquiring
Character data comparison means for comparing the first character data acquired by the character data acquisition means and the second character data in the arrangement order to detect a difference portion;
Two or more first character images that are character images acquired from the first image, and two or more second character images that are character images acquired from the second image, the character data comparing means Character image comparison means for comparing pixel values of pixels constituting the first character image and the second character image respectively corresponding to the compared first character data and second character data, and detecting a difference portion Prepared ,
The image output apparatus, wherein the output unit outputs information indicating a difference location detected by the character data comparison means and information indicating a difference location detected by the character image comparison means . The detection unit further includes a replacement detection means for detecting a place where the character of the second character image is replaced with respect to the first character image,
The image inspection apparatus according to claim 1, wherein the output unit outputs information indicating a place where a character detected by the replacement detection unit is replaced. The detection unit further includes an additional detection unit that detects a location where one or more characters of the second character image are added to the first character image,
The image inspection apparatus according to claim 1, wherein the output unit outputs information indicating a location where the character detected by the addition detection unit is added. The detection unit further includes a deletion detection unit that detects a location where one or more characters of the second character image are deleted from the first character image,
The image inspection apparatus according to claim 1, wherein the output unit outputs information indicating a location where the character detected by the deletion detection unit is deleted. The detector is
Morphological analysis means for performing morphological analysis for each of the first character data and the second character data acquired by the character data acquisition means,
The image inspection apparatus according to claim 1, wherein the character data comparison unit compares morphemes or words respectively obtained for the first character data and the second character data by the morphological analysis, and detects a difference portion. The detection unit performs layout recognition for each of the first image and the second image, and detects one or more regions;
Correspondence determining means for determining a correspondence relationship between one or more areas recognized by the layout recognition means for the first image and one or more areas recognized for the second image;
The character data comparing means is acquired for each recognized area for a combination of one or more areas recognized by the layout recognition means for the first image and one or more areas recognized for the second image. The first character data corresponding to the first character image and the second character data corresponding to the second character image are compared along the arrangement order to detect a difference portion,
The correspondence determination means includes first character data and second character data that the character data comparison means performs for a combination of one or more areas recognized for the first image and one or more areas recognized for the second image. According to the detection result of the difference between and the corresponding area,
The character image acquisition means acquires the first character image and the second character image for each set of regions determined to be supported by the correspondence determination means,
The image inspection apparatus according to claim 1, wherein the character image comparison unit compares the first character image and the second character image for each region to detect a difference portion. The character image comparison means detects a difference portion by comparing a difference between luminance values of pixels constituting at least the first image and the second image with a threshold value,
The detection unit includes a threshold setting unit that sets a value obtained by statistically processing a difference in luminance value of pixels constituting the first image and the second image as the threshold. The image inspection apparatus according to claim 6. A threshold receiving unit for receiving a threshold input;
The character image comparison unit detects a difference portion by comparing a difference in luminance value of pixels constituting at least the first image and the second image with a threshold received by the threshold receiving unit. The image inspection apparatus according to claim 7. The character image comparison means detects a difference portion by comparing a difference between luminance values of pixels constituting at least the first character image and the second character image with a threshold value,
An evaluation result receiving unit that receives a user's evaluation result for one or more differences output by the output unit;
A luminance value acquisition unit that acquires information about the luminance values of the first character image and the second character image for the location of the difference corresponding to each evaluation result received by the evaluation result reception unit;
The combination of the evaluation result and the information related to the luminance value of the portion corresponding to the evaluation result and the combination of the threshold value are learned, and the threshold value corresponding to the first character image and the second character image to be compared is acquired. The image inspection apparatus according to claim 1, further comprising a threshold acquisition unit. A first image storage unit that stores a first image including two or more characters, a second image storage unit that stores a second image including two or more characters, a character image acquisition unit, and a character data acquisition unit; An image inspection method performed using a detection unit including a character data comparison unit and a character image comparison unit, and an output unit,
The detection unit compares the first image with the second image, and detects a difference portion; and
The output unit includes an output step of outputting information indicating a location of the difference detected by the detection unit;
The detecting step includes
A character image acquisition step in which the character image acquisition means acquires a character image from each of the first image and the second image;
The character data acquisition means is character data corresponding to two or more first character data that is character data corresponding to a character image included in the first image and character data corresponding to a character image included in the second image. A character data acquisition step for acquiring the above second character data;
The character data comparison unit detects the difference between the first character data acquired in the character data acquisition step and the second character data in the arrangement order, and the character image comparison. The means includes two or more first character images that are character images acquired from the first image, and two or more second character images that are character images acquired from the second image, wherein the character data Character image comparison for detecting a difference portion by comparing pixel values of pixels constituting the first character image and the second character image respectively corresponding to the first character data and the second character data compared in the comparison step With steps ,
The image inspection method in which the output step outputs information indicating a difference portion detected in the character data comparison step and information indicating a difference portion detected in the character image comparison step . A computer accessible to a first image storage unit storing a first image including two or more characters and a second image storage unit storing a second image including two or more characters;
A detector for comparing the first image with the second image to detect a difference portion;
A program for functioning as an output unit that outputs information indicating the location of the difference detected by the detection unit,
The detector is
A character image acquisition means for acquiring a character image from each of the first image and the second image;
Two or more first character data that is character data corresponding to a character image included in the first image; and two or more second character data that are character data corresponding to a character image included in the second image; Character data acquisition means for acquiring
Character data comparison means for comparing the first character data acquired by the character data acquisition means and the second character data in the arrangement order to detect a difference portion;
Two or more first character images that are character images acquired from the first image, and two or more second character images that are character images acquired from the second image, the character data comparing means Character image comparison means for comparing pixel values of pixels constituting the first character image and the second character image respectively corresponding to the compared first character data and second character data, and detecting a difference portion Prepared ,
The output unit is a program for outputting information indicating a difference location detected by the character data comparison means and information indicating a difference location detected by the character image comparison means .

Images