JP7171269B2 - Image matching system, image matching method, and program - Google Patents

Description

The present invention relates to an image matching system, an image matching method, and a program.

2. Description of the Related Art A visual inspection by an operator has been the main method for comparing document image data with printed matter printed based on the document image data to determine whether or not the printed matter has been printed correctly. Instead of the visual inspection, the printed matter is digitized and an inspection is adopted in which the digitized data of the printed matter and the document image data are collated.

As an example of the digitization of printed matter, an image of the printed matter is captured using an electronic device such as a scanner device and an imaging device, and prescribed image processing is performed on the captured data to generate digitized data of the printed matter. be done.

The copying machine described in Japanese Patent Application Laid-Open No. 2002-200001 outputs one sample image as a pre-process for continuous image formation on several hundred sheets, and the user visually checks the sample image. For example, if the user determines that the red color is insufficient, an adjustment is made to increase the red color.

The copying machine described in Patent Document 1 outputs a sample image again after adjustment. When a sample image of the quality requested by the user is obtained, the sample image is regarded as a non-defective image, and the reference image data acquisition mode is started. In the reference image data acquisition mode, a sensor unit is used to read a transfer sheet on which a non-defective image has been formed, thereby acquiring reference image data. After the preprocessing is completed, continuous image formation is performed while image quality determination is performed using the reference image data.

Various changes may occur in the digitized data when the printed matter is digitized. In collation that simply compares the document image data and the digitized data of the printed matter, there is concern that an erroneous determination may occur due to changes in the digitized data. The occurrence of erroneous determination causes a problem that prescribed inspection accuracy cannot be obtained.

The degree of change in digitized data at the stage of digitizing printed matter is affected by the type of media used in the printed matter, individual differences in the equipment used for digitization, and deterioration of printed matter over time, so it cannot be calculated simply. Difficult to correct. Therefore, instead of correcting the change in the stage of digitizing printed matter by calculation, it is a method of confirming the identity of digitized data by comparing digitized data that were digitized at the same time using the same equipment. is adopted. Such a method is effective when creating a plurality of prints from the same document image data.

In Patent Document 2, images formed on a plurality of sheets are read, one of the plurality of read images is set as a reference image, and the other is set as an inspection image, and the feature amount of the reference image and the feature amount of the inspection image are calculated. An image forming apparatus is disclosed that determines whether an inspection image is good or bad by collation.

Japanese Patent No. 5168651 JP 2016-1415 A

However, when printing a large amount of the same printed matter and inspecting whether or not the original image data is printed correctly for all the printed matter, if the original image data and the digitized data of all the printed matter are collated, the digitizing device Due to the influence of the change that occurs between the printed matter and the digitized data due to the above, it is difficult to achieve the determination accuracy between the original image data and the digitized data.

In the copier disclosed in Patent Document 1, an operator prepares correct digitized data of printed matter in advance and registers it as a sample image. Operation is troublesome, such as comparing with reference data after repeating the preparation of sample images and visual checks.

The image forming apparatus described in Patent Document 2 is effective for inspection when a plurality of printed materials are created from the same document image data, but the image selected as the reference image may match the document image data. Sex is not 100%.

On the other hand, the image forming apparatus described in Patent Document 2 is based on the premise that the reference image matches the document image data. In this case, an erroneous determination may occur when the image selected as the reference image does not match the document image.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image collating system, an image collating method, and a program capable of suppressing erroneous judgments in inspection of printed matter.

In order to achieve the above object, the following aspects of the invention are provided.

An image matching system according to a first aspect includes a document image data acquisition unit that acquires document image data, an electronic data acquisition unit that acquires electronic data of each of a plurality of printed materials based on the document image data, and a plurality of electronic a master image data selection unit that selects master image data from converted data; a determination unit that compares document image data with master image data to determine whether or not the master image data matches the document image data; An image collating system comprising: a collating unit that collates master image data determined by a determining unit to match document image data with digitized data of a plurality of printed materials.

According to the first aspect, it is determined whether or not the master image data matches the original image data of the printed matter when comparing the electronic data of the printed matter with any of the electronic data of the printed matter as the master image data. do. As a result, it is possible to collate the printed matter using the master image data that matches the document image data of the printed matter.

A printed matter represents a medium on which an image or the like is formed based on document image data. Images and the like may include photographs, pictures, patterns, graphics, characters, and the like.

Electronic data of printed matter represents a data format that can be subjected to prescribed processing using a processing device such as a computer.

A match can include not only an exact match, but also a substantial match that is not strictly a match but includes an acceptable margin of error.

According to a second aspect, in the image matching system of the first aspect, a display control section may be provided for displaying a master image representing master image data and a document image representing document image data on a display device.

According to the second aspect, it is possible to visually determine whether or not the master image displayed on the display device matches the document image.

The display controller can transmit a signal representing the master image data and a signal representing the document image data to the display device.

A third aspect is the image matching system of the second aspect, comprising a determination information acquisition unit for acquiring determination information indicating whether or not the master image displayed on the display device and the document image match, may be configured such that when the judgment information acquisition unit acquires judgment information indicating that the master image matches the document image, the master image data and the digitized data of a plurality of printed materials are collated.

According to the third aspect, when the determination information indicating that the master image matches the document image is acquired, the master image data and the digitized data of the printed matter are collated.

The determination information can be input by an operator using an input device.

A fourth aspect is the image matching system of the second aspect, wherein the determination unit determines step by step whether or not the master image data matches the document image data, and the display control unit performs A configuration may be adopted in which the display contents of the determination result displayed on the display device are changed.

According to the fourth aspect, it is possible to perform the display according to the stepwise determination result.

A fifth aspect is the image matching system of the fourth aspect, wherein the determination unit calculates a probability that the master image data matches the document image data, and the display control unit performs the following according to the probability calculated using the determination unit. A configuration may be adopted in which the display contents of the determination result displayed on the display device are changed.

According to the fifth aspect, display can be performed according to the probability that the master image data matches the document image data.

According to a sixth aspect, in the image collation system of the fifth aspect, the determination unit may determine that the master image data matches the document image data when the probability is equal to or greater than a specified threshold value.

According to the sixth aspect, whether or not the master image data matches the document image data can be determined based on the probability that the master image data matches the document image data.

A seventh aspect is the image matching system according to any one of the first to sixth aspects, further comprising a correction processing unit that performs correction processing on the document image data, and the determination unit includes the master image data and the correction processing unit. A configuration may also be adopted in which the document image data subjected to the correction processing is collated with the document image data.

According to the seventh aspect, image analysis can be performed between the master image data and the document image data.

An eighth aspect is the image matching system according to any one of the first to seventh aspects, further comprising a similar original image data acquisition unit for acquiring similar original image data similar to the original image data, wherein the determination unit is a master The image data and the similar document image data are compared to determine whether or not the master image data matches the similar document image data. Alternatively, the similar document image data and the master image data determined to be inconsistent may be collated with the electronic data of a plurality of printed materials.

According to the eighth aspect, it is possible to collate the master image data that does not match the similar document image data with the electronic data of the printed matter.

The similar document image data obtaining unit can obtain a plurality of similar document image data.

Obtaining similar document image data may include obtaining a plurality of candidate images and selecting one or more pieces of similar document image data from the plurality of candidate images. Acquisition of similar document image data may include a mode of acquiring predetermined similar document image data.

According to a ninth aspect, in the image collation system of the eighth aspect, when the similar document image data acquisition unit acquires a plurality of candidate image data that are candidates for similar document image data, for each of the plurality of candidate image data, a plurality of candidate image data based on the degree of similarity between each of the plurality of candidate image data calculated using the similarity calculation unit and the document image data; and a similar document image data selection unit that selects similar document image data from the similar document image data.

According to the ninth aspect, the processing range can be limited when identifying similar document image data. This is expected to shorten the processing period.

As an example of the degree of similarity, a difference value of average hash values between each of the plurality of candidate image data and the document image data can be applied.

A tenth aspect is the image matching system according to the eighth aspect, wherein the similar original image data obtaining unit selects similar original image data from a plurality of candidate image data that are candidates for similar original image data using an external computer. may be configured to acquire.

According to the tenth aspect, an external computer is used to perform similar document image data selection processing, thereby suppressing lengthening of the processing period until matching between similar document image data and master image data in an image matching system. can. In particular, it is effective when there are many candidate image data.

An eleventh aspect is the image matching system according to any one of the eighth to tenth aspects, wherein the determination unit determines step by step whether or not the master image data and the similar document image data match. good too.

According to the eleventh aspect, it can be determined step by step whether or not the master image data matches the similar document image data.

In the eleventh aspect, it is possible to determine whether or not the master image data matches the similar document image data by calculating the matching probability between the master image data and the similar document image data and comparing the matching probability with a prescribed threshold value. .

A twelfth aspect is the image matching system according to any one of the eighth to eleventh aspects, wherein a region similar to the similar document image data without matching is selected from the regions included in the document image data as the first similar region. a similar region detection unit for detecting, the determination unit detects a second similar region corresponding to the first similar region in the master image data, analyzes the first similar region and the second similar region, and detects the master image data may be configured to determine whether or not matches the document image data.

According to the twelfth aspect, it is possible to limit the processing range when determining whether or not the master image data and the document image data match. This is expected to shorten the processing period.

A thirteenth aspect is the image collation system of the twelfth aspect, wherein the determination unit determines whether or not the first similar region and the second similar region match as the degree of similarity between the similar original image data and the master image data. It is good also as a structure which judges objectively.

According to the thirteenth aspect, whether or not the first similarity region and the second similarity region match can be determined step by step.

In the thirteenth aspect, whether or not the master image data matches the document image data can be determined based on the probability that the first similar region and the second similar region match.

In the thirteenth aspect, it can be determined that the master image data matches the document image data when the probability that the first similar region and the second similar region match is equal to or greater than a prescribed threshold value.

A fourteenth aspect is the image matching system of the twelfth aspect or the thirteenth aspect, wherein the determination unit matches the first similar region and the second similar region, and matches the entire master image data and the entire document image data. may be used together.

According to the fourteenth aspect, it is possible to improve the accuracy of determining whether or not the master image data matches the document image data.

A fifteenth aspect is the image matching system according to any one of the twelfth to fourteenth aspects, wherein the matching unit matches the second similar region with a region in the electronic data corresponding to the second similar region. may be

According to the fifteenth aspect, the processing range can be limited when matching the master image data and the digitized data. This is expected to shorten the processing period.

A sixteenth aspect is the image matching system according to the fifteenth aspect, wherein the matching unit compares the second similar region with the region in the electronic data corresponding to the second similar region, and the entire master image data and the electronic A configuration may also be used in which matching with the whole of the converted data is also used.

According to the sixteenth aspect, it is possible to improve the accuracy of matching between the master image data and the digitized data.

A seventeenth aspect is the image matching system according to any one of the first aspect to the sixteenth aspect, wherein the master image data selection unit selects the digitized data of a first printed matter from among the plurality of printed matter as the master image data. Alternatively, the collating unit may be configured to collate the digitized data of the second and subsequent printed materials from among the plurality of printed materials with the master image data.

According to the seventeenth aspect, it is possible to collate the electronic data with the master image data without being affected by the change when the printed matter is electronically converted.

An image matching method according to an eighteenth aspect includes a document image data acquisition step of acquiring document image data, an electronic data acquisition step of acquiring electronic data of each of a plurality of printed materials based on the document image data, and a plurality of electronic a master image data selection step of selecting master image data from converted data; a determination step of comparing the document image data and the master image data to determine whether or not the master image data matches the document image data; and a matching step of matching the master image data determined to match the document image data in the determining step with the electronic data of the plurality of printed materials.

According to the eighteenth aspect, it is possible to obtain the same effect as the first aspect.

In the 18th aspect, the same matters as those specified in the 2nd to 17th aspects can be appropriately combined. In that case, the components responsible for the processes and functions specified in the image matching system can be grasped as the components of the image matching method responsible for the corresponding processes and functions.

A program according to a nineteenth aspect provides a computer with a document image data acquisition function for acquiring document image data, an digitized data acquisition function for acquiring digitized data of each of a plurality of printed materials based on the document image data, and a plurality of digitized A master image data selection function that selects master image data from data, a determination function that compares document image data and master image data to determine whether or not the master image data matches the document image data, and a determination function This is a program that implements a collation function of collating master image data determined to match document image data using , and digitized data of a plurality of printed materials.

According to the nineteenth aspect, it is possible to obtain the same effect as the first aspect.

In the 19th aspect, the same matters as those specified in the 2nd to 17th aspects can be appropriately combined. In that case, the components responsible for the processes and functions specified in the image matching system can be grasped as components of the program responsible for the corresponding processes and functions.

According to the present invention, it is determined whether or not the master image data matches the document image data of the printed matter when comparing the electronic data of the printed matter with any of the electronic data of the printed matter as the master image data. . As a result, it is possible to collate the printed matter using the master image data that matches the document image data of the printed matter.

FIG. 1 is an explanatory diagram of digitized data of printed matter. FIG. 2 is a schematic diagram of an image matching method according to the embodiment. FIG. 3 is a schematic diagram of a conventional image matching method. FIG. 4 is a schematic diagram of matching of master image data. FIG. 5 is an explanatory diagram of a configuration example of a matching screen. FIG. 6 is an explanatory diagram of a first example of the determination result display screen. FIG. 7 is an explanatory diagram of a second example of the determination result display screen. FIG. 8 is an explanatory diagram of a third example of the determination result display screen. FIG. 9 is a schematic diagram of matching of master image data according to the third embodiment. FIG. 10 is a schematic diagram showing an example of determination of similar document image data. FIG. 11 is a schematic diagram of matching of master image data according to the fourth embodiment. FIG. 12 is a diagram showing a specific example of similar document images. FIG. 13 is an enlarged view of the similar area. FIG. 14 is an overall configuration diagram of the image matching system. 15 is a block diagram showing the hardware configuration of the image processing apparatus shown in FIG. 14. FIG. 16 is a functional block diagram of the image matching system shown in FIG. 14. FIG. FIG. 17 is a flow chart showing the procedure of the image matching method according to the embodiment. FIG. 18 is a flow chart showing the procedure of the master image data matching step in the image matching method according to the first embodiment. FIG. 19 is a flow chart showing the procedure of the master image data matching step in the image matching method according to the second embodiment. FIG. 20 is a functional block diagram of an image matching system according to the third embodiment. FIG. 21 is a functional block diagram of an image matching system according to a modification of the third embodiment. FIG. 22 is a flow chart showing the procedure of the master image data matching process in the image matching method according to the third embodiment. FIG. 23 is a flow chart showing the procedure of the similar document image data obtaining process shown in FIG. FIG. 24 is a flow chart showing procedures in another example of the master image data matching process shown in FIG.

Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this specification, the same constituent elements are denoted by the same reference numerals, and overlapping descriptions are omitted as appropriate.

[Overview of image matching system]
FIG. 1 is an explanatory diagram of digitized data of printed matter. As shown in FIG. 1, when a plurality of prints 14 are produced using a printer 12 based on document image data 10A representing a document image 10, the plurality of prints 14 are correctly printed according to the document image data 10A. It is necessary to check whether or not

Conventionally, the manuscript image 10 and the printed matter 14 are visually checked, but there is a limit to obtaining a certain degree of accuracy, such as mistakes caused by operator's carelessness. Therefore, a method is conceivable in which the electronic data 18 is compared with the electronic data 18 by using the electronic device 16 to convert the printed matter 14 into the electronic data 18 .

Here, the branch number of the printed matter 14 and the branch number of the electronic data 18 shown in FIG. 1 indicate the number of the printed matter. n is an integer of 1 or more. An example of the computerized device 16 is an image scanning device. The image scanning device can image the printed matter 14 using an imaging device such as an image sensor and generate scan data of the printed matter. Scanned data of printed matter is an example of the electronic data 18 .

Examples of image sensors include CCD image sensors and CMOS. CCD is an abbreviation for Charge-Coupled Device. CMOS is an abbreviation for Complementary Metal Oxide Semiconductor.

FIG. 2 is a schematic diagram of an image matching method according to the embodiment. The image matching method shown in this embodiment automatically uses the first digitized data 18 - 1 generated in the process of digitizing the printed matter 14 continuously using the digitizing device 16 as the master image data 20 . Then, the computerized data 18 of the second and subsequent sheets are collated with the master image data 20 . The first digitized data 18-1 corresponds to an example of the digitized data of the first printed matter.

In general, it is easy to extract the first digitized data 18-1 when digitizing a plurality of printed materials 14 with the same order number continuously. Therefore, in this embodiment, the computerized data 18-1 of the first printed matter 14 is used as the master image data 20. FIG.

In this method, the digitized data 18 are collated with each other, so there is no need to correct changes in the image that occur in the process of digitization using the digitizing device 16, making it possible to develop the system at a very low cost. . Further, since the computerized data 18 are collated with each other, accuracy of collation can be improved. Furthermore, correction processing between the document image data 10A and the electronic data 18 becomes unnecessary, and the entire processing period can be shortened.

FIG. 3 is a schematic diagram of a conventional image matching method. As shown in FIG. 3, conventionally, a method of collating document image data 10A and electronic data 18 has been common. However, when the document image data 10A and the electronic data 18 are collated, technical difficulty is extremely high in order to ensure a certain accuracy of collation.

Therefore, in the image matching method according to the present embodiment, instead of the document image data 10A, the digitized data 18-1 of the first printed matter 14-1 is used as the master image data 20, and the master image data 20 and the second printed matter are The digitized data 18 of the printed matter 14 after 14-2 is collated.

FIG. 4 is a schematic diagram of matching of master image data. Hereinafter, collation of master image data means collation of the master image data 20 shown in FIG. 2 and the document image data 10A shown in FIG.

As shown in FIG. 4, the computerized data 18-1 used as the master image data 20 and the document image data 10A are collated to confirm whether or not the master image data 20 matches the document image data 10A. . As a result, all printed materials 14 can be collated with the document image 10 . In matching the master image data 20, the degree of similarity between the master image data 20 and the document image data 10A may be calculated. The probability that both match each other can be calculated as the degree of similarity. Verification of the master image data 20 will be described in detail below.

[Verification of Master Image Data According to First Embodiment]
FIG. 5 is an explanatory diagram of a configuration example of a matching screen. Visual observation by the operator is applied to the collation of the master image data 20 according to the first embodiment.

The display device 22 shown in FIG. 5 displays a master image data matching screen 23 . The master image data matching screen 23 displays the document image 10 and the master image 21 side by side. The operator can visually confirm the match or mismatch between the document image 10 and the master image 21 by viewing the master image data matching screen 23 displayed on the display device 22 .

The operator can input the verification result of the master image data 20 using an input device (not shown in FIG. 5). A keyboard, a mouse, etc. can be applied as the input device. A touch panel type display device 22 may be applied to serve as both the input device and the display device 22 . The input device is illustrated in FIG. 14 with reference numeral 105 attached.

As another example of collation of the master image data 20 using visual observation, the document image data 10A is printed using a printing device, and the printed matter of the document image data 10A and the master image 21 displayed using the display device 22 are visually observed. Examples include:

For example, a thumbnail image of the document image 10 is printed on a form used when managing a plurality of printed materials 14 by lot. When inspecting a plurality of printed materials 14 on a lot-by-lot basis, the operator visually checks the master image data 20 displayed on the master image data matching screen 23 and the thumbnail images of the document images 10 printed on the forms. can match.

[Function and Effect of Verification of Master Image Data According to First Embodiment]
According to the collation of master image data according to the first embodiment, it is possible to obtain the following effects.

[1]
In the process of collectively digitizing a plurality of printed materials 14 using the digitizing device 16, the first digitized data 18-1 is used as the master image data 20 instead of the document image data 10A. The master image data 20 and the computerized data 18 of the second and subsequent printed materials 14 are collated. As a result, there is no need to correct changes in the image that occur in the process of digitizing the printed matter 14 using the digitizing device 16, and collation of the printed matter can be achieved at low cost and with high collation accuracy. At that time, the master image data 20 is collated with the document image data 10A. As a result, all the printed matter 14 can be treated as having been collated with the document image data 10A. Similar effects can be obtained from other embodiments described below.

[2]
Verification of the master image data 20 applies visual inspection of the master image 21 corresponding to the master image data. As a result, it is possible to shorten the period of time spent for collating the master image data 20 . Moreover, collation of the master image data 20 can be realized using a display device and an input device.

[Verification of Master Image Data According to Second Embodiment]
Image analysis is applied to collation of the master image data 20 according to the second embodiment. When image analysis is applied to collation of master image data 20 , it is necessary to correct image changes that occur between printed matter 14 and computerized data 18 due to the performance of computerized equipment 16 .

Examples of image changes include color changes during digitization, print range changes due to the size of the printed matter 14, and inclinations when generating the digitized data 18. FIG. Therefore, the document image data 10A is subjected to correction processing such as color correction, alignment, and tilt correction, and then the master image data 20 is verified. This has the effect of increasing the accuracy of collation by visual inspection by the operator.

Instead of correcting the document image data 10A, the electronic data 18 may be corrected, or both the document image data 10A and the electronic data 18 may be corrected.

In collation of the master image data 20 shown in this embodiment, the degree of matching between the document image data 10A and the master image data 20 is calculated. The calculation result includes position information indicating which positions of the document image data 10A and the master image data 20 match. For example, the master image data 20 and the original image data 10A may be associated with each pixel or each area, and the degree of matching may be calculated for each associated pixel or area. A matching probability between the document image data 10A and the master image data 20 may be calculated as the degree of matching between the master image data 20 and the document image data 10A.

In the collation of the master image data 20 according to the present embodiment, when displaying the judgment result of the collation between the master image data 20 and the document image data 10A, The displayed content is changed.

FIG. 6 is an explanatory diagram of a first example of the display screen of the determination result. The first determination result display screen 30 shown in FIG. 6 is applied when it is determined that the master image data 20 matches the document image data 10A. The document image 10 and the master image 21 are displayed side by side on the first determination result display screen 30 shown in FIG. Also, the first determination result display screen 30 displays the first character information 32 representing the determination result and the OK button 24 .

The first character information 32 shown in FIG. 6 indicates that the master image data 20 matches the document image data 10A. Also, the first character information 32 includes the matching probability between the master image data 20 and the document image data 10A.

The document shown in FIG. 6 represents the document image 10 . The same applies to FIGS. 7 and 8. FIG. X, which represents the matching probability between the master image data 20 and the document image data 10A, represents an arbitrary numerical value exceeding 0% and less than 100%.

The display device 22 can display the first determination result display screen 30 when the matching probability X between the master image data 20 and the document image data 10A exceeds the first threshold. The display of the master image data 20 and the document image data 10A can be omitted from the first determination result display screen 30 showing that the master image data 20 matches the document image data 10A. Clicking the OK button 24 may transmit a signal representing the result of the determination in matching the master image data 20 . The same applies to the examples shown in FIGS.

FIG. 7 is an explanatory diagram of a second example of the determination result display screen. Second character information 32A representing the determination result is displayed on the second determination result display screen 30A shown in FIG. The second character information 32A can be applied when the probability that the master image data 20 matches the document image data 10A is relatively low. The match probability Y shown in FIG. 7 is less than the match probability X shown in FIG. That is, if the second threshold is less than the first threshold, the display device 22 displays the second determination result display screen 30A when the match probability Y is equal to or greater than the second threshold and less than the first threshold.

FIG. 8 is an explanatory diagram of a third example of the determination result display screen. Third character information 32B representing the determination result is displayed on the third determination result display screen 30B shown in FIG. The third character information 32B can be applied when the probability that the master image data 20 matches the document image data 10A is even lower. The matching probability Z shown in FIG. 8 is an arbitrary numerical value less than the matching probability Y shown in FIG. That is, when the match probability Z is less than the second threshold, the display device 22 displays the third determination result display screen 30B.

[Function and Effect of Matching Master Image Data According to Second Embodiment]
According to the collation of master image data according to the second embodiment, it is possible to obtain the following effects.

[1]
Image processing is applied to collation of the master image data 20, and the degree of matching between the master image data 20 and the document image data 10A is calculated. As a determination result, the degree of matching between the master image data 20 and the document image data 10A is displayed. As a result, visual verification of the master image data 20 can be assisted.

[2]
As the degree of matching, the probability that the master image data 20 matches the document image data 10A is applied. This enables determination based on the matching probability between the master image data 20 and the document image data 10A.

[3]
The content of character information representing the determination result is changed according to the degree of matching. This can assist the determination based on the matching probability between the master image data 20 and the document image data 10A.

[Application example of the second embodiment]
The degree of matching between the master image data 20 and the original image data 10A derived based on the image processing applied to collation of the master image data according to the second embodiment may be used as the determination result.

That is, when the degree of matching between the master image data 20 and the document image data 10A is equal to or greater than a prescribed threshold value, it is determined that the master image data 20 matches the document image data 10A. On the other hand, if the degree of matching between the master image data 20 and the document image data 10A is less than the specified threshold, it is determined that the master image data 20 does not match the document image data 10A.

The determination result can be displayed on the display device 22 shown in FIG. 5 and the like. Also, the determination result can be automatically transmitted.

[Verification of Master Image Data According to Third Embodiment]
FIG. 9 is a schematic diagram of matching of master image data according to the third embodiment. In the collation of the master image data 20 according to the third embodiment, the similarity is calculated by collating document image data that may be printed in error, and the similarity is used for collation of the master image data 20 .

Here, the document image data that may be erroneously printed is image data in a range where erroneously printed matter may be mixed with correctly printed matter. For example, document image data that may be printed erroneously corresponds to a plurality of document image data in a lot, which is a unit in which printed matter is printed continuously in the same period. In other words, document image data that may be printed by mistake is document image data that may be mistaken for regular document image data and that differs from regular document image data.

By limiting the range of document image data that may be printed erroneously, it is possible to prevent endless calculation of the degree of similarity between document image data that may be erroneously printed.

The degree of similarity between document image data that may be erroneously printed is calculated prior to collation between the master image data 20 and the document image data 10A. Whether or not similar document image data 40 exists is determined from the result of calculating the degree of similarity between document image data that may be erroneously printed. An example of the similar document image data 40 is document image data that has the highest degree of similarity with the document image data 10A among document image data that may be printed by mistake. A plurality of similar document image data 40 may exist. For example, the similar document image data 40 can be data whose degree of similarity with the document image data 10A exceeds a prescribed threshold.

When the similar document image data 40 exists, the master image data 20 and the similar document image data 40 are collated in addition to the collation between the master image data 20 and the document image data 10A. A matching degree such as matching probability may be applied as the matching result.

Table 1 below shows an example of the determination result in collation of the master image data 20 .

Any value greater than 0 percent and less than 100 percent may apply to the third threshold value shown in Table 1 above. For example, the third threshold may be 90 percent. Note that the determination threshold for the matching probability with the document image data 10A and the determination threshold for the matching probability with the similar document image data 40 may be set separately.

If the matching probability between the master image data 20 and the document image data 10A is equal to or higher than the third threshold and the matching probability between the master image data 20 and the similar document image data 40 is less than the third threshold, the master image data 20 is the document It can be determined that they match the image data 10A. That is, when the probability that the master image data matches the document image data 10A is high and the probability that the master image data matches the similar document image data 40 is low, it is determined that the master image data 20 matches the document image data 10A. can.

If the probability of matching between the master image data 20 and the document image data 10A is less than the third threshold, and the probability of matching between the master image data 20 and the similar document image data 40 is greater than or equal to the third threshold, the master image data 20 is the document It can be determined that there is a mismatch with the image data 10A.

That is, when the probability that the master image data matches the document image data 10A is low and the probability that the master image data matches the similar document image data 40 is high, the master image data 20 does not match the document image data 10A. can be determined.

If the probability of matching between the master image data 20 and the document image data 10A is greater than or equal to the third threshold, and the probability of matching between the master image data 20 and the similar document image data 40 is greater than or equal to the third threshold, determination can be made impossible.

That is, when the probability that the master image data 20 matches both the document image data 10A and the similar document image data 40 is high, determination may be impossible.

FIG. 10 is a schematic diagram showing an example of determination of similar document image data. FIG. 10 shows an example of applying an average hash value as an index representing similarity. In calculating the average hash value, the image data to be processed is reduced, converted to gray scale, and the average value of the pixel values is calculated. hash as

In the example shown in FIG. 10, the average hash value of the document image data 10A is abc123, the average hash value of the document image data 10B is abc124, and the average hash value of the document image data 10C is xgf328. In such a case, the document image data 10B having the closest average hash value to the document image data 10A can be determined as the similar document image data 40. FIG.

An example of the case where the similar document image data 40 exists is the case where two or more types of document images exist in the same lot. Postcards having the same pattern and different character information are examples of the case where two or more types of document images exist in the same lot. If the textual information is different, the presence or absence of textual information may be included.

Selection of the similar document image data 40 may be performed in an external computer of the image processing apparatus in which the document image data 10A and the like are managed, or a plurality of document image data are transferred to the image processing apparatus as described above. , may be implemented in an image processing device. The image processing apparatus here refers to the image processing apparatus 102 shown in FIG. 16, the image processing apparatus 102A shown in FIG. 20, the image processing apparatus 102B shown in FIG. 21, and the like. The external computer is shown in FIG. 21 with reference numeral 190 attached.

[Function and Effect of Verification of Master Image Data According to Third Embodiment]
According to the collation of master image data according to the third embodiment, it is possible to obtain the following effects.

[1]
Document image data that are likely to be erroneously printed are collated with each other to calculate the degree of similarity. Similar document image data 40 that is similar to document image data 10A is specified based on the calculated similarity. In collation of the master image data 20, collation of the master image data 20 and the similar original image data 40 is also used. This makes it possible to classify the comparison determination of the master image data 20 in more detail.

[2]
Apply the average hash value as similarity. As a result, the similar document image data 40 can be specified accurately and easily.

[Verification of Master Image Data According to Fourth Embodiment]
FIG. 11 is a schematic diagram of matching of master image data according to the fourth embodiment. In the matching of master image data according to the fourth embodiment, image analysis technology is used to detect the positions of similar regions between document image data, the similar regions are analyzed at high density, and a matching decision is made based on the analysis results. do.

Here, the similar area means a similar area that does not match between document image data. That is, the document image data 10A and the similar document image data 40 are different only in the similar area.

In addition, when matching the master image data 20 with the document image data 10A, the accuracy of matching of the master image data 20 can be improved by performing overall matching and analyzing the similar regions of both at high density.

First, the first similar region 50 is detected from the document image data 10A. Also, a fourth similar region 52 is detected from the similar document image data 40 . Known image analysis processing can be applied to detect the first similar region 50 and the fourth similar region 52 . For example, by comparing pixel values of corresponding pixels in the document image data 10A and the similar document image data 40, the first similar region 50 and the fourth similar region 52 can be detected. A plurality of first similar regions 50 and fourth similar regions 52 may be provided as long as they are the same in number.

When the master image data 20 is collated, the entire master image data 20 and the original image data 10A are collated. Also, the second similar region 53 in the master image data 20 is detected using the information on the position of the first similar region 50 in the document image data 10A.

The first similarity region 50 and the second similarity region 53 are analyzed at high density. A determination result of the master image data 20 is derived based on the result of the overall matching and the analysis result of the first similar region 50 and the second similar region 53 . Matching probability can be applied to the result of global matching and the result of analysis between similar regions. Table 2 below shows examples of determination results.

Any value greater than 0 percent and less than 100 percent may be applied to the fourth and fifth thresholds shown in Table 2 above. For example, take the fourth threshold as 90 percent. The fifth threshold may be less than 90 percent or greater than or equal to 90 percent. Note that the determination threshold for the matching probability in the overall matching and the determination threshold for the matching probability in the similar area matching may be set to the same value.

When overall matching and similar area matching are used together, if the matching probability of overall matching is equal to or higher than the fourth threshold, and if the matching probability of similar area matching is equal to or higher than the fifth threshold, the master image data 20 is the document image data 10A. It can be determined that they match. Note that similar region matching is an example of the above-described analysis of similar regions.

If the matching probability of overall matching is equal to or higher than the fourth threshold and the matching probability of similar area matching is equal to or lower than the fifth threshold, it can be determined that the master image data 20 does not match the document image data 10A. Further, when the matching probability of the overall matching is equal to or lower than the fourth threshold and the matching probability of the similar area matching is equal to or higher than the fifth threshold, it can be determined that the master image data 20 does not match the document image data 10A. Furthermore, when the matching probability of overall matching is equal to or less than the fourth threshold and the matching probability of similar region matching is equal to or less than the fifth threshold, it can be determined that the master image data 20 does not match the document image data 10A.

FIG. 12 is a diagram showing a specific example of similar document images. FIG. 13 is an enlarged view of the similar area. A document image 10 shown in FIG. 12 includes a first similar region 50 . A similar document image 41 corresponding to similar document image data 40 includes a fourth similar region 52 . The fourth similarity region 52 contains character information, whereas the first similarity region 50 does not contain character information.

Collation that analyzes similar regions at high density can also be applied to collation between the master image data 20 and the electronic data 18 . By comparing the second similarity region 53 of the master image data 20 and the third similarity region of the electronic data 18, it is possible to determine whether the master image data 20 and the electronic data 18 match. As a result, it is possible to improve the accuracy of matching and speed up processing when matching the master image data 20 and the digitized data 18 as well.

[Function and Effect of Matching Master Image Data According to Fourth Embodiment]
According to the collation of master image data according to the fourth embodiment, it is possible to obtain the following effects.

[1]
A combination of global matching and similar region matching is used. This makes it possible to improve the accuracy of matching and speed up processing.

[2]
Combined use of similar area matching is applied to matching between the master image data and the electronic data 18 . As a result, in matching between the master image data and the electronic data 18, it is possible to improve the accuracy of matching and speed up the processing.

In the present embodiment, an example is given in which both overall matching and similar region matching are used. However, only similar region matching is applied to ensure a certain degree of accuracy in matching processing and to further speed up matching processing. It is possible to

[Overall Configuration of Image Matching System]
FIG. 14 is an overall configuration diagram of the image matching system. The image matching system 100 includes an image processing device 102 , a display device 104 , a scanner device 106 and a barcode reader 108 .

The image processing device 102 checks the computerized data 18 of the printed matter 14 shown in FIG. The image processing device 102 acquires the electronic data 18 and checks the electronic data 18 using the master image data 20 . Also, the image processing apparatus 102 performs collation of the master image data 20 .

The display device 104 functions as the display device 22 shown in FIG. 5 and the like. The display device 104 displays screens used for collating the electronic data 18 of the printed matter 14, such as the master image data collation screen 23 shown in FIG. 5 and the first determination result display screen 30 shown in FIG.

An input device 105 is connected to the image processing device 102 . 14 shows a keyboard 105A and a mouse 105B as the input device 105. FIG. An operator can operate the input device 105 to transmit information such as selection information to the image processing device 102 .

A scanner device 106 generates electronic data 18 of the printed matter 14 . Electronic data 18 generated using the scanner device 106 is transmitted to the image processing device 102 . The scanner device 106 corresponds to an example of the electronic equipment 16 shown in FIG.

A barcode reader 108 reads a barcode written on a form attached to the printed matter 14 and transmits information represented by the barcode to the image processing apparatus 102 . An example of information represented by a barcode is the identification number of the printed matter 14 . Examples of the identification number of the printed material 14 include an order number assigned to a plurality of printed materials created using the same document image data, and a lot number representing a unit of continuous printing in the same period.

[Configuration of image processing device]
[Hardware configuration]
15 is a block diagram showing the hardware configuration of the image processing apparatus shown in FIG. 14. FIG. The image matching system 100 shown in FIG. 14 can implement various functions of the image matching system 100 by executing a specified program using the image processing device 102 shown in FIG.

The image processing apparatus 102 includes a control unit 120 , a memory 122 , a storage device 124 , a network controller 126 , a power supply device 128 , a display controller 130 , an input/output interface 132 and an input controller 134 . Note that I/O shown in FIG. 15 represents an input/output interface.

The control unit 120, the memory 122, the storage device 124, the network controller 126, the display controller 130, and the input/output interface 132 are connected via a bus 136 so as to be capable of data communication.

<Control part>
The control unit 120 functions as an overall control unit, various calculation units, and a storage control unit of the image processing apparatus 102 . The control unit 120 executes programs stored in ROM (read only memory) provided in the memory 122 .

The control unit 120 may download a program from an external storage device via the network controller 126 and execute the downloaded program. The external storage device may be communicably connected to the image processing apparatus 102 via the network 140 .

The control unit 120 uses a RAM (random access memory) provided in the memory 122 as a calculation area, and cooperates with various programs to execute various processes. Accordingly, various functions of the image matching system 100 are realized.

The control unit 120 controls reading data from the storage device 124 and writing data to the storage device 124 . The control unit 120 may acquire various data from an external storage device via the network controller 126 . The control unit 120 can execute various types of processing such as computation using the various types of acquired data.

The controller 120 may include one or two or more processors. Examples of processors include FPGAs (Field Programmable Gate Arrays) and PLDs (Programmable Logic Devices). FPGAs and PLDs are devices whose circuitry can be changed after manufacturing.

Another example of the processor is an ASIC (Application Specific Integrated Circuit). An ASIC comprises circuitry specifically designed to perform specific processing.

The controller 120 can apply two or more processors of the same type. For example, the controller 120 may use two or more FPGAs, or two PLDs. The controller 120 may employ two or more processors of different types. For example, the controller 120 may employ one or more FPGAs and one or more ASICs.

When a plurality of control units 120 are provided, the plurality of control units 120 may be configured using one processor. As an example of configuring a plurality of control units 120 with one processor, one processor is configured using a combination of one or more CPUs (Central Processing Units) and software, and this processor functions as a plurality of control units 120 There is a form to do. Note that software in this specification is synonymous with a program.

Another example of configuring the plurality of controllers 120 with a single processor is a form of using a processor that implements the functions of the entire system including the plurality of controllers 120 with a single IC chip. SoC (System On Chip) is a representative example of a processor that implements the functions of the entire system including a plurality of control units 120 with a single IC chip. Note that IC is an abbreviation for Integrated Circuit.

In this way, the control unit 120 is configured using one or more of various processors as a hardware structure.

<memory>
The memory 122 includes ROM (not shown) and RAM (not shown). The ROM stores various programs executed in the image matching system 100 . The ROM stores parameters, files, and the like used for executing various programs. The RAM functions as a temporary storage area for data, a work area for the control unit 120, and the like.

<Storage device>
The storage device 124 non-temporarily stores various data. The storage device 124 may be externally attached to the image processing device 102 . A large-capacity semiconductor memory device may be applied instead of or in combination with the storage device 124 .

<Network controller>
Network controller 126 controls data communication with external devices. Controlling data communications may include managing data communications traffic. The network 140 connected via the network controller 126 can apply a known network such as a LAN (Local Area Network).

<Power supply>
A large-capacity power supply such as a UPS (Uninterruptible Power Supply) is applied to the power supply 128 . The power supply device 128 supplies power to each unit of the image processing apparatus 102 when the commercial power supply is interrupted due to a power failure or the like.

<Display controller>
The display controller 130 functions as a display driver that controls the display device 104 based on command signals sent from the control unit 120 .

<Input/output interface>
The input/output interface 132 communicably connects the image processing apparatus 102 and an external device. The input/output interface 132 may apply a communication standard such as USB (Universal Serial Bus).

<Input controller>
The input controller 134 converts the format of the signal input using the input device 105 into a format suitable for processing by the image processing device 102 . Information input from the input device 105 via the input controller 134 is transmitted to each section via the control section 120 .

Note that the hardware configuration of the image processing apparatus 102 shown in FIG. 15 is an example, and additions, deletions, and changes can be made as appropriate.

[Functional Blocks of Image Matching System]
16 is a functional block diagram of the image matching system shown in FIG. 14. FIG. The image processing apparatus 102 includes an original image data acquisition section 160 , an electronic data acquisition section 162 , a master image data selection section 164 , a master image data collation section 166 , an electronic data collation section 168 and a display control section 170 . Although two input devices 105 are shown in FIG. 16, both of them correspond to the input device 105 shown in FIG.

<Original image data acquisition unit>
The document image data acquisition unit 160 acquires the identification number of the printed matter 14 read using the barcode reader 108 . The document image data acquisition unit 160 accesses the host computer 180 and acquires the document image data 10A of the printed matter 14 from the host computer 180 based on the identification number of the printed matter 14 .

The document image data acquiring section 160 transmits the document image data 10A of the printed matter 14 to the master image data collating section 166 . The document image data acquisition section 160 may store the document image data in a document image data storage section (not shown).

<Electronized data acquisition part>
The electronic data acquisition unit 162 acquires the electronic data 18 of the printed matter 14 from the scanner device 106 . The electronic data acquisition unit 162 transmits the acquired electronic data 18 to the master image data selection unit 164 . The electronic data acquisition unit 162 may store the electronic data in an electronic data storage unit (not shown).

<Master image data selection section>
A master image data selection unit 164 selects master image data 20 from a plurality of digitized data 18 . In this embodiment, the digitized data 18 of the first printed matter is selected as the master image data 20 .

The master image data 20 is not limited to the digitized data 18 of the first printed matter. The master image data selection unit 164 may select any one of the digitized data 18 of the second and subsequent printed materials 14 as the master image data 20 based on predetermined selection conditions for the master image data 20. .

For example, the digitized data 18 of the first to tenth printed matter 14 are acquired, the images corresponding to these are displayed on the display device 104, and each image corresponding to each digitized data 18 displayed on the display device 104 is displayed. , may be selected by the operator using the input device 105 .

<Master image data matching section>
The master image data matching unit 166 performs matching of the master image data 20 . The description of matching the master image data 20 is as described above. Here, description of collation of the master image data 20 is omitted. The master image data matching unit 166 in the embodiment is an example of a determination unit that determines whether or not the master image data matches the document image data.

The master image data collation unit 166 transmits the document image data 10A and the master image data 20 to the display control unit 170 in order to display the master image data collation screen 23 shown in FIG.

In addition, the master image data collating section 166 transmits the master image data 20 whose match with the document image data 10A has been confirmed to the digitized data collating section 168 for collation of the digitized data 18 .

Further, the master image data matching unit 166 acquires information indicating whether or not the master image data 20 transmitted from the input device 105 matches the document image data 10A. When master image data 20 matches document image data 10A, master image data selection section 164 transmits master image data 20 to master image data collation section 166 .

The master image data matching unit 166 in the embodiment is an example of a determination information acquiring unit that acquires determination information indicating whether or not the master image and the document image match. Also, the master image data matching unit 166 in the embodiment is an example of a determination unit that determines step by step whether or not the master image data matches the document image data.

The master image data matching section 166 may include an image processing section 167 . The image processing unit 167 corrects changes that occur when the electronic data 18 is generated from the printed matter 14 in the document image data 10A. The image processing unit 167 also calculates the matching probability between the master image data 20 and the corrected document image data 10A. The image processing unit 167 can perform various image processing related to image collation. The image processing unit 167 in the embodiment is an example of a correction processing unit that performs correction processing on document image data.

Master image data matching unit 166 causes display device 104 to display the matching probability between master image data 20 and document image data 10A via display control unit 170 .

<Electronized data matching unit>
The computerized data collation unit 168 performs collation of the computerized data 18 using the master image data 20 that has been confirmed to match the document image data 10A. The computerized data collation unit 168 may collate the computerized data 18 using the information transmitted from the input device 105 .

The computerized data collation unit 168 can transmit the collation result of the computerized data 18 to the display control unit 170 and cause the display device 104 to display the collation result of the computerized data 18 . For example, if electronic data 18 does not match master image data 20, display device 104 may display an error message.

<Display control part>
The display control unit 170 controls the display device 104 . The display controller 170 includes the display controller 130 shown in FIG. The display control unit 170 causes the display device 104 to display the information acquired from the master image data collation unit 166 and the electronic data collation unit 168 . The display control unit 170 in the embodiment is an example of a display control unit that changes the display content of the determination result to be displayed on the display device according to the stage of determination.

[Procedure of image matching method]
FIG. 17 is a flow chart showing the procedure of the image matching method according to the embodiment. The image collation method shown in this embodiment includes a document image data acquisition step S10, an electronic data acquisition step S12, a master image data selection step S14, a master image data collation step S16, a match determination step S18, an electronic data collation step S20, It includes a print number determination step S22 and an excess number determination step S24.

In the document image data acquisition step S10, the document image data acquisition unit 160 shown in FIG. Get order quantity.

In the document image data acquisition step S10, the document image data acquisition unit 160 reads the document image data 10A from the host computer 180 using the order number of the printed matter 14 as an index. The document image data acquisition step S10 may include a document image data storage step of storing the document image data 10A read from the host computer 180 in a document image data storage unit (not shown).

Further, in the document image data acquisition step S10, the document image data acquisition section 160 transmits the document image data 10A to the master image data collation section 166. FIG.

In the digitized data acquisition step S12, the digitized data acquisition unit 162 acquires the digitized data 18 of the printed matter 14 read using the scanner device 106 and the number of digitized data 18 . The computerized data acquisition step S<b>12 may include a printed matter setting step in which the operator sets the printed matter 14 on the scanner device 106 .

The digitized data acquisition step S12 may include a printed matter reading step of reading the printed matter 14 using the scanner device 106 and an digitized data transmission step of transmitting the digitized data 18 of the printed matter 14 from the scanner device 106. Note that the document image data acquisition step S10 and the electronic data acquisition step S12 may be reversed in order, or both may be performed in parallel.

In the master image data selection step S14, the master image data selection unit 164 selects the master image data 20 from the electronic data 18 acquired in the electronic data acquisition step S12. Further, in the master image data selection step S<b>14 , the master image data selection section 164 transmits the master image data 20 to the master image data collation section 166 .

In the master image data collation step S16, the master image data collation unit 166 collates the master image data 20. FIG. In the match determination step S18, it is determined whether or not the master image data 20 matches the document image data 10A based on the comparison of the master image data 20. FIG.

If the master image data 20 does not match the document image data 10A, the determination is No. If the determination is No, the process proceeds to the excess number determination step S24. In the case of No determination, the master image data matching unit 166 may cause the display device 104 to display character information indicating that the master image data 20 does not match the document image data 10A.

On the other hand, if the master image data 20 matches the document image data 10A, the determination is Yes. In the case of Yes determination, the master image data collating unit 166 transmits the master image data 20 to the electronic data collating unit 168 . In the case of Yes judgment, it progresses to computerized data collation process S20.

In the computerized data collation step S20, the computerized data collation unit 168 collates each of the plurality of computerized data 18 included in the same order number with the master image data 20. FIG. Known image processing can be applied to collation between the computerized data 18 and the master image data 20 . For example, there is a process of dividing the digitized data 18 and the master image data 20 into a plurality of areas and comparing feature amounts for each area.

In the computerized data matching step S20, when the computerized data 18 that does not match the master image data 20 is found, the display device 104 may display a warning screen indicating that there is a rejected product. In the computerized data collating step S20, collation between the computerized data 18 and the master image data 20 may be stopped when the computerized data 18 that does not match the master image data 20 is found.

In the number-of-prints determination step S22, the computerized data collating unit 168 compares the number of computerized data 18 collated with the master image data 20 and the ordered number of sheets. If the number of digitized data 18 collated with the master image data 20 is less than the ordered number, the determination is No. In the case of a No determination, the computerized data collation step S20 and the print number determination step S22 are repeatedly performed until a Yes determination is made in the print number determination step S22.

On the other hand, if the number of electronic data 18 collated with the master image data 20 is greater than or equal to the ordered number in the print number determination step S22, the determination is Yes. In the case of Yes determination, the process proceeds to the excess number determination step S24. In the number excess determination step S24, the electronic data collating unit 168 determines whether or not the number of electronic data 18 acquired in the electronic data acquisition step S12 matches the ordered number.

In the excess number determination step S24, if the acquired number of electronic data 18 does not match the ordered number and the acquired number of electronic data 18 exceeds the ordered number, a No determination is made. If the determination is No, the process proceeds to error notification step S26.

In the error notification step S26, the computerized data matching unit 168 notifies an error indicating that the acquired number of computerized data 18 exceeds the ordered number. Examples of error notification include a mode of displaying character information on the display device 104 and a mode of generating sound from a speaker (not shown). After the error notification step S26, the image matching method is terminated.

On the other hand, in the number excess determination step S24, if the acquired number of electronic data 18 matches the ordered number, the determination is Yes. If the determination is Yes, the image matching method is terminated. Note that the excess number determination step S24 and the error notification step S26 may be performed as steps before the computerized data collation step S20 is performed.

In the number-of-prints determination step S22, all the electronic data 18 to be compared are compared with the master image data 20, and all the electronic data 18 to be compared match the master image data 20, and the number-of-prints determination step S24. , the display device 104 may display information such as that all products are accepted when the acquired number of electronic data 18 matches the ordered number.

[Procedure of Master Image Data Matching Step in Image Matching Method According to First Embodiment]
FIG. 18 is a flow chart showing the procedure of the master image data matching step in the image matching method according to the first embodiment. In the image collation method according to the first embodiment, the master image data collation step S16 shown in FIG. 17 includes the display step S100 and the input information acquisition determination step S102.

In the display step S100, the master image data matching unit 166 causes the display device 104 to display the master image data 20 and the document image data 10A side by side via the display control unit 170. FIG. The display device 104 displays the master image data matching screen 23 shown in FIG. The operator can use the input device 105 to input information indicating whether or not the master image data 20 displayed on the master image data matching screen 23 matches the document image data 10A.

Further, the operator may collate the thumbnail image of the printed matter 14 printed on the form with the master image data 20 displayed on the master image data collation screen 23, and input information indicating whether or not the two match. .

In the input information acquisition determination step S102, the master image data matching unit 166 determines whether or not information indicating whether or not the master image data 20 matches the document image data 10A has been input from the input device 105 or not. In the input information acquisition determination step S102, if the master image data matching unit 166 has not acquired the input information from the input device 105, the determination is No. In the case of No determination, the input information acquisition determination step S102 is continued until a Yes determination is made in the input information acquisition determination step S102.

On the other hand, in the input information acquisition determination step S102, when the master image data matching unit 166 acquires the input information from the input device 105, the determination is Yes. In the case of Yes determination, the master image data matching step S16 is ended, and the process proceeds to the matching determination step S18 shown in FIG.

In the match determination step S18, the master image data matching unit 166 determines whether or not the master image data 20 matches the document image data 10A based on the input information acquired from the input device 105. FIG.

[Procedure of Master Image Data Matching Step in Image Matching Method According to Second Embodiment]
FIG. 19 is a flow chart showing the procedure of the master image data matching step in the image matching method according to the second embodiment. When the matching probability is applied to the matching in the master image data matching step S16, the master image data matching step S16 shown in FIG. 17 includes the image processing step S120 shown in FIG. An acquisition determination step S126 is included.

In the image processing step S120, the image processing unit 167 performs a process of correcting changes that occur when creating the electronic data 18 that will become the master image data 20 for the document image data 10A.

Correction processing may include at least one of color correction, change of print range, and correction of tilt. It should be noted that the image processing step S120 can be omitted if the changes that occur when creating the electronic data 18 can be ignored.

In the match probability calculation step S122, the image processing unit 167 calculates the match probability between the master image data 20 and the document image data 10A. The calculation of the matching probability between the master image data 20 and the document image data 10A is as described above. Description here is omitted.

In the display step S124, the master image data matching unit 166 displays the first determination result display screen 30 shown in FIG. 6, the second determination result display screen 30A shown in FIG. 7, and the third determination result display screen 30B shown in FIG. Either one is displayed on the display device 104 .

The input information acquisition determination step S126 is the same as the input information acquisition determination step S102 shown in FIG. Description here is omitted.

[Image Matching System According to Third Embodiment]
<Explanation of function block>
FIG. 20 is a functional block diagram of an image matching system according to the third embodiment. The image processing apparatus 102A in the image matching system 100A shown in FIG. 20 has a similar document image data acquisition unit 172 added to the image processing apparatus 102 shown in FIG.

The similar document image data obtaining unit 172 obtains the similar document image data 40 shown in FIG. As an example of acquiring the similar document image data 40, a plurality of candidate image data are acquired, and one or more similar document image data 40 are selected from among them using the similar document image data selection unit 174. be done.

Another example of acquisition of the similar document image data 40 is acquisition of preselected similar document image data 40 from the host computer 180 based on the identification information of the printed matter 14 . Note that the similar document image data selection unit 174 can be omitted.

In such a case, a computer external to the image processing apparatus 102A calculates the degree of similarity of the candidate image data for each lot of printed matter, and selects the similar document image data 40 based on the degree of similarity. Stored in computer 180 .

Similar document image data obtaining portion 172 transmits similar document image data 40 to image processing portion 167 . The image processing unit 167 uses the similar document image data 40 when matching the master image data 20 .

<Modification>
FIG. 21 is a functional block diagram of an image matching system according to a modification of the third embodiment. The image matching system 100B shown in FIG. 21 includes an image processing device 102B. The image processing apparatus 102B does not include the similar document image data selection unit 174 shown in FIG.

That is, the external computer 190 includes a similar document image data selection section 196 corresponding to the similar document image data selection section 174 shown in FIG. External computer 190 acquires candidate image data of similar document image data 40 from host computer 180 via data input/output unit 192 .

The external computer 190 uses the similarity calculator 194 to calculate the similarity of the candidate image data. External computer 190 selects similar document image data 40 using similar document image data selection unit 196 . External computer 190 uses data input/output unit 192 to transmit similar document image data 40 or information necessary for specifying similar document image data 40 stored in host computer 180 to host computer 180 .

The image processing apparatus 102</b>B acquires the similar document image data 40 from the host computer 180 via the similar document image data acquiring section 172 .

When there is a large number of candidate image data, it takes a long time to calculate the degree of similarity and select similar document image data. Therefore, the external computer 190 is used to calculate the degree of similarity and select similar document image data, and the image processing apparatus 102 acquires the similar document image data. This enables the image processing apparatus 102 to collate the similar document image data 40 and the master image data 20 immediately after obtaining the similar document image data 40 .

<Procedure of master image data matching process>
FIG. 22 is a flow chart showing the procedure of the master image data matching step in the image matching method according to the second embodiment. The master image data matching process shown in FIG. 17 includes a similar document image data acquisition process S140, a document image data match probability calculation process S142, a similar document image data match probability calculation process S144, a determination process S146, and a determination result display shown in FIG. including step S148.

In the similar document image data obtaining step S140, the similar document image data obtaining unit 172 shown in FIG. In the document image data matching probability calculation step S142, the image processing unit 167 calculates the matching probability between the master image data 20 and the document image data 10A.

In the similar document image data matching probability calculation step S144, the image processing unit 167 calculates the matching probability between the master image data 20 and the similar document image data 40. FIG. Note that the document image data match probability calculation step S142 and the similar document image data match probability calculation step S144 may be switched in order, or both may be performed in parallel.

In the determination step S146, the master image data matching unit 166 determines whether the master image data 20 matches the document image data 10A and whether the master image data 20 matches the similar document image data 40. FIG. Examples of determination in determination step S146 are as shown in Table 1 above.

In the determination result display step S148, the master image data matching unit 166 causes the display device 104 to display the determination result in the determination step S146. In the master image data collation process shown in FIG. 22, the master image data collation unit 166 can advance the procedure to the coincidence judgment process S18 shown in FIG.

<Procedure for obtaining similar original image data>
FIG. 23 is a flow chart showing an example of the similar document image data obtaining process shown in FIG. The flowchart shown in FIG. 23 can be applied to acquisition of similar original image data according to the third embodiment and the fourth embodiment. The similar document image data acquisition step S140 shown in FIG. 22 includes a candidate image data acquisition step S160, a similarity degree calculation step S162, and a similar document image data setting step S164 shown in FIG.

In the candidate image data obtaining step S160, the similar original image data obtaining unit 172 obtains candidate image data as candidates for the similar original image data 40. FIG. In candidate image data acquisition step S160, document image data acquisition section 160 can acquire a plurality of candidate image data.

In the similarity calculation step S162, the similar document image data selection unit 174 calculates the similarity of the candidate image data. As an example of the degree of similarity, a difference value between average hash values obtained by subtracting the average hash value of the candidate image data from the average hash value of the original image data can be applied. The similar document image data selection unit 174 in the embodiment is an example of a similarity calculation unit that calculates the degree of similarity between each of a plurality of candidate image data and the document image data. Also, the similar document image data selection unit 174 in the embodiment is an example of a component of the similar document image data acquisition unit.

That is, the similarity calculation step S162 can include an average hash value calculation step of calculating the average hash value of the document image data and the average hash value of the candidate image data. Further, the similarity calculation step S162 may include a difference calculation step of calculating a difference value between average hash values obtained by subtracting the average hash value of the candidate image data from the average hash value of the document image data.

In similar document image data setting step S164, similar document image data selection unit 174 selects, as similar document image data 40, one or more candidate image data whose degree of similarity with document image data 10A exceeds a prescribed threshold. If there is only one candidate image data, the candidate image data can be selected as the similar document image data 40 in the similar document image data setting step S164 without executing the similarity degree calculation step S162.

After the similar document image data 40 is selected in the similar document image data setting step S164, the similar document image data obtaining step ends. In the similar document image data setting step S164, if the degree of similarity of all the candidate images with the document image data 10A is equal to or less than a prescribed threshold, the similar document image data selection unit 174 selects the similar document image data 40 without selecting the similar document image data 40. The document image data setting step S164 is ended. If the similar original image data 40 is not set, the master image data collating section 166 performs the master image data collating step S16 shown in FIG. 17 without performing the master image data collating step shown in FIG.

When similar document image data 40 is stored in advance in host computer 180, candidate image data acquisition step S160, similarity degree calculation step S162, and similar document image data setting step S164 are performed in an external computer. In such a case, in the similar document image data acquisition step S140 shown in FIG.

<Procedure in another example of the master image data matching process>
FIG. 24 is a flow chart showing procedures in another example of the master image data matching process shown in FIG. The flowchart shown in FIG. 24 is applied to the image matching method according to the fourth embodiment.

The master image data matching step S16 shown in FIG. 17 includes a similar original image data obtaining step S180, a similar region detecting step S182, an overall matching step S184, a similar region matching step S186, and a determining step S188 shown in FIG.

In the similar document image data obtaining step S180, the similar document image data obtaining unit 172 obtains the similar document image data 40. FIG. The procedure shown in FIG. 23 can be applied to the similar document image data acquisition step S180.

In the similar area detection step S182, the master image data matching unit 166 detects the first similar area 50 from the document image data 10A, and uses the position information of the first similar area 50 in the document image data 10A to extract the first similar area from the master image data 20. A second similar region 53 is detected.

In the overall matching step S184, the master image data matching unit 166 performs overall matching between the master image data 20 and the document image data 10A. In the similar area matching step S186, the master image data matching unit 166 detects the second similar area 53 from the master image data 20 and performs matching between the first similar area 50 and the second similar area 53. FIG.

In the determination step S188, the master image data matching unit 166 matches the master image data 20 with the document image data 10A based on the result of the overall matching in the overall matching step S184 and the result of the similar region matching in the similar region matching step S186. Derivation of the determination result indicating whether or not

In the determination step S188, it can be determined step by step whether or not the master image data 20 and the document image data 10A match. Further, in the determination step S188, it can be determined step by step whether or not the first similar region 50 is similar to the second similar region 53 . As an example of the stepwise determination, there is a determination using a predetermined threshold after calculating the matching probability of the matching object. The master image data matching unit 166 in the embodiment is an example of a determining unit that determines step by step whether or not the first similar region and the second similar region match.

The detection of the first similar region 50 in the similar region detection step S182 and the detection of the second similar region 53 in the similar region matching step S186 may be performed using the image processing unit 167. FIG. The master image data matching unit 166 in the embodiment is an example of a similar area detection unit.

[Example of application to a program]
The function of each part of the image matching system 100 and the function of each step of the image matching method described above can be realized by executing a program using a computer. For example, the computer may have a document image data acquisition function for acquiring the document image data 10A, an electronic data acquisition function for acquiring electronic data of a plurality of printed materials based on the document image data, and a master image data selection from a plurality of electronic data. a master image data selection function that compares the document image data with the master image data to determine whether or not the master image data matches the document image data; It is possible to configure a program that realizes a collation function for collating the master image data that has been processed and a plurality of electronic data.

[Combination of Embodiments]
The first to fourth embodiments described above can be combined as appropriate.

[Terminology]
A match may include a substantial match with a range of errors that may be considered a match, but not an exact match.

Image data means electronic data representing an image. Electronic data means a form of information that can be processed using a computer or the like. An image represents an object represented on printed matter or displayed using a display device. Images may include characters, graphics, patterns, and the like. It should be noted that in this specification, the terms "image" and "image data" may be used interchangeably.

Acquisition of data may include not only a mode of acquiring data via a data input section but also a mode of reading data stored in a storage section.

In the embodiments of the present invention described above, it is possible to appropriately change, add, or delete constituent elements without departing from the gist of the present invention. The present invention is not limited to the embodiments described above, and many modifications are possible within the technical concept of the present invention by those having ordinary knowledge in the field.

10 Original image 10A Original image data 10B Original image data 10C Original image data 12 Printer 14 Printed matter 16 Computerized equipment 18 Computerized data 20 Master image data 21 Master image 22 Display device 23 Master image data verification screen 24 OK button 30 First Judgment result display screen 30A Second judgment result display screen 30B Third judgment result display screen 40 Similar document image data 41 Similar document image 50 First similar region 52 Fourth similar region 53 Second similar region 100 Image matching system 100A Image matching system 100B image matching system 102 image processing device 102A image processing device 102B image processing device 104 display device 105 input device 105A keyboard 105B mouse 106 scanner device 108 barcode reader 120 control unit 122 memory 124 storage device 126 network controller 128 power supply device 130 display controller 132 input/output interface 134 input controller 136 bus 140 network 160 original image data acquisition unit 162 digitized data acquisition unit 164 master image data selection unit 166 master image data collation unit 167 image processing unit 168 digitized data collation unit 170 display control unit 180 Host computer 190 External computer 192 Data input/output unit 194 Similarity calculation unit 196 Similar original image data selection unit S10 to S188 Each step of the image matching method

Claims (34)

a document image data acquisition unit for acquiring document image data;
an digitized data acquisition unit that acquires digitized data of each of a plurality of printed materials based on the document image data;
a master image data selection unit that selects master image data from the plurality of digitized data;
a determination unit that compares the document image data and the master image data and determines whether or not the master image data matches the document image data;
a collation unit for collating the master image data determined by the determination unit to match the document image data with the electronic data of the plurality of printed materials;
a similar document image data acquisition unit that acquires similar document image data similar to the document image data;
with
The determination unit compares the master image data with the similar document image data to determine whether or not the master image data matches the similar document image data,
The matching unit uses the determination unit to match the master image data determined to match the document image data and not match the similar document image data with the electronic data of the plurality of printed materials. image matching system. 2. The image collation system according to claim 1, further comprising a display control section for displaying a master image representing said master image data and a document image representing said document image data on a display device. a determination information acquisition unit configured to acquire determination information indicating whether or not the master image displayed on the display device and the document image match;
When the verification unit acquires determination information indicating that the master image matches the document image using the determination information acquisition unit, the collation unit compares the master image data with the electronic data of the plurality of printed materials. 3. The image collating system according to claim 2, which collates the . The determination unit determines step by step whether the master image data matches the document image data,
3. The image matching system according to claim 2, wherein the display control unit changes display contents of the determination result displayed on the display device according to the stage of the determination. The determination unit calculates a probability that the master image data matches the document image data,
5. The image matching system according to claim 4, wherein the display control section changes display contents of the determination result to be displayed on the display device according to the probability calculated using the determination section. 6. The image collation system according to claim 5, wherein the determination unit determines that the master image data matches the document image data when the probability is equal to or greater than a specified threshold. a correction processing unit that performs correction processing on the document image data;
7. The image collation system according to claim 1, wherein the determination unit collates the master image data with the document image data corrected by the correction processing unit. When the similar document image data acquiring unit acquires a plurality of candidate image data that are candidates for the similar document image data, the degree of similarity between each of the plurality of candidate image data and the document image data is calculated. a similarity calculation unit;
a similar document image for selecting the similar document image data from the plurality of candidate image data based on the degree of similarity between each of the plurality of candidate image data and the document image data calculated using the similarity calculation unit; a data selection unit;
The image matching system according to any one of claims 1 to 7, comprising: 8. The similar document image data acquiring unit acquires similar document image data selected from a plurality of candidate image data as candidates for the similar document image data using an external computer . 1. The image matching system according to item 1 . 10. The image matching system according to claim 1 , wherein the determination unit determines step by step whether or not the master image data and the similar document image data match. a similar region detection unit that detects, as a first similar region, a region that is similar to the similar document image data without matching the similar document image data from the regions included in the document image data;
The determination unit detects a second similar region corresponding to the first similar region in the master image data, analyzes the first similar region and the second similar region, and determines whether the master image data is the document. 11. The image matching system according to any one of claims 1 to 10 , wherein it is determined whether or not the image data matches. 12. The determining unit according to claim 11 , wherein the determining unit determines step by step whether or not the first similar region and the second similar region match as the degree of similarity between the similar document image data and the master image data. image matching system. 13. The image matching according to claim 11 or 12 , wherein the determination unit performs both matching between the first similar region and the second similar region and matching between the entire master image data and the entire document image data. system. 14. The image collating system according to any one of claims 11 to 13 , wherein the collating unit collates the second similar area with an area in the electronic data corresponding to the second similar area. The collation unit collates the second similar area with an area in the electronic data corresponding to the second similar area, and collates the entire master image data with the entire electronic data of the printed matter. 15. The image matching system according to claim 14 , which is used together. The master image data selection unit selects digitized data of a first printed matter from among the plurality of printed matter as the master image data,
16. The image matching system according to any one of claims 1 to 15 , wherein the matching unit compares computerized data of second and subsequent printed materials among the plurality of printed materials with the master image data. a manuscript image data obtaining step of obtaining manuscript image data;
an electronic data acquisition step of acquiring electronic data of each of a plurality of printed materials based on the document image data;
a master image data selection step of selecting master image data from the plurality of digitized data;
a determination step of comparing the document image data and the master image data to determine whether or not the master image data matches the document image data;
a collation step of collating the master image data determined to match the document image data in the determination step with the electronic data of the plurality of printed materials;
a similar document image data acquiring step of acquiring similar document image data similar to the document image data;
including
The determining step compares the master image data with the similar document image data to determine whether or not the master image data matches the similar document image data,
In the collation step, the image collation is collated with the digitized data of the plurality of printed materials with the master image data determined to match the document image data and not match the similar document image data in the determination step. Method. In the similar document image data acquiring step, when a plurality of candidate image data as candidates for the similar document image data are acquired, a degree of similarity between each of the plurality of candidate image data and the document image data is calculated. a similarity calculation step;
Similar document image data selection for selecting the similar document image data from the plurality of candidate image data based on the degree of similarity with the document image data in each of the plurality of candidate image data calculated in the similarity calculation step process and
18. The image matching method according to claim 17, comprising: 18. The image collation according to claim 17, wherein said similar document image data acquisition step acquires similar document image data selected from a plurality of candidate image data as candidates for said similar document image data using an external computer. Method. 20. The image collating method according to claim 17, wherein in the determining step, whether or not the master image data and the similar document image data match is determined step by step. a similar region detection step of detecting, as a first similar region, a region similar to the similar document image data without matching the similar document image data from the regions included in the document image data;
The determination step detects a second similar region corresponding to the first similar region in the master image data, analyzes the first similar region and the second similar region, and determines whether the master image data corresponds to the document. 21. The image matching method according to any one of claims 17 to 20, wherein it is determined whether or not the image data matches. 22. The method according to claim 21, wherein the determining step determines step by step whether or not the first similar region and the second similar region match as the degree of similarity between the similar document image data and the master image data. image matching method. 23. The image matching according to claim 21 or 22, wherein the determining step uses both matching between the first similar region and the second similar region and matching between the entire master image data and the entire document image data. Method. 24. The image collating method according to any one of claims 21 to 23, wherein said collating step collates said second similar area with an area in said electronic data corresponding to said second similar area. The matching step includes matching the second similar region with a region in the electronic data corresponding to the second similar region, and matching the entire master image data with the entire electronic data of the printed matter. 25. The image matching method according to claim 24, which is used in combination. to the computer,
document image data acquisition function for acquiring document image data;
an electronic data acquisition function for acquiring electronic data of each of a plurality of printed materials based on the document image data;
a master image data selection function for selecting master image data from the plurality of digitized data;
a determination function of comparing the document image data and the master image data to determine whether or not the master image data matches the document image data ;
A collation function for collating the master image data determined to match the document image data using the determination function and the electronic data of the plurality of printed materials;
A program for realizing a similar document image data acquisition function for acquiring similar document image data similar to the document image data ,
The determination function compares the master image data with the similar document image data to determine whether the master image data matches the similar document image data,
The matching function uses the determination function to match the master image data determined to match the document image data and not match the similar document image data with the electronic data of the plurality of printed materials. program. to the computer,
When a plurality of candidate image data that are candidates for the similar document image data are obtained using the similar document image data obtaining function, the degree of similarity between each of the plurality of candidate image data and the document image data is calculated. a similarity calculation function to calculate, and
Similar document image for selecting the similar document image data from the plurality of candidate image data based on the degree of similarity with the document image data in each of the plurality of candidate image data calculated using the similarity calculation function. 27. The program according to claim 26, which implements a data selection function. 27. The program according to claim 26, wherein the similar original image data acquisition function acquires similar original image data selected from a plurality of candidate image data as candidates for the similar original image data using an external computer. 29. The program according to any one of claims 26 to 28, wherein the determination function determines step by step whether the master image data and the similar document image data match. to the computer,
realizing a similar area detection function for detecting, as a first similar area, an area similar to the similar document image data without matching the similar document image data from the areas included in the document image data;
The determination function detects a second similar region corresponding to the first similar region in the master image data, analyzes the first similar region and the second similar region, and determines whether the master image data is the document. 30. The program according to any one of claims 26 to 29, which determines whether or not the image data matches. 31. The method according to claim 30, wherein the determination function determines step by step whether or not the first similar region and the second similar region match as the degree of similarity between the similar document image data and the master image data. program. 32. The program according to claim 30 or 31, wherein the determination function uses both matching between the first similar region and the second similar region and matching between the entire master image data and the entire document image data. 33. The program according to any one of claims 30 to 32, wherein the collation function collates the second similarity area with an area in the electronic data corresponding to the second similarity area. The matching function performs matching between the second similar region and the region in the electronic data corresponding to the second similar region, and matching between the entire master image data and the entire electronic data of the printed matter. 34. The program according to claim 33, used together.

Images