JP2022011037A - Image processing device, image processing method and program - Google Patents

Abstract

To solve a problem that, in the case where a plurality of document images having a data structure similar to the query document is registered in a registered document image, a correct registered document cannot be selected when a similar form is selected by the matching process, and as a result, an erroneous value may be extracted.SOLUTION: A plurality of candidates of a similar form is detected by matching processing, a region in which the comparison is easily performed between similar slip candidates is selected, and a correct value is made possible to be extracted by performing a determination process using the data format of the value attribute to allow you to select the correct registration document.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image processing apparatus, an image processing method and a program.

In recent years, due to changes in the working environment due to the spread of computers, the digitization of documents handled in business is progressing. For example, many companies, including companies that perform ordering and application screening, are working on digitizing various forms such as invoices, quotations, purchase orders, and application forms. In such work, it is specified which form document corresponds to from a large amount of form documents to be handled, and processing is performed according to the workflow for each form document. For example, the workflow includes data registration work. Specifically, the information of the determined data items (form name, form number, issuer company information, issue date, billing details, etc.) is extracted from the specified form documents, and the accounting system and expense settlement are performed. Register data in a predetermined business system such as a system. The person in charge of this data registration work visually confirms the description contents of these data items described in various form documents and manually inputs the description contents into the predetermined business system, so that the business efficiency Is declining. Therefore, in order to improve the efficiency of data registration work, efforts are being made to automate the work. Specifically, the form recognition process for scanning and digitizing the obtained form documents to specify the form type and the optical character recognition (hereinafter referred to as OCR) process for optically recognizing character information are applied. By doing so, the data items and description contents in the form are automatically extracted.

As a method for specifying the form type, there is a method of Patent Document 1. In this method, first, the features of the form format (ruled line information and image feature information of character strings) and confirmation information (position information in which item values are entered, etc.) are registered in advance. When estimating the scanned form, the form format of the document to be processed is specified based on the characteristics of the registered form format. Then, the form is identified by collating with the confirmation information based on the specified form format, and the item value is extracted. If the collation fails, an error is presented. As a result, the user workload can be greatly reduced by automatically extracting the corresponding item value. Further, even when another form is input in which the form format is similar but the positions of the described items are different, it is possible to prevent erroneous detection.

Japanese Unexamined Patent Publication No. 2005-242786

In this method, the item name information in the form is used as the confirmation information when collating the form. Therefore, it cannot be applied when the item name is not described in the form. In addition, if there is a discrepancy at the time of collation, even if another form is registered in which the form format is similar but the position of the described item is different, the processing is interrupted as a form identification error. Therefore, the item value cannot be extracted.

In order to solve the above-mentioned problem, when a plurality of registered document images similar to the acquired document image are selected in an image processing device in which a plurality of document images are registered as registered document images, for comparison determination. Character recognition is performed on the area of the acquired document image determined by the area determining means and the area determining means, the attributes of the area are determined, and the acquired document image is used. It is characterized by comprising a comparison means for comparing with the attribute of the area of the registered document image corresponding to the area, and a determination means for determining a registered document image having the same attribute as a similar document image.

According to the present invention, even when a plurality of registered document images having similar layout structures are registered, the most similar similar document images can be correctly identified, and the extraction accuracy of item values can be improved.

It is a block diagram which shows an example of the hardware composition of the image processing apparatus in one Embodiment of this invention. It is a figure which shows the whole flow in the image processing apparatus of this embodiment. It is a figure explaining the block selection process in this embodiment. It is a figure explaining the matching process in this embodiment. It is a figure explaining the value area position estimation process in this embodiment. It is a figure explaining the same kind document image extraction process in this embodiment. It is a figure explaining the data format and priority determination table of the value attribute used in the same type document image extraction processing in this embodiment. It is a figure explaining the document image registration process in 1st Embodiment of this invention. It is a figure which shows an example of the confirmation screen in this embodiment. It is a figure explaining the document image registration process in the 2nd Embodiment of this invention. It is a figure which shows an example of the learning result of the appearance frequency of the data format in the 2nd Embodiment of this invention. It is a figure explaining the same kind document image extraction process in the 2nd Embodiment of this invention.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. However, the components described in this embodiment are merely examples, and the scope of the present invention is not intended to be limited thereto.

[First Embodiment]
<Scan Assist System>
FIG. 1 is an example of a block diagram showing a hardware configuration of an image processing apparatus according to an embodiment of the present invention. With such a configuration, the image processing apparatus 100 scans the item values (hereinafter referred to as value values) corresponding to the items (hereinafter referred to as value attributes) necessary for inputting to a predetermined business system. Executes scan assist processing that automatically extracts from the form.

As shown in FIG. 1, the image processing device 100 includes a CPU 111, a ROM 112, a RAM 113, a storage unit 114, an image processing unit 115, a user interface 116, an image reading unit 117, an image output unit 118, and a display device 119. These devices are communicably connected by the data bus 110. Further, it is connected to an external information processing device, a cloud system, a business system, etc. (not shown) via an external interface 120.

The CPU 111 is a controller board for comprehensively controlling the image processing device 100. The CPU 111 boots an OS (operating system) by a boot program stored in the ROM 112. The controller program stored in the storage unit 114 is executed on this OS. The controller program is a program for controlling the image processing device 100. The CPU 111 comprehensively controls each device connected by the data bus 110. The RAM 113 operates as a temporary storage area such as a main memory or a work area of the CPU 111.

The storage unit 114 is a non-volatile memory such as an HDD that can be read and written, and various data such as the above-mentioned controller program and processing results are recorded in the storage unit 114.

The image processing unit 115 analyzes the document image read by the image reading unit 117, which will be described later, and stored on the storage unit 114, and generates information for scan assist. The analysis process consists of the following four processes. The first is a block selection process (BS process) for extracting a block (object area) of an object such as a character string in a document image. The second is an optical character recognition process (OCR process) for extracting character string information from a character string image. The third is a process of determining the type of the document image by obtaining the degree of similarity between the registered document image and the newly scanned document image. The fourth is a process of generating information for scan assist based on the determination process. The information for scan assist consists of item values (for example, form name, form number, billing company information, billing amount, etc.) required to be input to various business systems.

The user interface 116 is an input / output device including, for example, a keyboard, a mouse, a touch panel, and hard keys. The user interface 116 receives various set values or designated values from the user and transmits instruction information to the CPU 111.

The image reading unit 117 is a scanner device, and can acquire a document image in an image data format by reading a paper document or the like using an optical reading device such as a CCD. When the CPU 111 acquires a document image from the image reading unit 117, it stores it in the storage unit 114. Then, when the scan assist process is executed, the CPU 111 reads the document image stored in the storage unit 114 into the RAM 113.

The image output unit 118 is a printer device, and for example, the image output unit 118 can execute a process of outputting image data related to a document image to a storage medium. Alternatively, the image output unit 118 may have a printing function and may execute a process of outputting a document image to an output medium such as a paper medium.

The display device 119 is a display device such as an LCD or a CRT, and displays display data generated by the CPU 111.

The external interface 120 transmits and receives various data such as image data and extracted form information to and from an external device via a network such as a LAN, a telephone line, or a proximity radio such as infrared rays.

The image processing device 100 described above is an example, and may be an image processing device 100 having no one of the image reading unit 117, the image output unit 118, and the display device 119. In that case, the necessary information may be mutually communicated via the external interface 120. Further, some functions of the image processing device 100 may be executed by the external processing device by communicating with each other via the external interface 120. The external processing device may be mounted on a computer device such as a server, or may be mounted on a cloud server on the Internet. Others may be provided with other configurations, if necessary.

<Overall processing flow>
The overall flow of the image processing apparatus 100 of the present embodiment will be described with reference to FIG. FIG. 2 is a flowchart showing the entire scan assist process for the newly scanned document image in the image processing device 100. The flow of FIG. 2 is activated by receiving a document image scan instruction from the user by the user interface 116. At that time, the CPU 111 holds the information about the activated user name in the RAM 113. The user name can be obtained by using an authentication device (not shown) or the like. In the process according to the flowchart shown in FIG. 2, the program code stored in the storage unit 114 is expanded in the RAM 113 and executed by the CPU 111.

In step S201 of FIG. 2, the CPU 111 scans a document via the image reading unit 118, acquires a document image, and stores the document image in the storage unit 114. This document image is hereinafter referred to as a query document image.

In step S202, the CPU 111 reads the query document image stored in the storage unit 114 into the RAM 113, and performs correction processing on the query document image. The correction processing includes color conversion processing, correction processing for document images such as gradation correction, and rotation correction processing. The rotation correction process is performed by calculating the rotation angle by utilizing the fact that the character strings and lines in the document image are arranged in the horizontal direction digitally, and rotating the image by using the calculated rotation angle. ..

In step S203, the CPU 111 causes the image processing unit 115 to execute the block selection (BS) processing for the query document image. The block selection process is a process (area division process) in which an area in an image is divided into object blocks and the attributes of each block are determined. The attribute information for each block obtained by the block selection process is used as information for similarity calculation and the like described later. The details of the block selection process will be described later with reference to FIG.

In step S204, the CPU 111 performs a matching process between the query document image and the registered document image group. The matching process is a process of calculating the degree of similarity between the query document image and all the registered document images. At this time, it can be determined that the registered document image whose similarity is equal to or higher than the predetermined value is likely to be the same type of document image as the query document image. Here, the registered document image group is a collection of previously processed document images registered in the database stored on the storage unit 114 in step S212 described later. The registered document image group may not be the document image itself, but may be a feature quantity that can be used for matching processing such as a block selection processing result of each document image. The CPU 111 reads the registered document image group from the storage unit 114 into the RAM 113 and uses it. The details of the matching process will be described later with reference to FIG.

In step S205, the CPU 111 extracts a candidate image that is likely to be the same type of document as the query document from the registered document image group. This uses the matching result obtained in step S204 to extract a registered image having a degree of similarity equal to or higher than a predetermined value as a document candidate image of the same type. If the similarity with all the registered document image groups does not exceed the predetermined value, it is determined that the document image of the same type as the query document image did not exist.

In step S206, the CPU 111 determines whether or not there is a registered document group extracted and determined as the same type document candidate image in step S205. If the same type document candidate image is extracted, the process proceeds to step S207. On the other hand, if the same type document candidate image is not extracted, the process is skipped and the process proceeds to step S211.

In step S207, the CPU 111 estimates in the query document image the value region position corresponding to each registered document image determined to be the same type document candidate image obtained in step S204. The value area position estimation process is a process of estimating where all the value area positions registered in association with the registered document image are located in the query document image. The details of the value area position estimation process will be described later with reference to FIG.

In step S208, the CPU 111 performs the same type document image extraction process. The same-type document image extraction process is a process of determining the most similar registered document image from a plurality of detected same-type document candidate images as a similar document image. The details of the same type document supplementary image determination process will be described later with reference to FIGS. 6 and 7.

In step S209, the CPU 111 acquires the value value described in the value region position estimated from the same type document image determined in step S208 with respect to the query document image. Specifically, the value value for each value attribute can be acquired by performing OCR processing on each BS text block obtained by the BS processing. Since the BS text block used in the determination process in step S208 has already been subjected to the OCR process, it may be read from the storage unit 114 into the RAM 114 and used.

In step S210, the CPU 111 displays a value value estimation result confirmation screen as shown in FIG. 9 on the display device 119. Here, as a recommendation for each item of the query document image, the value value acquired in step S209 is displayed in a filled state. On the other hand, when it is determined in step S206 that there is no document candidate image of the same type, since the value value that can be recommended has not been acquired, all the item fields are displayed in a blank state. Then, the user will correct the recommended value value while looking at this confirmation screen.

In step S211 the CPU 111 performs a document image registration process. In the document image registration process, when the user modifies the recommendation result of the query document image, the document image information that reflects the user's modification is registered in the database so that it can be referred to in the next and subsequent processes. It is a process to make it. The details of the document image registration process will be described later with reference to FIG.

In step S212, the CPU 111 transmits information necessary for registration to a predetermined business system via the external interface 120. Examples of the information to be transmitted are the scan image and scan information (date / time information, scan setting information, etc.) obtained in step S201, and the item name and item value information confirmed / corrected by the user in step S210. .. Needless to say, the information to be transmitted is not limited to this, and may include user information such as scanning work or confirmation / correction work.

<Block selection processing>
The block selection process (regional division process) performed in step S203 of FIG. 2 will be described with reference to FIG. FIG. 3A is an example of the document image read in step S201. FIG. 3B shows the result of dividing the read query document image into object blocks. The image processing unit 115 determines attributes such as characters (TEXT) / lines (LINE) / tables (TABLE) / photographs (PHOTO) / drawings (PICTURE) for the objects on the query document image. In the object block, the area with each attribute is shown as a defined area. FIG. 3C shows a processing flow of block selection.

A method of block selection processing will be described with reference to FIG. 3 (c). In step S301, the image processing unit 115 generates a black-and-white binary image by performing a binarization process on the document image.

In step S302, the image processing unit 115 extracts a pixel block surrounded by a black pixel contour. This is extracted by performing contour line tracking on the binary image generated in step S301. When the area of the black pixel mass obtained by the contour line tracking is larger than the predetermined area, the contour line tracking is also performed on the white pixels inside, and the white pixel mass is extracted. Further, the recursive extraction of the black pixel mass from the inside of the white pixel mass having a certain area or more is repeated.

In step S303, the image processing unit 115 classifies the black pixel block obtained in step S302 by size and shape, and classifies the black pixels into regions having different attributes. For example, a pixel block having an aspect ratio close to 1 and a certain size is defined as a pixel block corresponding to a character. The portion where adjacent characters are well aligned and grouped is referred to as a character area (TEXT). A flat pixel block is defined as a line region (LINE). The table area (TABLE) is a range occupied by a black pixel block having a certain size or larger and containing a rectangular white pixel block in a well-aligned manner. The area where irregular pixel clusters are scattered is referred to as a photographic area (PHOTO). Then, a pixel block having an arbitrary shape other than that is used as a drawing area (PICTURE). Hereinafter, the block determined to be the character area is referred to as a BS text block. The information in the BS text block can also be used for OCR processing, and OCR processing may be performed as necessary in this step.

<Matching process>
The outline of the matching process performed in step S204 of FIG. 2 will be described with reference to FIG. 4 (a) is a query document image 400, and FIGS. 4 (b) and 4 (c) are examples of registered document images 410 and 420. It can be seen that the query document image 400 shown in FIG. 4A and the registered document image 410 shown in FIG. 4B have similar text arrangements. On the other hand, it can be seen that the query document image 400 shown in FIG. 4A and the registered document image 420 shown in FIG. 4C are not similar in text arrangement. Therefore, the image processing unit 115 calculates the similarity between FIGS. 4 (a) and 4 (b) and the similarity between FIGS. 4 (a) and 4 (c) by using an appropriate method. Since the degree of similarity of the former is higher, the registered document image 410 shown in FIG. 4B is selected as the result of matching.

Subsequently, the concept of the similarity calculation process using the BS text block will be described. FIG. 4D shows a BS text block determined to be a character area with respect to the query document image 400 of FIG. 4A. The dotted line is a BS text block, and ID 401 to ID 405 are IDs of each text block. It can be seen that the character string in FIG. 4 (a) corresponds to the BS text block of FIG. 4 (d). Similarly, FIG. 4 (e) shows a BS text block for the registered document image 410 of FIG. 4 (b), and is composed of BS text blocks of IDs 411 to 416. FIG. 4 (f) shows a BS text block for the registered document image 420 of FIG. 4 (c), and is composed of BS text blocks with IDs 421 to 425.

The similarity calculation using the BS text block is a method of calculating the similarity by paying attention to how similar the shape and arrangement of the BS text block are. For example, when comparing FIG. 4 (d) and FIG. 4 (e), first, the arrangement of a plurality of text blocks located at the top of each document is compared. Specifically, when ID401, ID402, ID403, and ID404 in FIG. 4D are compared with ID4111, ID412, ID413, and ID414 in FIG. 4E, the arrangement of each text block is similar. ing. Further, the arrangement of a plurality of text blocks located at the bottom of each document of FIGS. 4 (d) and 4 (e) is compared. Specifically, the ID 405 in FIG. 4 (d) and the ID 416 in FIG. 4 (e) are aligned so as to be in the same position. Then, ID 403, ID 404, ID 405 in FIG. 4 (d) and ID 414, ID 415, ID 416 in FIG. 4 (e) become similar. Therefore, when comparing FIGS. 4 (d) and 4 (e), both the arrangement of the text blocks at the top of the document and the arrangement of the text blocks at the bottom of the document are similar, so that the same type of document is possible. It is judged that the sex is high, and a high degree of similarity is calculated. In this way, by comparing the upper part and the lower part of the document, as shown in FIG. 4, it becomes easy to compare the document in which the number of lines in the character area changes according to the number of products and the like.

On the other hand, in FIGS. 4 (d) and 4 (f), although there is a BS text block that slightly overlaps, there is no BS text block having a high degree of similarity. Therefore, it is determined that the document images are not of the same type, and a low degree of similarity is calculated.

The method of calculating the similarity is not limited to the above method, and the amount of movement when aligning the BS text blocks, the area of overlap between the BS text blocks, the number of similar BS text blocks, etc. It can be calculated using information.

<Value area position estimation process>
The value region position estimation process performed in step S207 of FIG. 2 will be described with reference to FIG. FIG. 5A is a query document image 500 composed of BS text blocks 501 to 510. FIG. 5B is a registered document image 520 obtained as a document candidate image of the same type in step S204, and is composed of BS text blocks 521 to 530. FIG. 5C is a registered document image 540 obtained as a document candidate image of the same type in step S204, and is composed of BS text blocks 541 to 552. Further, in the registered document images 520 and 540 of FIGS. 5 (b) and 5 (c), a value attribute is given to some BS text blocks. Specifically, ID 521 is given a title attribute representing the title of the form, ID 522 is given a value attribute such as the issuer attribute representing the company name of the form issuing company, and ID 523 is given a value attribute such as the issuer attribute representing the telephone number of the form issuing company. Has been done. In the present embodiment, a form number (issueNo), a form issuance date (issueDate), a subtotal (subTotal), a total tax (taxTotal), a total (total), etc. are given, but other value attributes are given. It doesn't matter.

The value area position estimation process is a process of estimating a BS text block in a query document image whose shape and arrangement are similar to those of the BS text block in the registered document image to which the value attribute is given. Specifically, the CPU 111 uses each of the BS text blocks 521 to 524, 527, and 259 to which the value attribute of the registered document image 520 shown in FIG. 5 (b) is attached to the query document image shown in FIG. 5 (a). Project to the same position on 500. For example, when a BS text block (ID 523) to which the issuerTel attribute of the registered document image 520 is added is projected at the same position on the query document image, it is assumed that the BS text block (ID 523) is projected at the position of ID 563 as shown in FIG. 5 (d). At this time, when the BS text block of the query document image 500 in the vicinity of the projected ID 563 is searched, it is determined that the ID 502 and ID 503 are located in the vicinity. Therefore, in order to determine whether the ID 563 should correspond to either the ID 502 or the ID 503, the arrangement between the text blocks is compared.

First, the ID 563 is aligned with the ID 502, and the arrangements of the text blocks of the query document 500 and the registered document 520 are compared to calculate the similarity. Next, the ID 563 is aligned at the same position as the ID 503, and the arrangement of the text blocks of the query document 500 and the registered document 520 is compared to calculate the similarity. Finally, when the calculated similarity is compared, the similarity of the arrangement of the text blocks is higher when the ID 563 is aligned with the ID 503, so that the text blocks are associated with each other at the position having the higher similarity. That is, each BS text block ID 522, ID 523, and ID 524 of the registered document 520 are associated with each BS text block ID 502, ID 503, and ID 504 of the query document 500, respectively.

On the other hand, the BS text blocks 541 to 544, 548, 550, and 552 to which each value attribute of the registered document image 540 shown in FIG. 5C is also projected are similarly projected onto the query document 500 to block the text. Compare the arrangement of. As a result of the comparison of similarity, each BS text block ID 542, ID 543, and ID 544 of the registered document 540 are associated with ID 502, ID 503, and ID 504 of the query document 500, respectively. The same processing is performed for the BS text block to which other value attributes are added. As a result, as a result of associating the value attributes of each registered document 520 and 540 with each BS text block of the query document, the estimation result of the value area position as shown in FIG. 5 (e) can be obtained.

<Same type document image extraction process>
The same type document image extraction process performed in step S208 of FIG. 2 will be described with reference to FIGS. 6 and 7. FIG. 6A shows a flow chart of the same type document image extraction process, and FIG. 6B shows the result of determining the order to be used for the determination for the BS text block. Further, FIG. 7A is an example of the data format of the value attribute, and FIG. 7B shows a priority determination table of the comparison order among the value attributes. Here, the data format of the value attribute is a correspondence table that defines the characteristics of the character string to be extracted as the value value for each value attribute. For example, it is defined in advance whether or not a specific character string is included in the character string, whether or not the character string structure conforms to a predetermined data format, and whether or not the character type used is limited. deep. The data format is not limited to these, and various conditions may be registered as the data format in addition to the character string information such as the character string length and the character size.

In the priority determination table of FIG. 7 (b), when the data formats are compared between different value attributes, the priorities are determined in descending order of the difference in the data formats shown in FIG. 7 (a). This is because the higher the priority, the easier it is to determine an appropriate value attribute from a plurality of value attributes assigned to the BS text block when determining the same type of document image. For example, in the case of the company name (issuer) and form number (issuerNo) of the form issuer, the priority is low (as 3) because it is only judged whether or not the specific character string is included in the issuer attribute. Set. In addition, in the case of the company name (issuer) and telephone number (issuerTel) of the form issuer, the priority is high (as 1) because it can be judged by the presence / absence of a specific character string, the presence / absence of conformity with the format format, and the presence / absence of limitation of the character type. Set. Further, in the case of the subtotal (subTotal) and the tax total (taxTotal), since the data formats are exactly the same, the priority is set to 0 because it cannot be discriminated.

In step S601 of FIG. 6, the CPU 111 determines whether or not the registered document is determined to be one from the plurality of candidates in step S606 described later. When a plurality of document images remain as candidates, the candidates are narrowed down by repeating the subsequent processes of step S602 and subsequent steps. If only one is determined, the registered document image can be determined to be the same type of document image, so that the process ends.

In step S602, the CPU 111 determines the BS text block of the query document image 500 to be used when comparing and determining the same type document images. The BS text block used for the determination is determined based on the following conditions from the value region positions estimated in step S207.
Condition 1 One BS text block is given different value attributes. Condition 1-1 Area where the priority of comparison judgment is 1. Condition 1-2 Area where the priority of comparison judgment is 2. Condition 1-3 Comparison judgment. Area with priority of 3 Condition 2 Area where one value attribute is given to one BS text block Condition 3 No area for judgment.

Based on the above conditions, the order of the BS text blocks used for the determination is determined with respect to the estimation result of the value region position shown in FIG. 5 (e) obtained in step S207. FIG. 6 (b) shows the estimation result of FIG. 5 (e) with the determination order added. As shown in FIG. 6B, the same value attribute is given to the ID 501 and ID 507 of the query document image 500 in both the registered documents 520 and 540, and any of the above conditions 1 to 3 is given. Because it does not apply to, it is not subject to processing. Further, the ID 509 of the query document image 500 is given different value attributes (taxTotal and subTotal) between the registered document 520 and the registered document 540. However, since it has the same data format, it is judged that it is not suitable for comparison, and it is also excluded from processing. Different value attributes (issuerTel and issuer) are given to the ID 503 of the query document image 500, and the combination is the highest priority (1) in the priority determination table of FIG. 7 (b). , The judgment order is set to 1.

In the same way, the judgment order is determined for the remaining text blocks. Then, when the area determination in step S602 is executed, ID 503, which is a BS text block to which the determination order 1 is assigned, is selected in the first process. When step S602 is executed for the second time or later, the BS text boxes to which the determination orders 2 and 3 are assigned may be selected in order. That is, the region is determined with ID 504 and ID 502 for the second time, ID 510 for the third time, and no area for the fourth time. The priority determination table is not limited to this, and a priority determination table of three or more combinations may be used for condition 1. Further, for the condition 2, the area may be determined by using the priority determination table based on the ease of determination by the value attribute of a single unit.

In step S603, the CPU 111 determines whether or not the BS text block for comparison determination is determined in step S602. If there is a determined BS text block, the process proceeds to step S604, and if not, the process proceeds to step S607.

In step S604, the CPU 111 applies the OCR process to the BS text block for determination determined in step S602, and acquires the character string of the OCR result. Specifically, first, the character string "○○ Co., Ltd." is acquired by applying the OCR process to the ID 503 of the query document image 500. Since the character string information acquired here can be reused in the value value acquisition process of step S209 of FIG. 2, it is stored in the storage unit 114.

In step S605, the CPU 111 compares the character string obtained in step S604 with the data format defined in FIG. 7A. Then, the value attribute to be given to the character string at the position of the value area of the query document image 500 is determined based on the comparison result with the data format. Here, the value attribute may be determined by determining which of the issuerTel attribute and the issuer attribute is suitable as the attribute given to the ID 503. Therefore, it is determined whether or not the character string "○○ Co., Ltd." of ID503 corresponds to the data formats of the issuerTel attribute and the issuer attribute, respectively. When compared with the data format of the issuerTel attribute, it can be seen that neither the format nor the character type conditions are met. On the other hand, it can be seen that the data format of the issuer attribute satisfies the condition that a specific character string is included. Therefore, since it did not apply to the data format of the issuerTel attribute, it is presumed that the value attribute given to the ID 503 of the query document image 500 is the issuer attribute. If the data format applies to a plurality of attributes to be compared, it is presumed that there is a possibility that the data format can be applied to either of the attributes to be compared.

In step S606, the CPU 111 determines which of the same type document candidate images used for comparison is appropriate as the same type document based on the result obtained in step S605, and returns to step S601 again. In the example shown in FIG. 5, since it is estimated that the issuer attribute should be given to the ID 503 in step S605, the registered document image 540 in which the issuer attribute is associated with the ID 503 in FIG. 6B is the same type document image. Determined to be. If the registered document image cannot be specified as one based on the estimation result for one BS text block in step S605, the process of step S605 is performed again with the next BS text block as a comparison target by returning to step S602. Will be. In that case, in step S606, the same type of document is determined from the candidates based on all the comparison results of the text blocks targeted in step S605 so far. In this case, a registered document image in which the ratio of the value attribute determined to match the data format in the area used for comparison is equal to or higher than a predetermined threshold value may be selected.

In step S607, the CPU 111 selects the registered document image having the maximum similarity calculated in the matching process of step S204 of FIG. 2 as the same type document image, and ends the process.

<Document image registration process>
The document image registration process performed in step S211 of FIG. 2 will be described with reference to FIG.

In step S801 of FIG. 8, the CPU 111 determines whether or not the similarity calculated by the matching process of step S204 is equal to or less than a predetermined value with respect to the document image of the same type determined in step S208. If the value is equal to or less than the predetermined value, the process proceeds to step S802, and if the value is equal to or more than the predetermined value, the process proceeds to step S804.

In step S802, the CPU 111 determines whether or not the modification has been made by the user in step S210. If the correction is made, the process proceeds to step S803, and if the correction is not made, additional registration to the database is not required, so the process ends.

In step S803, the CPU 111 determines whether or not the correction content by the user is a correction caused by an estimation error in the value area. If the correction is caused by an estimation error in the value area, the process proceeds to step S804. On the other hand, if the OCR error is simply corrected or the character string assigned before and after the recommended value value is deleted, it is determined that additional registration in the database is unnecessary, and the process is terminated. Specific examples of deleting the character strings given before and after the value are "Co., Ltd." given to the company name (issuer) and "Yen" given to the total (total) and subtotal (subTotal). And so on.

FIG. 9 is an example in which the confirmation screen of the value value estimation result is displayed on the display device 119. The confirmation screen 900 displays the scanned query document image 901 and the estimation result 902 showing the value value described in the value region position estimated from the same type document image in step S209.

In step S804, the CPU 111 acquires the value value for each item input on the confirmation screen of the value value estimation result of FIG.

In step S805, the CPU 111 generates a BS text block by performing a block selection process on the query document image. The block selection process is the same as the block selection process performed in step S203 of FIG.

In step S806, the CPU 111 acquires the position information (coordinates and size) of the BS text block in which the value value acquired in step S804 is described. For the value area position, the full-text OCR process may be executed on the query document image, and the BS text block in which the character strings match may be searched. Alternatively, it can be acquired by having the user select an area in which the correct value is described on the confirmation screen of the value value estimation result shown in FIG.

In step S807, the CPU 111 registers the document image information in the database stored in the storage unit 114 based on the results obtained in step S804 and step S806, and ends the process. The contents to be registered are the position information of the BS text block, and the value area position information to which the value attribute and the value attribute are added. It goes without saying that the registered contents are not limited to this, and necessary information such as query image data, OCR results of value values, scan setting conditions, and operator information may be registered.

With the above processing flow, even when a plurality of registered document images having a layout structure similar to that of the query document image are registered, the most similar registered document image can be specified as a similar document image. In addition, it is possible to accurately estimate the value value for each item as form information. Even when determining a similar document image, the comparison process is limited to the BS text block that can be easily compared and determined, so that the efficient similarity determination process can be performed.

[Second Embodiment]
In the present embodiment, by providing a mechanism for learning the appearance frequency of the data format of the value attribute, it is possible to make a comparative judgment even for a BS text block having the same value attribute, which could not be determined in the first embodiment. The purpose is to do. The same numbers are assigned to the same processes as in the first embodiment, and detailed explanations are omitted.

<Document image registration process>
The document image registration process performed in step S211 of FIG. 2 in the present embodiment will be described with reference to FIGS. 10 and 11. FIG. 10 is a diagram for explaining the processing flow of the document image registration process, and FIG. 11 is a diagram showing the result of learning the appearance frequency of the data format of the value attribute. Note that FIG. 11 is the same as the drawing shown in FIG. 7 (a) except that the total number of times for each registered document image is added.

In steps S801 to S807 of FIG. 10, the CPU 111 registers the query document image in the database stored in the storage unit 114 by performing the same processing as in the first embodiment.

In step S1001, the CPU 111 performs a registration process of the appearance frequency of the data format. The appearance frequency registration process is a process of totaling the data formats used corresponding to each value attribute in the query document image determined as the same type document image for each registered document image. Specifically, in step S209 of FIG. 2, for each value attribute assigned to the query document image 500, it is determined which condition of the data format defined in FIG. 11 is applicable, and the reference document image is associated with the registered document image. Tally. For example, since the ID 503 to which the issuer attribute is given meets the condition of the specific character string "Co., Ltd.", the appearance frequency is increased by 1 and the total is 29 times. Further, since ID 510 to which the total attribute is given and ID 507 and ID 509 to which the subTotal attribute is given meet the condition of the specific character string "circle", the appearance frequency is increased 3 times and totaled to 27 times. Since the ID 504 to which the issuerTel attribute is given meets the condition of the format "##-####-####", the frequency of appearance is increased by 1 and counted as 11 times.

<Same type document image extraction process>
The same type document image extraction process performed in step S208 of FIG. 2 in this embodiment will be described with reference to the detailed process flow of FIG. 6A and FIG. 12 to explain the difference from the first embodiment. 12 (a) shows the query document image 500, and FIGS. 12 (b) and 12 (c) show the registered document image 540 and the registered document image 1200, which are the same type document candidate images selected in step S205 of FIG. Represents. FIG. 12D shows the estimation result of the value region position obtained in step S207. Note that FIGS. 12 (a) and 12 (b) are the same document images as those in FIGS. 5 (a) and 5 (c) used in the description of the first embodiment.

In step S601 of FIG. 6A, the CPU 111 determines whether or not one candidate for the same type of document has been determined. If it is determined to be one, the process is terminated, and if a plurality of candidates remain, step S602 and subsequent steps are repeated.

In step S602, the CPU 111 determines a BS text block to be used when determining a document image of the same type. The BS text block used for the determination is determined based on the following conditions from the value region positions estimated in step S207.
Condition 1 One BS text block is given different value attributes. Condition 1-1 Area where the priority of comparison judgment is 1. Condition 1-2 Area where the priority of comparison judgment is 2. Condition 1-3 Comparison judgment. Area with priority of 3 Condition 2 Area where one value attribute is given to one BS text block Condition 3 Common value attribute is given to one BS text block
Area where there is a significant difference in the frequency of appearance of data formats between registered documents to be compared Condition 4 No area for judgment.

The condition 3 added in the present embodiment will be described with reference to FIGS. 11 and 12. As shown in FIG. 12D, ID501, ID502, ID503, and ID504 are selected as the regions to which the common value attribute is given to one BS text block as the candidates satisfying the condition 3. Then, in this region, a region where there is a significant difference in the data format conditions of each value attribute between the registered document image 540 and the registered document image 1200 is detected. The presence or absence of a significant difference may be determined by the frequency of appearance shown in FIG. For example, the ID 501 to which the title attribute is assigned does not have a data format condition in which there is a significant difference in the appearance frequency, so that it is determined that the determination is not possible. Similarly, it is determined that the ID 502 to which the issueNo attribute is added cannot be determined. On the other hand, in the ID 503 to which the issuer attribute is given, "Co., Ltd." is frequently used in the registered document image 540 as a specific character string, whereas "Co., Ltd." is frequently used in the registered document image 1100. There is. Therefore, it is determined that the condition 3 can be used as a comparative judgment. Similarly, it is determined that the ID 504 to which the issuerTel attribute is given can be compared depending on whether or not there is a notation of "(##)-####-####" as the format.

In step S605, the CPU 111 compares the data formats with respect to the BS text block selected in step S602. Specifically, when comparing the data format of the issuer attribute with the character string "○○ Co., Ltd." of ID503, there is a high possibility that the registered document image 540 uses a specific character string "Co., Ltd." a lot. Can be judged. Also, when comparing the data format of the issuerTel attribute with the character string "03-1234-5678" of ID504, the format "(##)-####-####" is not described, so it is registered. It can be determined that the possibility of the document image 1100 is low.

In step S606, the CPU 111 receives the determination result in step S605 and determines the same type of document image. Here, in the comparison of the appearance frequency of the data formats, the registered document image 540 judged to have a high possibility of being the same type document image is selected.

Since other processes are the same as those in the first embodiment, the description thereof will be omitted.

By the above processing, by learning the appearance frequency of the data format for each registered document image, it becomes possible to perform comparison even with the same value attribute, which could not be determined in the first embodiment. .. In addition, the accuracy of specifying the same type of document image can be improved. As a result, it is possible to improve the estimation accuracy of the value value.

[Other embodiments]
The present invention is also realized by executing the following processing. The software (program) that realizes the functions of the above-described embodiment is supplied to the system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads and executes the program. It is a process to do.

Further, the program code for realizing the function of the present embodiment may be executed by one computer (CPU, MPU) or may be executed by a plurality of computers collaborating with each other. May be good. Further, the program code may be executed by a computer, or hardware such as a circuit for realizing the function of the program code may be provided. Alternatively, a part of the program code may be implemented by hardware and the rest may be executed by a computer.

100 Image processing system 111 CPU
112 ROM
113 RAM
114 Storage unit 115 Image processing unit 116 User interface 117 Image reading unit 118 Image output unit 119 Display device 120 External interface

Claims (12)

In an image processing device in which multiple document images are registered as registered document images,
When a plurality of registered document images similar to the acquired document image are selected, the area determination means for determining the area for comparison determination and the area determination means.
Character recognition is performed on the area of the acquired document image determined by the area determining means, the attribute of the area is determined, and the area of the registered document image corresponding to the area of the acquired document image is determined. A comparison method to compare with the attributes of
An image processing apparatus including a determination means for determining a registered document image having the same attribute as a similar document image. The image processing apparatus according to claim 1, wherein the area determining means determines an area to which a plurality of different attributes are given to one area as an area for comparison determination. The claim means that the comparison means refers to a correspondence table in which a data format is defined for each attribute, and determines the attribute from the data format of the area in which the character recognition of the acquired document image is performed. The image processing apparatus according to 1. The area determination means is characterized in that when data formats are compared between different attributes, the area for comparison determination is determined based on the priority determined in descending order of the difference in the data formats of the correspondence table. The image processing apparatus according to claim 3. 3. The image processing apparatus according to. The image processing apparatus according to claim 5, wherein the data format has at least one of specific character string information, a data format, and a character type appearing in the extracted item. The image according to any one of claims 1 to 6, further comprising an acquisition means for acquiring an item value described in an area estimated from a similar document image determined by the determination means. Processing equipment. The image processing according to claim 7, further comprising a registration means for registering the acquired document image as a registered document image when the area of the item value acquired by the acquisition means is incorrect. Device. The image processing apparatus according to claim 8, wherein the registration means registers the appearance frequency of the data format used in the acquired document image for each attribute associated with the similar document image. The claim is characterized in that, when the same attribute is assigned to one region, the region determining means determines a region for comparison determination based on a significant difference in the appearance frequency of the data format. 9. The image processing apparatus according to 9. In the image processing method in an image processing device in which a plurality of document images are registered as registered document images,
When a plurality of registered document images similar to the acquired document image are selected, an area determination step for determining an area for comparison determination and an area determination step,
Character recognition is performed on the area of the acquired document image determined by the area determining means, the attribute of the area is determined, and the area of the registered document image corresponding to the area of the acquired document image is determined. Comparison steps to compare with the attributes of
An image processing method comprising: a determination step of determining a registered document image having the same attribute as a similar document image. A program for making a computer function as one means of the image processing apparatus according to any one of claims 1 to 10.

Images