JP2022092498A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

To provide an image processing apparatus that can accurately determine the likelihood of an extraction position of an index item extracted from a document image.SOLUTION: An image processing apparatus 100 acquires image data of a document from a scanner unit 122, extracts an extraction block from a text block of the image data, and calculates an evaluation value of the likelihood of an extraction position of the extraction block. Here, the text block has a selected block and an unselected block as a first text block group to be a reference of extraction processing, and has the selected block and the unselected block as a second text block group subjected to the extraction processing. In the calculation of the index value, the image processing apparatus calculates the evaluation value based on the similarity between the first text block group and the second text block group.SELECTED DRAWING: Figure 9

Description

The present invention relates to an image processing device, an image processing method, and a program that identify a scanned document and recommend a file name or the like.

Conventionally, a paper document such as a form is scanned by an image reading device and digitized, and a file name and metadata of the digitized document image are set based on the description contents of the document, and the document is saved in a predetermined storage. There is a system to manage. As a method of setting this file name and metadata, there is a method of setting based on the recognition result of OCR processing on the document image. At that time, in general, the document type is registered in association with the index extraction rule such as the file naming rule, the metadata naming rule, and the destination rule. When the same type of document is scanned, the document type is recognized and the index item is extracted by using the associated index extraction rule. Then, the file name, metadata, and destination are automatically determined based on the extracted index items. This makes it possible to recommend the automatically determined file name, metadata, and destination to the user.

In a system as described above, the extraction of index items generally uses a text block obtained when OCR processing is performed on a document image. However, the text block may be excessively divided depending on how the character strings described in the document image are arranged. Also, the text blocks may be extra-combined. In these cases, the index item intended by the user may not be extracted. Therefore, using the information of the text block registered in the index extraction rule that the user did not select as an index item in the past, the text block to be extracted as an index item is divided and combined at an appropriate position. ing.

A method of dividing a text block to be extracted as an index item at an appropriate position is disclosed in the following patent documents. For example, in Patent Document 1, the text block of the final index item is extracted by dividing the text block of the input document corresponding to the non-selected text block that the user has not selected in the past as the index item.

In a system in which a text block extracted as an index item may be divided and combined at an appropriate position, there is a case where it is desired to know the certainty of the extraction position of the text block extracted as an index item. For example, there is a case where the text block cannot be divided or combined, or a case where the index item intended by the user is not extracted even when the text block is divided or combined. In such cases, the user needs to check and correct any mistakes in the extracted index fields. At that time, it is a heavy load for the user to check all the items including the place where the index item is correctly extracted. Therefore, it is necessary to preferentially inform the user of the index item whose extraction position is not certain.

A method for calculating the certainty of a text block extracted as an index item is disclosed in the following patent documents. For example, Patent Document 2 discloses a method of calculating an evaluation value of an extraction position based on a distance between an extraction position of a text block registered in an index extraction rule and a text block of an input document.

Japanese Unexamined Patent Publication No. 2019-134364 Japanese Unexamined Patent Publication No. 2020-087112

In the method of Patent Document 1, the text block of the input document corresponding to the non-selected text block that the user has not previously selected as an index item is divided. This allows you to extract a text block of the final index item. However, no judgment has been made regarding the certainty of the extraction position of the text block extracted as an index item.

In the method of Patent Document 2, the evaluation value is calculated using only the distance information between the extraction position of the text block registered in the index extraction rule and the text block of the input document. However, depending on how the character strings described in the document image are arranged, the text blocks may be excessively divided or excessively combined. In such a case, the certainty of the extraction position cannot be accurately determined only by the distance information between the extraction position of the text block registered in the index extraction rule and the text block of the input document.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing apparatus capable of accurately determining the certainty of the extraction position of an index item extracted from a document image.

In order to achieve the above object, the image processing apparatus in the present invention is
Image acquisition means for acquiring image data of manuscripts,
A text block acquisition means for acquiring a text block of image data,
An extraction means for extracting an extraction block from the text block,
A calculation means for calculating the evaluation value of the certainty of the extraction position of the extraction block extracted by the extraction means, and a calculation means.
Equipped with
The text block has a selected block and a non-selected block as a first text block group as a reference of the extraction process, and has the extracted block and a non-extracted block as a second text block group to be the target of the extraction process. death,
The calculation means is characterized in that the evaluation value is calculated based on the degree of similarity between the first text block group and the second text block group.

According to the present invention, it is possible to accurately determine the certainty of the extraction position of the index item extracted from the document image.

The figure which shows the whole structure example of the system which concerns on 1st Embodiment. The figure which shows the hardware configuration example of the image forming apparatus which concerns on 1st Embodiment. The flowchart of the processing procedure which concerns on 1st Embodiment. The flowchart of the index extraction processing procedure which concerns on 1st Embodiment. The figure which shows the example of the block selection process which concerns on 1st Embodiment. The figure which shows the example of the index extraction rule which concerns on 1st Embodiment. The figure which shows the example of the text block and OCR block which concerns on 1st Embodiment. The figure which shows the index information and the recommendation block extraction result which concerns on 1st Embodiment. The flowchart of the processing procedure of the recommendation score calculation which concerns on 1st Embodiment. The figure which shows the example of the selection, the non-selection block and the extraction, and the non-extraction block which concerns on 1st Embodiment. The flowchart of the processing procedure of the recommendation score calculation which concerns on 2nd Embodiment. The figure which shows the example of the selection, the non-selection block and the extraction, and the non-extraction block which concerns on 2nd Embodiment. The flowchart of the processing procedure of the recommendation score calculation which concerns on 3rd Embodiment. The figure which shows the example of the selection, the non-selection block and the extraction block which concerns on 3rd Embodiment. The figure explaining the recommendation score calculation processing method which concerns on 4th Embodiment. The figure explaining the method of changing the numerical value of the width similarity parameter which concerns on 4th Embodiment. The flowchart of the processing procedure of the recommendation score calculation which concerns on 5th Embodiment. The figure which shows the example of the selection, the non-selection block and the OCR block which concerns on 5th Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the following embodiments do not limit the present invention according to the scope of claims, and not all combinations of features described in the present embodiment are essential for the means for solving the present invention. do not have.

[First Embodiment]
<System configuration>
FIG. 1 is a diagram showing an overall configuration example of the system according to the first embodiment. As shown in FIG. 1, the image forming apparatus (image processing apparatus) 100 is connected to the LAN 102 and can communicate with a terminal 101 or the like such as a PC via the Internet 103 or the like. In order to realize this embodiment, the terminal 101 such as a PC is not always necessary, and only the image forming apparatus 100 may be configured.

The image forming apparatus 100 is a multifunction device (MFP: Multifunction Peripheral) having a display / operation unit, a scanner unit, and a printer unit. The image forming apparatus 100 can be used as a scanning terminal for scanning a document document and acquiring a document image by using a scanner unit (image acquisition means) included in the image forming apparatus 100. Further, the image forming apparatus 100 has a display / operation unit such as a touch panel and a hard button. As a result, the file name and the recommendation result of the storage destination are displayed, and the user interface for receiving the instruction from the user is displayed.

In this embodiment, an example of performing scan processing, index extraction processing, and file transmission with the image forming apparatus 100 alone will be described. However, a system may be configured such that a part of the functions of the image forming apparatus is performed by another terminal 101. For example, first, the scan process is executed by the image forming apparatus 100, and the scanned image is transmitted to another terminal 101 via a network. Then, the index extraction process may be executed on the other terminal 101, the index extraction result may be returned to the image forming apparatus 100, the image forming apparatus may generate a file, and the file may be transmitted.

<Hardware configuration of image forming apparatus 100>
FIG. 2 is a diagram showing a hardware configuration example of the image forming apparatus 100 according to the first embodiment. The control unit 110 includes a CPU 111, a storage device 112, a printer I / F unit 113, a network I / F unit 114, a scanner I / F unit 115, and a display / operation I / F unit 116. In the control unit 110, each of these units is communicably connected to each other via the system bus 117. The storage device 112 includes a ROM 118, a RAM 119, an HDD 120, and the like. The control unit 110 controls the operation of the entire image forming apparatus 100. Next, each part will be described in detail.

The CPU 111 reads and executes the control program stored in the storage device 112. As a result, the CPU 111 functions as a means for executing each process (reading control, image processing, display control, etc.) in the flowchart described later.

The storage device 112 stores and holds the above program, image data, metadata, setting data, processing result data, and the like. The storage device 112 includes a ROM 118 which is a non-volatile memory, a RAM 119 which is a volatile memory, an HDD 120 which is a large capacity storage area, and the like. The ROM 118 is a non-volatile memory that holds a control program and the like, and the CPU 111 reads and controls the control program. The RAM 119 is a volatile memory used as a temporary storage area such as a main memory and a work area of the CPU 111.

The printer I / F unit 113 connects the printer unit 121 and the control unit 110 via the system bus 117. The printer unit 121 receives the image data generated by the CPU 111 via the printer I / F unit 113, and performs a printing process on the recording paper using the received image data.

The network I / F unit 114 connects the control unit 110 (image forming apparatus 100) to the LAN 102 via the system bus 117. The network I / F unit 114 transmits image data to an external device on the LAN 102, and receives various information from the external device on the LAN 102.

The scanner I / F unit 115 connects the scanner unit 122 and the control unit 110 via the system bus 117. The scanner unit 122 reads the document document to generate scanned image data, and inputs the scanned image data to the control unit 110 via the scanner I / F unit 115. The scanner unit 122 is provided with a document feeder, and can feed a plurality of documents placed on the tray one by one and continuously read them.

The display / operation I / F unit 116 connects the display / operation unit 123 and the control unit 110 via the system bus 117. The display / operation unit 123 is provided with a liquid crystal display unit having a touch panel function, hard buttons, and the like.

As described above, the image forming apparatus 100 according to the present embodiment can provide an image processing function by the above hardware configuration.

<Detailed explanation of this embodiment using a flowchart>
In the present embodiment, the image forming apparatus 100 reads the document manuscript, generates a file name using the character string of the OCR result on the first page, and recommends it to the user through the display / operation unit 123. If the extraction position of the recommended index item is not certain, the image forming apparatus 100 notifies the user that the extraction position is not certain. The index item in the present embodiment is a character string extracted for use as a file name among the character strings acquired from the document image. The user confirms whether the recommended file name is correct in the display / operation unit, and if it is incorrect, gives a correction instruction to the image forming apparatus 100. The image forming apparatus 100 updates the index extraction rule when there is a correction based on the operation result of the user.

FIG. 3 is a flowchart of the processing procedure according to the first embodiment. This processing procedure is executed in the image forming apparatus 100. Specifically, the CPU 111 of the image forming apparatus 100 loads the processing program stored in the ROM 118 into the RAM 119 and executes it to execute the processing of each step of FIG. As a result, the CPU 111 functions as a processing unit.

In step S300, when the image forming apparatus 100 receives the user's instruction from the display / operation unit 123, the scanner unit 122 reads a plurality of document documents one by one from the tray of the document feeder. As a result, the image forming apparatus 100 acquires image data for acquiring an image (document image) of the document manuscript.

In step S301, the CPU 111 analyzes the image data acquired in step S300 and extracts an index item for use as a file name. In addition, a recommendation score indicating the certainty of the extraction position of the index item extracted at this time is calculated for each index item. The details of the index extraction will be described later with reference to FIG.

In step S302, the CPU 111 acquires the threshold value of the recommendation score from the HDD 120 after expanding it to the RAM 119. The threshold value of the recommendation score may be obtained by receiving a user's instruction from the display / operation unit 123 in advance, or may obtain a predetermined fixed setting. Then, if the recommendation score of the index item calculated in step S301 is less than the threshold value, the process proceeds to step S303. If all the recommendation scores of the index items are equal to or higher than the threshold value, the file name and metadata using each index item are determined, and the process proceeds to step S304. The criteria for determining whether to proceed to step S303 or step S304 is not limited to this. For example, a plurality of threshold values may be set and it may be determined whether to proceed to step S303 or step S304.

In step S303, the CPU 111 presents the index item extracted in step S301 and the file name and metadata using each index item to the user by using the display / operation unit 123. After that, the CPU 111 prompts the user for confirmation and accepts the user to correct the file name and the like. Upon accepting confirmation and modification by the user, the CPU 111 determines the file name and metadata. As the index item to be presented, only the index item whose recommendation score is less than the threshold value in step S302 may be presented.

In step S304, the CPU 111 creates a file from the image data acquired in step S300 and assigns the file name determined in step S302 or step S303. In this embodiment, an example of converting to PDF (Portable Document Form) as a file format and saving the file is shown. In the case of PDF, the image data can be divided into pages and saved, and when a plurality of document documents are scanned in step S300, the image data corresponding to each page is stored as one separate page. Save to file.

In step S305, the CPU 111 transmits the file created in step S304 to the destination through the LAN 102.

<Detailed description of index extraction process (S301)>
The details of the index extraction process will be described with reference to FIG. FIG. 4 is a flowchart of the index extraction processing procedure according to the first embodiment. A processing procedure of correcting the orientation of one image data, determining the type of the document, and extracting the index according to the type of the document will be described with reference to FIG. Hereinafter, while explaining the processing procedure based on the flowchart of FIG. 4, a specific example of each processing will be described with reference to other drawings as necessary.

In step S400, the CPU 111 detects the tilt angle from the image data and rotates the image in the opposite direction by the detected tilt to acquire the tilt-corrected image data. The tilt that is the target of tilt correction is a tilt that prevents the document from being read straight due to wear of the rollers in the document feeder when scanned by the scanner unit 122, or cannot be printed straight when printing the document. Is. In the tilt detection, first, the objects included in the image data are detected, and the adjacent object groups in the horizontal direction or the vertical direction are connected. Then, the inclination is obtained by acquiring how much the angle connecting the center positions of the connected objects is inclined from the horizontal direction or the vertical direction. The tilt detection is not limited to the above method. For example, the center coordinates of an object included in the image data are acquired, and while rotating the center coordinate group in units of 0.1 degrees, the angle at which the ratio of the center coordinate groups arranged in the horizontal or vertical direction is the highest is obtained as the inclination. You may. By correcting the tilt of the image by this tilt correction, it is possible to improve the processing accuracy of each of the rotation correction, the block selection process, and the OCR process, which will be described later. OCR is an abbreviation for Optical Character Recognition.

In step S401, the CPU 111 obtains an image obtained by rotating the image in 90-degree increments so that the characters in the document are oriented upright with respect to the tilt-corrected image acquired in step S400. Using the tilt-corrected image acquired in step S400 as a reference image, four images are prepared: a reference image, an image rotated by 90 degrees, an image rotated by 180 degrees, and an image rotated by 270 degrees. Then, a simple OCR process capable of high-speed processing is executed for each image, and the image having the largest number of recognized characters with a certain degree of certainty or higher is acquired as the rotation-corrected image. The rotation correction method is not limited to the above.

In step S402, the CPU 111 acquires a text block from the rotation-corrected image acquired in step S401. The text block refers to a block determined to have a character attribute in the image. First, the block selection process is executed in order to acquire the text block. The block selection process is a determination process in which an image is classified into a foreground area and a background area, and then the foreground area is divided into a text block and other blocks.

Specifically, first, contour line tracking is performed on a query image binarized to black and white, and a block of pixels surrounded by a black pixel contour is extracted. Then, for a mass of black pixels whose area is larger than a predetermined area, contour line tracking is also performed on the white pixels inside, the mass of white pixels is extracted, and the mass of white pixels having a certain area or more is further detected. A mass of black pixels is recursively extracted from the inside. The black pixel mass thus obtained is used as the foreground region, classified by size and shape, and classified into regions having different attributes.

For example, a pixel block having an aspect ratio close to 1 and a constant size is defined as a pixel block corresponding to a character, and a portion where adjacent characters can be grouped in a well-aligned manner is defined as a character area (TEXT). A flat pixel block is defined as a line region (LINE). The table area (TABLE) is a range occupied by a black pixel block having a certain size or larger and containing a rectangular white pixel block in a well-aligned manner. The area where irregular pixel clusters are scattered is referred to as a photographic area (PHOTO). Then, a pixel block having an arbitrary shape other than that is used as a drawing area (PICTURE). The block determined to have the character attribute is specified as the text block from the ones that are divided into areas for each attribute of the object in this way.

FIG. 5 is a diagram showing an example of the block selection process according to the first embodiment. FIG. 5A shows an image after rotation correction, and FIG. 5B shows a block selection processing result of the query image. The information of each text block (information indicating the attribute and the position of each block) obtained in this step is used in OCR processing, document matching similarity calculation, etc., which will be described later. The reason why only the text block is acquired in this step is that the position of the character string composed of the text block expresses the structure of the document image well and is closely related to the scan assist information. Therefore, it does not limit the use of the information of the block determined to have other attributes different from the attributes of the text block, such as the photo area and the table area, in the subsequent processing.

In step S403, the CPU 111 acquires the index extraction rule from the HDD 120, expands it into the RAM 119, and then acquires it. As a result, the CPU 111 acquires the extraction rule of the index item (extraction block) associated with the predetermined document type. FIG. 6 is a diagram showing an example of an index extraction rule according to the first embodiment. The index extraction rule shown in FIG. 6 represents a state in which a document with a form ID “0001” is registered. The index extraction rule is a combination of the document ID, the document identification information, and the index information for one registered document, and holds these combinations as many as the number of registered documents. The registered document (registered document) is a document that serves as a reference for extracting index items, and is a registered document.

Here, the document ID is an individual ID representing the type of document. The document identification information is information necessary for identifying the type of document used in the document matching described later, and the text block acquired in step S402 is used. Index information is a rule that extracts index items from an input document after the type of document is specified and assigns them to a file as a file name or metadata. The input document is a document to be extracted from the index item.

For the index information, the OCR block obtained by executing the OCR process in step S407 with respect to the text block acquired in step S402 is used. The OCR block is a block that represents the internal delimiter information of the text block based on the OCR result.

FIG. 7 is a diagram showing an example of a text block and an OCR block according to the first embodiment. When the OCR process is executed on the text block 701 of FIG. 7A, the OCR blocks 702, 703, 704, 705, and 706 of FIG. 7B are obtained. At the time of document registration, the user can select to extract one or more OCR blocks as index items in this OCR block unit. In FIG. 7C, the selection block 707 shows an example in which the user selects the OCR blocks 704, 705, and 706 as index items at the time of document registration. Hereinafter, the selection block will be described as being composed of one or more OCR blocks.

The index information holds the coordinates, width, and height of the OCR blocks constituting the selected block. Moreover, not only the selected block but also the coordinates, width, and height of the non-selected OCR block are retained. Hereinafter, the OCR block that the user did not select as an index item at the time of document registration is referred to as a non-selection block. Further, the non-selected block will be described as being composed of one or more OCR blocks. In FIG. 6, when the OCR process is executed on the text block 601 to obtain the OCR blocks 602 and 603. Further, in FIG. 6, the OCR block 602 shows a selection block, and the OCR block 603 shows a non-selection block. In the recommendation block extraction described later, the OCR block to be extracted as an index item from the input document is estimated by using the information of the selected block and the non-selected block registered in the index information for each index item.

The index information does not need to hold the character string itself of the OCR result, but may hold the coordinates, width, and height. In addition, the index information has information such as in what order the OCR results are connected to form a file name and how to assign them as metadata. In FIG. 6, index items corresponding to (a) title, (b) issuing company name, and (c) form number are extracted as file names, and a character string connecting them with an underscore, which is a separator, is used as the file name. An example of this is shown. Further, (d) an example of extracting an index item corresponding to the total amount and using it as metadata is shown.

In this embodiment, an example of using the extracted index item as a file name or metadata is shown, but a rule for using the index item may be retained in the folder information of the file destination. Further, as an index extraction rule, as shown in FIG. 6, thumbnails of scanned image data corresponding to the registered document may be held together.

In step S404, the CPU 111 executes document type identification. Document type identification is a process for identifying a document type. Document type identification determines whether a document of the same type as the input document is in the registered document group registered with the index extraction rule, and if there is a document of the same type, the document type is specified. .. In the present embodiment, first, the document similarity degree indicating how similar the shapes and arrangements of the text blocks acquired in step S402 are one-to-one between the input document and each registered document is calculated.

In the document similarity calculation, first, the entire text block of the input document and the entire text block of the registered document are aligned. Next, the square of the total area of each text block in the input document and the area overlapping the text block of the registered document is divided by the product of the total text block area of the input document and the total text block area of the registered document. Let the value be the document similarity. The document similarity calculation described above is performed between the input document and all the registered documents included in the index extraction rule acquired in step S403. Then, if the highest document similarity is a certain value or more, it is determined that the document is of the same type as the registered document. On the other hand, if the highest document similarity is smaller than a certain value, it is determined that the input document is a type that is not included in the registered document.

In step S405, if the input document is a registered document as a result of the document type identification executed in step S404, the CPU 111 advances the process to step S406. On the other hand, if it is not a registered document, the process ends.

In step S406, the CPU 111 assigns the input document the same document ID as the document determined to be the same type of document in step S405. Hereinafter, the registered document refers to a document corresponding to the document ID assigned to the input document in step S406 among the documents included in the index extraction rule acquired in step S403.

In step S407, the CPU 111 executes OCR processing on the text block acquired in step S402, and acquires the OCR block and the character string as a result. It is not necessary to execute OCR processing for all text blocks, and for speeding up the processing, the selected block and non-selected block registered in the index information of the registered document and the text block of the input document where the rectangle overlaps. It may be executed only for.

In step S408, the CPU 111 selects one of the index items registered in the index information of the registered document.

In step S409, the CPU 111 executes a recommendation block extraction that extracts an OCR block of the input document corresponding to the selection block of the index item selected in step S408. FIG. 8 is a diagram showing index information and a recommendation block extraction result according to the first embodiment. FIG. 8A shows an example of a selection block 801 and a non-selection block corresponding to the index item of the issuing company name registered in the index information of the registered document. FIG. 8B shows an example of the result of executing the recommendation block extraction for the index item of the issuing company name in the input document determined to be the same type as the document of FIG. 8A.

In the recommended block extraction, the blocks to be extracted as index items are extracted for each OCR block of the input document by using the information of the selected block and the non-selected block registered in the index information. Specifically, the block to be extracted is determined by using the shape such as the height and width of the OCR block, the coordinate values such as the left end, the right end, the upper end, and the lower end, the distance between the blocks, and the number of blocks. As a result, the selected OCR block and the non-selected OCR block are associated with the OCR block of the input document.

In FIG. 8, for example, the selected block 801 and the OCR block group 805, the non-selected block 802 and the OCR block group 804, and the non-selected block 803 and the OCR block group 806 are associated with each other. Finally, the OCR block group 805 associated with the selection block 801 is extracted as the OCR block group having the index item as the index item. Hereinafter, in the recommendation block extraction, the OCR block group of the input document extracted as the index item is referred to as an extraction block (extracted text block), and the OCR block group of the input document not extracted is referred to as a non-extraction block. Further, the extraction block and the non-extraction block will be described as being composed of one or more OCR blocks. There are other methods for extracting recommended blocks besides those described above. As the method of extracting the recommended block, it is sufficient that at least the OCR block group of the input document corresponding to the selected block of the index item selected in step S408 can be extracted.

In step S410, the CPU 111 calculates a recommendation score (evaluation value) indicating the certainty of the extraction position of the extracted index item. The recommendation score is calculated based on the information of the selected block and the non-selected block and the information of the extracted block and the non-extracted block for each index item selected in step S408. The details of the recommendation score calculation process will be described later.

In step S411, the CPU 111 determines whether or not the recommendation block extraction and the recommendation score calculation process have been completed for all the index items registered as the index information of the registered document. If the recommendation block extraction and the recommendation score calculation processing have been completed for all the index items, the processing is completed. On the other hand, if it is not completed, the process proceeds to step S408.

<Detailed explanation of recommendation score calculation process (S410)>
The details of the recommendation score calculation process will be described with reference to FIG. FIG. 9 is a flowchart of the processing procedure for calculating the recommendation score according to the first embodiment. A process of calculating a recommendation score indicating the certainty of the extraction position of the extracted index item based on the information of the selected block and the non-selected block and the information of the extracted block and the non-extracted block will be described with reference to FIG. Hereinafter, while explaining the processing procedure based on the flowchart of FIG. 9, a specific example of each processing will be described with reference to other drawings as necessary.

In step S900, the CPU 111 acquires a selection block in the registered document. FIG. 10 is a diagram showing an example of a selected block, a non-selected block and an extracted block, and a non-extracted block according to the first embodiment. FIG. 10A shows an example of the selected block 1001 and the non-selected blocks 1002 and 1003 in the registered document. In step S900, the selection block 1001 in FIG. 10A is acquired.

In step S901, the CPU 111 acquires the non-selected blocks 1002 and 1003 in the registered document.

In step S902, the CPU 111 acquires an extraction block in the input document. FIG. 10B shows an example of the extraction block 1004 and the non-extraction blocks 1005 and 1006 in the input document. In step S902, the extraction block 1004 in FIG. 10B is acquired.

In step S903, the CPU 111 acquires the non-extracted blocks 1005 and 1006 in the input document.

In step S904, the CPU 111 calculates the similarity between the selection block and the extraction block. FIG. 10C shows an example in which the selected block and the non-selected block and the extracted block and the non-extracted block are superimposed. In FIGS. 10A to 10C, blocks 1001 and 1007, 1002 and 1008, 1003 and 1009, 1004 and 1010, 1005 and 1011 and 1006 and 1012 represent the same block, respectively.

In step S904, the similarity parameter between the selection block 1007 and the extraction block 1010 is calculated. Here, the similarity parameter is calculated using the overlap between blocks, the shape such as height and width, the coordinate values such as the left end, right end, upper end, and lower end of the block, the distance of the blocks, and the number of blocks. For example, each similarity parameter is expressed by a numerical value of 0 to 1, and the higher the similarity is, the larger the numerical value is expressed.

Specifically, the extrinsic rectangle (x coordinate, y coordinate, width, height) of the selection block 1007 is (10, 5, 30, 15), and the extrinsic rectangle (x coordinate, y coordinate, width, height) of the extraction block 1010 is (x coordinate, y coordinate, width, height). The coordinates may be (5, 5, 70, 15). In this case, the overlap rate between the selected block and the extraction block is calculated to be "0.4". This is used as the overlap similarity parameter between the selection block and the extraction block. Further, the number of OCR blocks constituting the selected block may be 1, and the number of OCR blocks constituting the extraction block may be 3. In this case, the ratio of the number of blocks of the selected block and the number of extracted blocks is calculated to be "0.3". This is used as the block number similarity parameter between the selected block and the extracted block.

As described above, the similarity parameter does not have to be limited to one, and a plurality of similarity parameters may be calculated. As for the similarity parameter calculated here, the larger the numerical value is, the more similar the selection block 1007 and the extraction block 1010 are. Therefore, it is shown that the extraction position of the index item is probable. In addition, there are various methods for calculating the similarity parameter other than those described above. The method for calculating the similarity parameter may be at least one that can calculate the similarity parameter by using the information of the selection block and the extraction block.

In step S905, the CPU 111 calculates the similarity parameter between the selected block 1007 and the non-extracted blocks 1011 and 1012. Here, the similarity parameter is calculated using the overlap between blocks, the shape such as height and width, the coordinate values such as the left end, right end, upper end, and lower end of the block, the distance of the blocks, and the number of blocks. For example, each similarity parameter is expressed by a numerical value of 0 to 1, and the higher the similarity is, the larger the numerical value is expressed.

Specifically, the circumscribed rectangle (x coordinate, y coordinate, width, height) of the selection block 1007 is set to (30, 5, 30, 15). Further, the circumscribed rectangles (x coordinate, y coordinate, width, height) of the non-extracted blocks 1011 and 1012 are set to (5, 5, 20, 15) and (75, 5, 20, 15). In this case, the overlap ratio between the selected block 1007 and the non-extracted block 1011 is determined to be "0". This is used as the overlap similarity parameter between the selected block 1007 and the non-extracted block 1011. Further, the overlap ratio between the selected block 1007 and the non-extracted block 1012 is determined to be "0". This is used as the overlap similarity parameter of the selected block 1007 and the non-extracted block 1012.

As described above, the similarity parameter does not have to be limited to one, and a plurality of similarity parameters may be calculated. The similarity parameter calculated here indicates that the larger the numerical value is, the more similar the selected blocks 1007 and the non-extracted blocks 1011 and 1012 are, so that the extraction position of the index item is not certain.

In step S906, the CPU 111 calculates the similarity parameter between the non-selected blocks 1008 and 1009 and the extraction block 1010. Similar to step S905, the similarity parameter is calculated by using the overlap between blocks, the shape such as height and width, the coordinate values such as the left end, the right end, the upper end, and the lower end of the block, the distance of the blocks, and the number of blocks. The larger the numerical value of the similarity parameter calculated here is, the more similar the non-selected blocks 1008 and 1009 are to the extraction block 1010, and the more uncertain the extraction position of the index item is.

In step S907, the CPU 111 calculates the similarity parameter between the non-selected blocks 1008 and 1009 and the non-extracted blocks 1011 and 1012. Similar to steps S905 and S906, the similarity parameter is calculated using the overlap between blocks, the shape such as height and width, the coordinate values such as the left end, right end, upper end, and lower end of the block, the distance of the blocks, and the number of blocks. .. The larger the numerical value of the similarity parameter calculated here, the more likely the extraction position of the index item is likely because the non-selected blocks 1008 and 1009 and the non-extracted blocks 1011 and 1012 are similar.

In step S908, the CPU 111 acquires weights indicating the importance of each similarity parameter calculated in steps S904, S905, S906, and S907 after being expanded from the HDD 120 to the RAM 119. At this time, the weights for the similarity parameters calculated in steps S904 and S907 are defined so that the larger the numerical value of the similarity parameters, the higher the recommendation score. On the other hand, the weights for the similarity parameters calculated in steps S905 and S906 are defined so that the larger the numerical value of the similarity parameters, the lower the recommendation score. Also, the weights for each similarity parameter are defined so that the final recommendation score is normalized to 0 to 1. As the weight, a predetermined fixed setting may be acquired, or a predetermined setting for each document ID may be acquired.

In step S909, the CPU 111 calculates the weighted sum of the similarity parameters by using the similarity parameters calculated in steps S904, S905, S906, and S907 and the weights acquired in step S908. Finally, the processing ends with the value of this weighted sum as the recommendation score for the index item.

As described above, according to the first embodiment, the similarity parameter is calculated by comparing the OCR block group (first text block group) of the registered document and the OCR block group (second text block group) of the input document. Specifically, the similarity parameter between the selected block and the non-selected block in the registered document and the extracted block and the non-extracted block in the input document is calculated. Using these similarity parameters, a recommendation score indicating the certainty of the extraction position of the extracted index item is calculated for each index item. This makes it possible to calculate the recommendation score with high accuracy.

When there are index items whose recommendation score is less than the threshold value, the CPU 111 presents the file name and metadata using each index item to the user by using the display / operation unit 123. This makes it possible to inform the user when the extraction position of the index item is not certain. In addition, it is possible to accept confirmation and correction of index items by the user.

[Second Embodiment]
In the first embodiment, an example of calculating the recommendation score by using the similarity parameter between the selected block and the non-selected block in the registered document and the extracted block and the non-extracted block in the input document has been described. However, when calculating the similarity parameter using all non-selected blocks and non-extracted blocks, the recommendation score reflects the similarity parameter with a block at a long distance that is not relevant to the index item. Sometimes.

Therefore, in the present embodiment, an example of changing the weight (importance) of each similarity parameter so as to reduce the influence on the recommendation score as the distance from the selection block and the extraction block is longer will be described. In the following, only the parts that are different from the first embodiment will be described.

FIG. 11 is a flowchart of the processing procedure for calculating the recommendation score according to the second embodiment. A processing procedure for calculating a recommendation score indicating the certainty of the extraction position of the extracted index item will be described with reference to FIG. The difference between FIG. 11 and the flowchart of the first embodiment shown in FIG. 9 will be described. Steps S900 to S909 are the same as those in the first embodiment. In addition to the flow of the first embodiment, there is a configuration of steps S910 to S912. Hereinafter, while explaining the processing procedure based on the flowchart of FIG. 11, specific examples of each processing will be described with reference to other drawings as necessary.

In step S910, the CPU 111 determines whether there is a non-extracted block that does not overlap with the selected block. If present, the CPU 111 advances the process to step S912. If it does not exist, the CPU 111 advances the process to step S911.

In step S911, the CPU 111 determines whether there is a non-selected block that does not overlap with the extraction block. If present, the CPU 111 advances the process to step S912. If it does not exist, the CPU 111 advances the process to step S909.

In step S912, the CPU 111 changes the weight of each similarity parameter calculated from the non-selected block and the non-extracted block according to the distance from the selected block or the extracted block.

FIG. 12 is a diagram showing examples of selection, non-selection blocks and extraction, and non-extraction blocks according to the second embodiment. FIG. 12A shows an example of selected blocks 1201 and non-selected blocks 1202, 1203, 1204 in a registered document. FIG. 12B shows examples of extracted blocks 1205 and non-extracted blocks 1206, 1207, 1208 in the input document. Further, FIG. 12C shows an example of the extraction block 1209 and the non-extraction blocks 1210, 1211, 1212 in the input document when a document different from that of FIG. 12B is input.

The non-selected block 1204 is a block that is separated from the selected block 1201 and the extraction blocks 1205 and 1209. Further, the non-extracted blocks 1208 and 1212 are blocks that are separated from the selection block 1201 and the extraction blocks 1205 and 1209.

Here, for example, the overlap similarity parameter of the non-selected block 1204 and the non-extracted block 1208 is calculated as "0.1", and the overlap similarity parameter of the non-selected block 1204 and the non-extracted block 1212 is calculated as "0.5". In some cases. In this case, since the non-extracted blocks 1208 and 1212 are separated from the extracted blocks, the weight of the similarity parameter of these blocks is reduced. This reduces the effect of the overlap similarity parameter calculated from the non-extracted blocks 1208 and 1212, which are not directly related to the index item, on the recommendation score. In this way, the final recommendation score is calculated by obtaining the weighted sum of each similarity parameter by using the weights changed according to the distances from the selection block and the extraction block.

As described above, according to the second embodiment, the weight of the similarity parameter is changed so that the longer the distance between the selection block and the extraction block is, the smaller the influence on the recommendation score is. This can prevent the similarity parameter of the block at a long distance, which is not related to the index item, from being reflected in the recommendation score. As a result, the accuracy of the recommendation score calculation can be improved.

[Third Embodiment]
In the first embodiment, an example of calculating the recommendation score by using the similarity parameter between the selected block and the non-selected block in the registered document and the extracted block and the non-extracted block in the input document has been described. However, the non-selected block of the registered document and the non-extracted block of the input document do not always exist and may not exist.

Therefore, in the third embodiment, an example in which the weight for each similarity parameter is changed depending on the case where the non-selected block or the non-extracted block does not exist will be described. In the following, only the parts that are different from the first embodiment will be described.

FIG. 13 is a flowchart of the processing procedure for calculating the recommendation score according to the third embodiment. A processing procedure for calculating a recommendation score indicating the certainty of the extraction position of the extracted index item will be described with reference to FIG. The difference between FIG. 13 and the flowchart of the first embodiment shown in FIG. 9 will be described. Steps S900 to S909 are the same as those in the first embodiment. In addition to the flow of the first embodiment, there is a configuration of steps S913 to S915. Hereinafter, while explaining the processing procedure based on the flowchart of FIG. 13, specific examples of each processing will be described with reference to other drawings as necessary.

In step S913, the CPU 111 determines whether or not a non-selected block exists. If present, the CPU 111 advances the process to step S914. If it does not exist, the CPU 111 advances the process to step S915.

In step S914, the CPU 111 determines whether or not the non-extracted block exists. If present, the CPU 111 advances the process to step S909. If it does not exist, the CPU 111 advances the process to step S915.

In step S915, the CPU 111 changes the weight for each similarity parameter depending on the absence of the non-selected block or the non-extracted block.

FIG. 14 is a diagram showing an example of a selection, non-selection block, and an extraction block according to the third embodiment. FIG. 14A shows an example of the selected block 1401 and the non-selected block 1402 in the registered document. FIG. 14B shows an example in which the non-selected block does not exist in the registered document and only the selected block 1403 exists. Further, FIG. 14C shows an example in which the non-extracted block does not exist in the input document and only the extracted block 1404 exists.

Here, for example, as shown in FIG. 14 (A), the registered document has a selected block and a non-selected block, but as shown in FIG. 14 (C), the input document does not have a non-extracted block. In some cases. In this case, the weight of the overlap similarity parameter of the non-selected block 1402 and the extraction block 1404 is increased.

On the other hand, for example, as shown in FIG. 14 (B), the non-selected block may not exist in the registered document, and as shown in FIG. 14 (C), the non-extracted block may not exist in the input document. In this case, the weights of the shape similarity parameters of the selection block 1403 and the extraction block 1404 are increased.

There are various methods for changing the weight for the similarity parameter other than those described above, but any method may be used as long as it is a method of changing depending on whether or not a non-selected block and a non-extracted block exist. In this way, the final recommendation score is calculated by obtaining the weighted sum of each similarity parameter by using the weights changed according to the case where the non-selected blocks and the non-extracted blocks do not exist.

As described above, according to the third embodiment, the weight of each similarity parameter is changed according to the case where the non-selected block or the non-extracted block does not exist. As a result, the recommendation score can be calculated in consideration of the case where the non-selected block and the non-extracted block do not exist, and the accuracy of the recommendation score calculation can be improved.

[Fourth Embodiment]
In the first embodiment, an example of calculating the recommendation score by using the similarity parameter between the selected block and the non-selected block in the registered document and the extracted block and the non-extracted block in the input document has been described. However, the method of calculating the numerical value of the similarity parameter is not changed depending on whether the block of the registered document and the block of the input document match or when the difference is large. For example, if the width of the selected block of the registered document and the width of the extracted block of the input document match, it is assumed that the area of that block is a fixed-length character string area, so the accuracy of the extraction position of the index item is certain. Is considered to be high. Therefore, in such a case, we would like to calculate a high recommendation score.

Therefore, in the fourth embodiment, an example of changing the numerical value of each similarity parameter based on the difference between the registered document and the input document will be described. Specifically, based on the overlap between blocks of the registered document and the input document, the shape such as height and width, the coordinate values such as the left end, right end, upper end, and lower end of the block, the distance of the blocks, and the difference value of the number of blocks. Change the numerical value of each similarity parameter. In the following, only the parts that are different from the first embodiment will be described.

FIG. 15 is a diagram illustrating a recommendation score calculation processing method according to the fourth embodiment. FIG. 15A shows an example of the selected block 1501 and the non-selected block 1502 in the registered document. FIG. 15B shows an example of the extraction block 1503 and the non-extraction block 1504 in the input document. Further, FIG. 15C shows an example of the extraction block 1505 and the non-extraction block 1506 in the input document when a document different from that in FIG. 15B is input.

In this embodiment, each similarity parameter is calculated in steps S904, S905, S906, and S907. At the time of this calculation, the numerical value of each similarity parameter is changed based on the overlap between the blocks of the registered document and the input document, the height and width, the coordinate value, the distance, and the difference value of the number of blocks.

FIG. 16 is a diagram illustrating a method of changing the numerical value of the width similarity parameter according to the fourth embodiment. FIG. 16 shows an example in which the numerical value 1601 of the width similarity parameter is changed based on the difference value of the width between the selected block and the extracted block. Here, the area 1602 shows the case where the width of the selected block is larger than the width of the extraction block, and the area 1603 shows the case where the width of the extraction block is larger than the width of the selection block.

Here, the examples of FIGS. 15A and 15B will be described. In the example of FIG. 15A, the circumscribed rectangle (x coordinate, y coordinate, width, height) of the selection block 1501 is (10, 5, 30, 15). In the example of FIG. 15B, the circumscribed rectangle (x coordinate, y coordinate, width, height) of the extraction block 1503 is (10, 5, 30, 15). At this time, since the difference value between the widths of the selected block and the extraction block is "0", the numerical value 1601 of the width similarity parameter is set to "1". This makes it possible to increase the final recommendation score when the selected block and the extracted block match.

Further, it will be described with reference to the examples of FIGS. 15 (A) and 15 (C). In the example of FIG. 15A, the circumscribed rectangle (x coordinate, y coordinate, width, height) of the selection block 1501 is (10, 5, 30, 15). In the example of FIG. 15C, the circumscribed rectangle (x coordinate, y coordinate, width, height) of the extraction block 1505 is (10, 5, 60, 15). At this time, since the difference value between the widths of the selected block and the extraction block is “+30”, the numerical value 1601 of the width similarity parameter is set to “0.4”. On the other hand, when the difference value between the widths of the selected block and the extraction block is "-30", the numerical value 1601 of the width similarity parameter is set to a value larger than "0.4". In this way, when the absolute value of the difference value is the same, the numerical value of the width similarity parameter may be changed according to the positive or negative.

There are various methods for changing the numerical value of each similarity parameter other than those described above, but the numerical value of each similarity parameter is changed based on the difference value between the blocks of the registered document and the input document. Any method can be used. In this way, the final recommendation score is calculated by obtaining the weighted sum of each similarity parameter using the numerical value of the similarity parameter changed based on the difference value between the blocks of the registered document and the input document. do.

As described above, according to the present embodiment, the numerical value of each similarity parameter is changed based on the difference between the registered document and the input document. Specifically, based on the overlap between blocks of the registered document and the input document, the shape such as height and width, the coordinate values such as the left end, right end, upper end, and lower end of the block group, the block distance, and the difference value of the number of blocks. , Change the numerical value of each similarity parameter. As a result, the recommendation score can be calculated in consideration of the case where the block of the registered document and the block of the input document match, and the accuracy of the recommendation score calculation can be improved.

[Fifth Embodiment]
In the first embodiment, an example of calculating the recommendation score by using the similarity parameter between the selected block and the non-selected block in the registered document and the extracted block and the non-extracted block in the input document has been described. However, when the OCR block of the registered document and the OCR block of the input document are overlapped, the positions of the blocks may shift up and down. This can occur when the user slightly modifies the description position when creating a document, when the document is printed, or when the document is scanned. In such a case, the OCR block that is one line above or below the OCR block that is originally desired to be extracted may be extracted. Even in such a case, a high recommendation score may be calculated.

Therefore, in the present embodiment, when there are a plurality of lines of OCR blocks of the input document that overlap with the selection block of the registered document, an example of subtracting the recommendation score by using the penalty value which is the value for subtracting the score will be described. In the following, only the parts that are different from the first embodiment will be described.

FIG. 17 is a flowchart of the processing procedure for calculating the recommendation score according to the fifth embodiment. A processing procedure for calculating a recommendation score indicating the certainty of the extraction position of the extracted index item will be described with reference to FIG. First, the difference between FIG. 17 and the flowchart of the first embodiment shown in FIG. 9 will be described. Steps S900 to S909 are the same as those in the first embodiment. In addition to the flow of the first embodiment, there is a configuration of steps S916 to S918. Hereinafter, while explaining the processing procedure based on the flowchart of FIG. 17, specific examples of each processing will be described with reference to other drawings as necessary.

In step S916, the CPU 111 calculates the number of lines of the OCR block of the input document that overlaps with the selection block. FIG. 18 is a diagram showing an example of a selected, non-selected block and an OCR block according to the fifth embodiment. FIG. 18A shows selected blocks 1801 and non-selected blocks 1802 and 1803 registered in the index information of the registered document. FIG. 18B shows the OCR blocks 1804, 1805, 1806, 1807, 1807, 1808 of the input document. FIG. 18C shows an example in which the selection block and the OCR block of the input document are superimposed. In FIGS. 18A to 18C, the selection blocks 1801 and 1809 represent the same block. Also, the OCR blocks 1804 and 1810, 1805 and 1811, 1806 and 1812, 1807 and 1813, and 1808 and 1814 represent the same block, respectively.

In step S916, the number of lines in the OCR block of the input document that overlaps with the selected block is calculated. When the selection block 1809 overlaps with the OCR blocks 1810 and 1812 of the input document as shown in FIG. 18C, the number of lines of the OCR block of the input document overlapping with the selection block is calculated as 2. As a specific method for calculating the number of lines, first, an OCR block of an input document that overlaps with the selection block is obtained. Next, among the obtained OCR blocks, those in which the rectangles overlap in the column direction and do not overlap in the row direction are extracted. Then, the number of extracted OCR blocks is calculated as the number of lines of the OCR block of the input document that overlaps with the selection block.

In step S917, the CPU 111 determines whether the number of rows calculated in step S916 is 2 or more. If the calculated number of rows is 1, the process is completed, and if it is 2 or more, the process proceeds to step S918.

In step S918, the CPU 111 acquires the penalty value from the HDD 120 after expanding it to the RAM 119. Then, the penalty value is subtracted from the recommendation score calculated in step S909. Finally, the value after subtraction is updated as a recommendation score for the index item, and the process ends.

As described above, according to the present embodiment, when there are a plurality of lines of OCR blocks of the input document that overlap with the selection block of the registered document, the recommendation score is lowered by using the penalty value. As a result, the recommendation score can be calculated in consideration of the possibility of extracting the block one line above or below the block originally desired to be extracted, and the accuracy of the recommendation score calculation can be improved.

[Other embodiments]
Although the present invention has been described in detail based on the preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various embodiments within the range not deviating from the gist of the present invention are also included in the present invention. Will be. In addition, some of the above-described embodiments may be combined as appropriate.

The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 ... Image forming apparatus 110 ... Control unit 111 ... CPU
112 ... Storage device 123 ... Display / operation unit

Claims (9)

Image acquisition means for acquiring image data of manuscripts,
A text block acquisition means for acquiring a text block of image data,
An extraction means for extracting an extraction block from the text block,
A calculation means for calculating the evaluation value of the certainty of the extraction position of the extraction block extracted by the extraction means, and a calculation means.
Equipped with
The text block has a selected block and a non-selected block as a first text block group as a reference of the extraction process, and has the extracted block and a non-extracted block as a second text block group to be the target of the extraction process. death,
The calculation means is an image processing apparatus characterized in that the evaluation value is calculated based on the degree of similarity between the first text block group and the second text block group. The calculation means is
The similarity between the selected block and the extracted block, the similarity between the non-selected block and the non-extracted block, the similarity between the selected block and the non-extracted block, and the non-selected block and the extracted block. The image processing apparatus according to claim 1, wherein the evaluation value is calculated based on the degree of similarity with the above. The calculation means is
The evaluation value is calculated based on the importance of each text block.
The invention according to claim 1 or 2, wherein the importance of the non-selected block, which is far from the selected block, is reduced, and the importance of the non-extracted block, which is far from the extracted block, is reduced. Image processing equipment. The calculation means is
The evaluation value is calculated based on the importance of each text block.
The image processing apparatus according to any one of claims 1 to 3, wherein the importance is changed depending on the absence of the non-selected block or the non-extracted block. The calculation means is
A claim characterized in that the first text block group and the second text block group are compared, and the similarity is calculated based on the overlap, the shape, the coordinate value, the distance, or the difference value of the number of blocks. The image processing apparatus according to any one of 1 to 4. The calculation means is
The first text block group and the second text block group are compared, and the similarity is calculated based on their overlap and the similarity is calculated.
The image processing apparatus according to any one of claims 1 to 5, wherein when the selection block overlaps a plurality of lines of the second text block group, the evaluation value is lowered. An identification means for identifying the document type of the image data and
It has an extraction rule acquisition means for acquiring an extraction rule of the extraction block associated with the document type.
The image processing apparatus according to any one of claims 1 to 6, wherein the extraction means extracts the extraction block from the text block according to the extraction rule. An image data acquisition process for acquiring image data of a manuscript by an image acquisition means,
The text block acquisition process for acquiring the text block of the image data and
An extraction process for extracting an extraction block from the text block,
A calculation process for calculating the evaluation value of the certainty of the extraction position of the extraction block, and
Equipped with
The text block has a selected block and a non-selected block as a first text block group as a reference of the extraction process, and has an extracted block and a non-extracted block as a second text block group to be the target of the extraction process. ,
The image processing method, characterized in that, in the calculation step, the evaluation value is calculated based on the degree of similarity between the first text block group and the second text block group. A program comprising causing a computer to execute the image processing method according to claim 8.

Images