JP7379051B2 - Information processing device, control method for information processing device, and its program - Google Patents

Description

The present invention relates to an image processing device, a method for controlling the image processing device, and a program therefor, which has a function of dividing files when multiple documents are scanned and recommending file names, etc. for each document. .

There is a business flow in which paper documents such as forms are scanned with an image reading device and digitized. In this case, the digitized document image is generally given a file name and meta information according to the type and content of the document, and then stored in a predetermined folder. Here, manually naming files, assigning meta information, and specifying destination folders becomes a heavy burden when the number of paper manuscripts to be digitized increases. In particular, in a multifunction peripheral (MFP), due to restrictions on the user interface (UI), when it is necessary to input file names and meta information using a software keyboard, the load on the user becomes heavier. Furthermore, if it is necessary to specify a destination or create a new folder, the burden on the user will similarly increase.

As a method for solving this problem, there is a method disclosed in Patent Document 1. In this method, first, document types and index extraction rules such as file naming rules, meta information assignment rules, and destination rules are associated and registered in advance. Next, if a document of the same type is scanned, it will recognize the document type and use the associated index extraction rules to automatically determine the file name, meta information, and destination, and then notify the user. Recommend. In this way, it is possible to greatly reduce the user's workload.
Furthermore, the method disclosed in Patent Document 2 not only recommends registered documents, but also provides a confirmation UI that accepts users' correction of recommendation results and naming of unregistered documents. This makes it possible for the user to automatically learn index extraction rules simply by repeatedly performing naming and modification operations without having to initialize the index extraction rules.

Japanese Patent Application Publication No. 2011-15348 Japanese Patent Application Publication No. 2018-124656

However , when repeating the series of processes described in Patent Document 2 from the recommendation to receiving the user's confirmation result through the UI for the number of files, the processing time related to the recommendation is Waiting time occurs every time .

According to the present invention, when a series of multiple originals set by a user is continuously scanned by a scanner , the user's information regarding the image data is a control means that starts character recognition processing sequentially from the first page of the plurality of pages without waiting for an operation ; and a character corresponding to the result of the character recognition processing for the image data of the first page of the image data. A processing program that causes a computer to function as a providing means for providing a correction screen that displays a string and is capable of accepting correction instructions from a user for the character string, the processing program comprising While the correction screen for the provided image data of the first page is being displayed, the character recognition process for the image data of the subsequent page of the first page is executed by the control means. do.

According to the present invention, when a series of processes from a recommendation to receiving a user's confirmation result through the UI is repeated for the number of files, the waiting time corresponding to the processing time related to the recommendation can be reduced before confirmation of each file.

Diagram showing the overall configuration of a system according to an example A diagram showing the hardware configuration of an image processing device according to an embodiment Flowchart of image processing procedure according to the embodiment A diagram showing an example of file division position determination processing according to the embodiment Flowchart of processing procedure of OCR-related processing according to the embodiment A diagram showing an example of block selection processing according to the embodiment Flowchart of confirmation/correction processing according to the example Image example of the first page after file division according to the example Example of index extraction rule according to the embodiment Example of index extraction rule according to the embodiment Flowchart of recommendation content generation processing according to the embodiment Example of specifying a file name using the confirmation/correction screen according to the example Example of specifying a file name using the confirmation/correction screen according to the example Example of file name recommendation using the confirmation/correction screen according to the example Flowchart of processing procedures according to other embodiments Flowchart of confirmation/correction processing according to other embodiments Example of second index extraction rule according to other embodiments Example of second index extraction rule according to other embodiments Flowchart of recommendation content generation processing according to another embodiment

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following examples do not limit the claimed invention, and not all combinations of features described in the examples are essential to the solution of the invention. .

<System configuration>
FIG. 1 is a diagram showing the overall configuration of a system to which this embodiment can be applied. As shown in FIG. 1, the image processing apparatus 100 is connected to a LAN 102 and can communicate with a terminal 101 such as a PC via the Internet 103 or the like. Note that in realizing this embodiment, the terminal 101 such as a PC is not necessarily required, and only the image processing apparatus 100 may be configured.
The image processing apparatus 100 is, for example, a multifunction peripheral (MFP) having a display/operation section, a scanner section, and a printer section, and can be used as a scanning terminal that scans a document using the scanner section. It also has a display/operation unit such as a touch panel and hard buttons, and displays a user interface for displaying file name and storage destination recommendation results and accepting instructions from the user.
In this embodiment, an example will be described in which the image processing apparatus 100 alone performs scanning processing, file division processing, OCR-related processing, recommendation content generation processing, and file transmission. It does not matter if the system is configured to do so. For example, first, the image processing apparatus 100 executes a scan process and a file division process, and transmits the scanned image to another terminal 101 via the network. Then, the other terminal 101 executes OCR-related processing and recommendation content generation processing, sends the recommendation results back to the image processing device 100, generates a file in the image processing device 100, and sends the file. good.

<Hardware configuration of image processing device>
FIG. 2 is a block diagram showing the configuration of the image processing device 100. In the control unit 110, a CPU 111, a storage device 112, a printer I/F unit 113, a network I/F unit 114, a scanner I/F unit 115, and a display/operation I/F unit 116 can communicate with each other via a system bus 117. It is connected to the. Control unit 110 controls the overall operation of image processing device 100.
The CPU 111 functions as a means for executing each process (reading control, image processing, display control, etc.) in the flowchart described later by reading and executing a control program stored in the storage device 112.
The storage device 112 stores and holds the programs, image data, metadata, setting data, processing result data, and the like. The storage device 112 includes a ROM 118 which is a nonvolatile memory, a RAM 119 which is a volatile memory, and an HDD 120 which is a large capacity storage area.
The ROM 118 is a nonvolatile memory that holds control programs, etc., and the CPU 111 reads the control programs and performs control.
The RAM 119 is a volatile memory used as a temporary storage area such as the main memory and work area of the CPU 111.
Network I/F section 114 connects control section 110 to LAN 102 via system bus 117. The network I/F unit 114 transmits image data to an external device on the LAN 102 and receives various information from the external device on the LAN 102.

Scanner I/F section 115 connects scanner section 122 and control section 110 via system bus 117. The scanner unit 122 reads a document, generates scanned image data, and inputs the scanned image data to the control unit 110 via the scanner I/F unit 115. Note that the scanner unit 122 includes a document feeder, and is capable of feeding a plurality of documents placed on a tray one by one and reading them continuously.
The display/operation unit I/F unit 116 connects the display/operation unit 123 and the control unit 110 via the system bus 117. The display/operation section 123 includes a liquid crystal display section with a touch panel function, hard buttons, and the like.
The printer I/F section 113 connects the printer section 121 and the control section 110 via the system bus 117. The printer unit 121 receives image data generated by the CPU 111 via the printer I/F unit 113, and performs printing processing on recording paper using the received image data.
As described above, the image processing apparatus 100 according to the present embodiment can provide image processing functions with the above hardware configuration.

<Processing procedure executed by the image processing device>
An overview of the processing in this embodiment is that the image processing apparatus 100 reads a plurality of documents and divides the image data into a plurality of files according to file division settings. Then, based on the index extraction rules, a file name to be given to each file is generated using the character string of the OCR result of the first page of each file, and the recommended content is presented to the user through the display/operation unit 123. . The user checks on the display/operation unit 123 whether the file name is correct and the recommended content is presented, and if the file name is incorrect, gives a correction instruction. Based on the user's operation results, the index extraction rule is updated if there is a modification. After the user has finished checking and correcting all the files after division, a file name is given to each file and sent.
Next, the processing procedure executed by the image processing apparatus 100 in this embodiment will be described in detail using the flowchart of FIG. Note that the CPU 111 of the image processing apparatus 100 functions as a processing unit that executes the processing of each step in FIG. 3 by loading a processing program stored in the ROM 118 into the RAM 119 and executing it.
In step S301, when the image processing apparatus 100 receives a user's instruction from the display/operation unit 123, the scanner unit 122 reads a plurality of documents one by one from the tray of the document feeder to obtain image data.
In step S302, the CPU 111 obtains file division settings from the storage device 112. Note that the file division settings may be obtained by receiving instructions from the user from the display/operation unit 123 in advance, or may be fixed settings determined in advance.

In step S303, the CPU 111 determines the division positions of the plurality of image data acquired in step S301 based on the file division settings acquired in step S302. Then, the image data acquired in S301 is grouped according to the division position. This group is image data that will be made into one file when creating a file, which will be described later, and one group includes at least one piece of image data. Therefore, the image data is divided to generate multiple files. Examples of file division settings and division position determination will be described later using FIG. 4.
In step S304, the CPU 111 starts the confirmation/correction process shown in step S309 in parallel, and advances the process to step S305.
After the OCR-related processing in steps S305 to S307 below is completed, the processing in step S309 is executed in parallel from the file to which the unconfirmed attribute has been added.
In step S305, the CPU 111 selects one of the files for which OCR-related processing has not been performed from among the image groups determined in step S303, in the order of earliest read.
In step S306, the CPU 111 executes OCR-related processing on the image of the first page of the file selected in step S305. Details of the OCR-related processing will be described later.
In step S307, the CPU 111 assigns an unconfirmed attribute to the file selected in step S305 and subjected to OCR-related processing in step S306. The unconfirmed attribute is an attribute that indicates that the confirmation/correction processing started in parallel in step S304 is pending.

In step S308, the CPU 111 determines whether the OCR-related processing in step S306 has been completed for all image groups determined in step S303. If OCR-related processing has not been completed for all image groups, the process returns to step S305, and if OCR-related processing has been completed for all image groups, the process proceeds to step S310.
On the other hand, in step S309, which is processed in parallel, the CPU 111 presents the user with file names as recommendation contents for all the image groups determined in step S303. Thereafter, a confirmation/correction process is executed to finalize the file name based on the user's confirmation/correction instructions. Details of the confirmation/correction process will be described later.
In step S310, the CPU 111 determines whether the confirmation/correction process has been completed for all image groups determined in step S303. The process waits until the confirmation/correction process is completed for all image groups, and if the process is completed, the process advances to step S311.
In step S311, the CPU 111 displays on the display/operation unit 123 a list of file names determined after checking and correcting in step S309.
In step S312, the CPU 111 creates a file from each of the image groups determined in step S303. Each file is given a file name determined after confirmation and modification by the user in step S309. In this embodiment, an example will be shown in which the data is converted into PDF (Portable Document Format) and saved. In the case of PDF, images can be saved in pages, and each image data within the image group determined in step S303 is saved as a separate page in one file.
In step S313, the CPU 111 transmits the file created in step S312 to the destination via the LAN 102.

<Split position determination process>
Next, the division position determination process in step S303 will be explained.
FIG. 4 shows a processing image of page division processing for each file division setting.
FIG. 4(A) is an example of division positions when page number division is set as the file division setting. Page number division is a method of determining division positions for each preset number of pages. The number of pages is specified by the user on the display/operation unit 123, or a fixed number of pages is specified in advance according to scan settings.
For example, if 6 pages of image data are acquired in step S301 and division into every 2 pages is set, the division position will be between the 2nd and 3rd pages, and between the 4th and 5th pages. is set to
FIG. 4(B) is an example where partition page division is set as the file division setting. Divider page division is a method in which image data is analyzed and when it is recognized as a divider page, that divider page is removed and the dividing position is set at the position where the divider page was. The user needs to insert a partition page between the documents to be divided, and then set the document group in the document feeder.
For example, in step S301, if six pages of image data are acquired and the third and fifth pages are partition pages, first the image data of the third and fifth pages are removed. Then, the division positions are set between the images of the second page and the fourth page, and between the images of the fourth page and the sixth page.
As the partition page, blank paper with nothing printed on it or paper with a barcode or two-dimensional code printed on it may be used. When using a blank page as a partition page, check whether the page is blank or not based on the number of pixels included in the image data that is less than a certain pixel value, or whether it contains colored pixel clusters that are larger than a certain area. The partition page is recognized by determining the . When using paper with a barcode or two-dimensional code printed on it, first, the barcode or two-dimensional code is detected from within the image data. Then, the partition page is recognized based on whether or not the result of decoding the detected barcode or two-dimensional code includes information that identifies it as a partition page. Partition page sheets using barcodes or two-dimensional codes may be printed using a specific PC application. Alternatively, an application within the image processing device 100 may be used to receive the number of sheets, paper size, etc. from the display/operation unit 123, and the printer unit 121 may print the information.

FIG. 4(C) is an example where cover page division is set as the file division setting. Cover page division is a method of analyzing image data and setting a division position between one page and the previous page when it is recognized as a cover page.
For example, if six pages of image data are acquired in step S301 and the first, third, and fifth pages are recognized as cover pages, the dividing position is between the second and third pages. It is set between the 4th and 5th pages. As a method for recognizing displayed pages, for example, image data of a cover page is registered in advance, and recognition is performed by comparing the images. The image data for the cover page may be registered by using an application within the image processing device 100 to read the document that will become the cover page with the scanner unit 122, or may be registered from the outside via the LAN 102 or the network I/F unit 114. You may. In image comparison, it is recognized whether the page is the same as the registered cover page based on the degree of matching of local feature amounts calculated from the images. Note that the recognition of the cover page is not limited to the comparison of images.
As another method, for example, layout features such as the arrangement of characters and ruled lines included in the document in the image may be extracted and recognized by comparing the layout features. Further, instead of registering the cover page in advance, recognition may be performed based on whether the cover page contains a specific character string, barcode, or two-dimensional code.

<OCR related processing>
Next, details of the OCR-related processing in step S306 will be explained using FIG. 5. FIG. 5 is a flowchart showing a processing procedure for executing OCR-related processing including OCR and its preprocessing on one piece of image data.
In step S501, the CPU 111 detects the angle of inclination from the image data, and rotates the image in the opposite direction by the detected inclination, thereby obtaining image data with the inclination corrected.
The skew that is subject to skew correction is the skew that causes the document to not be read straight when being read by the scanner unit 122 due to wear of the rollers inside the document feeder, or the document cannot be printed straight when printing the document. It is.
In tilt detection, first, objects included in image data are detected, and groups of objects that are adjacent in the horizontal or vertical direction are connected. Then, the inclination is determined by obtaining how much the angle connecting the center positions of the connected objects is inclined from the horizontal or vertical direction. Note that the tilt detection is not limited to the above method. For example, you can obtain the center coordinates of an object included in image data, rotate the center coordinates in 0.1 degree increments, and find the angle at which the center coordinates are aligned horizontally or vertically at the highest angle. good. By correcting the tilt of the image by this tilt correction, it becomes possible to improve the processing precision of rotation correction, block selection processing, and OCR, which will be described later.

In step S502, the CPU 111 obtains an image by correcting the rotation of the image obtained in step S501 by rotating it in units of 90 degrees so that the characters in the document are oriented upright. Using the tilt-corrected image obtained in step S501 as a reference image, four images are prepared: an image rotated by 90 degrees, an image rotated by 180 degrees, and an image rotated by 270 degrees from the reference image. Then, a simple OCR process that is capable of high-speed processing is executed on each image, and the image with the largest number of characters recognized with a confidence level higher than a certain value is obtained as the image after rotation correction. . Note that the rotation correction method is not limited to the above.
In step S503, the CPU 111 executes block selection processing on the rotationally corrected image obtained in step S502. The block selection process is a determination process that classifies an image into a foreground area and a background area, and then divides the foreground area into text blocks and other blocks.

Specifically, first, contour tracing is performed on a query image that has been binarized into black and white, and a cluster of pixels surrounded by a black pixel contour is extracted. Then, for a block of black pixels whose area is larger than a predetermined area, contour tracing is also performed on the white pixels inside, the block of white pixels is extracted, and the block of white pixels with an area larger than a certain area is further extracted. Recursively extract a block of black pixels from inside. The cluster of black pixels obtained in this way is used as a foreground region, and is classified by size and shape into regions having different attributes. For example, a pixel block with an aspect ratio close to 1 and a size within a certain range is defined as a pixel block corresponding to a character, and a portion where adjacent characters can be grouped in a well-aligned manner is defined as a character area (TEXT). Furthermore, a flat pixel block is defined as a line area (LINE). The range occupied by a black pixel block that is larger than a certain size and includes a well-aligned rectangular white pixel block is defined as a table area (TABLE). An area where irregularly shaped pixel clusters are scattered is defined as a photo area (PHOTO). Then, a pixel block of any other shape is defined as a picture area (PICTURE). In this way, blocks determined to have character attributes are specified as text blocks from among the regions divided for each attribute of the object.

FIG. 6 is a diagram showing an example of the result of block selection processing. FIG. 6(A) shows the image after rotation correction, and FIG. 6(B) shows the block selection processing result of the query image. The information on each text block (information indicating attributes and the position of each block) obtained in this step is used in OCR processing, similarity calculation, etc., which will be described later. Note that the reason why only text blocks are extracted in this step is that the position of the character string well represents the structure of the document image and is closely related to the scan assist information. Therefore, this does not preclude the use of information about blocks determined to have other attributes, such as photo areas and table areas, in subsequent processing.
Returning to the explanation of the flowchart in FIG. 5, in step S504, the CPU 111 performs OCR on the text blocks obtained in step S503, and obtains character strings corresponding to each text block as a result.

<Confirmation/correction process>
The details of the confirmation/correction process (S309 in FIG. 3) will be explained. In this embodiment, the confirmation/correction process (S309) is started in step S304 of FIG. 3, and operates in parallel with the process flow (S305 to S308) of FIG. The confirmation/correction process is a process in which the file name to be given to each divided file is presented to the user as a recommendation and is determined based on the user's instructions. FIG. 7 is a flowchart showing the processing steps of the confirmation/correction process. FIG. 8 shows the image data of the first page of each image group, and the order of arrangement in FIGS. 8(A), 8(B), and 8(C) is in the order of earliest reading. 8(A) and FIG. 8(B) are documents of the same type, but have partially different written contents, and FIG. 8(C) is a document of a different type. Below, the behavior of the confirmation/correction process when the image data of FIG. 8 is read will be explained using the flowchart of FIG. 7.

In step S701, the CPU 111 checks whether there is a group to which an unconfirmed attribute has been added among the image groups scanned in step S301 of FIG. 3 and determined in step S303. The unconfirmed attribute is an attribute assigned in step S307 in FIG. 3 and means that it is before the actual confirmation/correction process is executed. The process waits until an image group with an unconfirmed attribute appears, and if there is even one image group, the process advances to step S702.
In step S702, the CPU 111 selects one of the image groups to which the unconfirmed attribute has been added in the order of earliest reading.
In step S703, the CPU 111 acquires the index extraction rule from the HDD 120, develops it in the RAM 119, and acquires it. Examples of index extraction rules are shown in FIGS. 9A and 9B. FIG. 9A shows a state in which one type of document with document ID "0001" is registered, and FIG. 9B shows a state in which a document with document ID "0002" is also registered. The index extraction rule is a combination of document ID, document identification information, and recommendation information for one registered document, and these combinations are retained as many as the number of registered documents. The document ID is a unique ID representing the type of document. The document identification information is information necessary to identify the type of document used in document matching, which will be described later, and uses a group of text blocks obtained by executing OCR-related processing. Recommendation information is a rule for extracting the content to be recommended from an input document, and the coordinates of a specific text block group in a registered document and the OCR results of that text block group are connected in what order to create a file name. It has information on the order in which it should be done. In addition, in this embodiment, details of an example of recommending a file name will be described later, but the recommendation information includes folder information of the destination of the file and information for adding OCR results of a group of text blocks as metadata. May be retained. Furthermore, as shown in FIGS. 9A and 9B, thumbnails of the read image data may be held together.
In step S704, the CPU 111 executes recommendation content generation processing to generate a file name to be recommended for the image group selected in step S702, based on the index extraction rule acquired in step S703. FIG. 10 is a flowchart showing the procedure of the recommendation content generation process.

<Recommendation content generation process>
In step S1000, CPU 111 executes document matching. Document matching is to determine whether a document of the same type as the input document is among the registered documents registered in the database of the image processing device along with index extraction rules, and to check whether there is a document of the same type. In this case, the type of document is determined. In this example, first, the degree of similarity, which indicates how similar the shape and arrangement of text blocks obtained through OCR-related processing are, is calculated on a one-to-one basis between the input document and each registered document. conduct. In the similarity calculation, first, the entire text block of the input document is aligned with the entire text block of the registered document. Next, the square of the sum of the areas where each text block in the input document overlaps with the text blocks of the registered document is divided by the product of the sum of the text block areas of the input document and the sum of the text block areas of the registered documents. Let the value be the similarity. The above similarity calculation is performed between the input document and all registered documents, and if the highest similarity is greater than a certain value, it is determined that the document is of the same type as the registered document, and the highest similarity is fixed. If it is smaller than the value, it is determined that the inputted document is of a type that is not included in the registered documents. Note that document matching is not limited to the above method. For example, a group of character strings included in a document obtained as a result of OCR-related processing may be held as the document identification information, and matching may be performed based on the similarity. Alternatively, image data or image feature amounts obtained from the image data may be retained as document identification information, and similarity of the image feature amounts may be used.
In step S1001, if the input document is a registered document as a result of the document matching executed in step S1000, the CPU 111 advances the process to step S1002, and if it is not a registered document, Finish the process.
In step S1002, the CPU 111 gives the input document the same document ID as the document determined to be of the same type in step S1000.

In step S1003, the CPU 111 obtains recommendation information linked to the document ID of the document determined in step S1000 from among the index extraction rules, and generates a file name as recommendation content based on the rule. Specifically, first, a group of text blocks for recommendations in the input document corresponding to a group of text blocks for recommendations at positions set for each type of document is obtained. At this time, the position of the entire image may be shifted between the input document and the registered document depending on the scan timing, or the size of the text block may change depending on the written content. Therefore, after aligning the entire text block of the input document with the entire text block of the registered document, a group of recommendation text blocks in the input document that overlap with the recommendation text blocks in the registered document are obtained. Then, the result of connecting character strings obtained as the OCR result of a group of recommendation text blocks in the acquired input document is recommended as a file name.

<Confirmation/correction screen>
Returning to the explanation of the flowchart in FIG. 7.
In step S705 in FIG. 7, the CPU 111 displays a confirmation/correction screen on the operation/display unit 123. This confirmation/correction screen will be explained with reference to FIGS. 11A and 11B. In addition, below, FIG. 11A and FIG. 11B will be collectively referred to as FIG. 11.
In FIG. 11, the display/operation unit 123 includes a touch panel display area 1101 and hard keys 1102 (number keys, start keys for various processes, etc.). A preview image of the read image data of the input document is displayed in the touch panel display area 1101. The preview image can be scrolled in any direction by the user swiping the preview image. Also displayed are a text field 1103 for displaying and editing the file name, a delete button 1104 for deleting the file name, and buttons 1105 and 1106 for enlarging and reducing the display magnification of the preview image. Also displayed are a button 1107 for canceling the determination of the file name and returning to the previous file, and a button 1108 for determining the file name.
Here, if the document shown in FIG. 8(A) is selected in step S702 and the index extraction rule shown in FIG. 9A is obtained in step S703, in step S704, the selected document is an unregistered document. It is determined that Therefore, the file name that is the recommendation content is not generated, and the file name text field 1103 is displayed blank as shown in FIG. 11(A). On this confirmation/correction screen, the user can instruct the file name to be given to the file by touching the text block on the image data displayed as a preview. For example, as shown in FIG. 11(B), when a text block 1109 representing a sender company is touched, the text block 1109 is highlighted on the preview. Then, in the file name text field 1103, "ABCCo.", which is the text of the OCR result of the text block 1109, is displayed. Similarly, the results of sequentially touching the text block 1110 representing the document number and the text block 1111 representing the date are shown in FIGS. 11(C) and 11(D), respectively. In this way, if you touch multiple text blocks, the file name will be the result of connecting them with a predetermined separator, the underscore.
On the other hand, if a file name that is the recommendation content is generated in step S704, the text block used for the file name is highlighted and the file name is displayed from the beginning, as shown in Figure 11(D). A confirmation/correction screen will be displayed. Note that it is not always necessary to use the text of the OCR result as it is as the file name. For example, as shown in FIG. 11(D), it is assumed that the text "2019/04/01" is obtained as the OCR result of the text block 1111. This text data may be identified as a date in YYYY/MM/DD format and converted to "20190401" in YYYYMMDD format. If the user presses the determination button 1108 and determines the file name, the process advances to step S706.

In step S706, the CPU 111 creates an index extraction rule corresponding to the selected document based on the content instructed by the user through the confirmation/correction screen in step S705.
In step S707, the CPU 111 determines whether the user modified the file name generated in step S704 in step S705. If the recommended content has been modified, the process advances to step S708; if it has not been modified, the process advances to step S711. Note that if it is determined in step S704 that the selected document is not a registered document, the process proceeds to step S708, as in the case where the document has been modified.
In step S708, the CPU 111 determines whether a document ID has been assigned to the selected image group. If a document ID has been assigned, the process advances to step S709; if a document ID has not been assigned, the process advances to step S710.
In step S709, the CPU 111 updates the document identification information and recommendation information of the corresponding document ID based on the index extraction rule created in step S706, and advances the process to step S711.
In step S710, the CPU 111 issues a new document ID, combines it with the document identification information and recommendation information created in step S706, registers a new index extraction rule, and advances the process to step S711.
According to steps S706 to S708 and S710 above, when the user specifies a file name on the confirmation/correction screen (Fig. 11(B) to (D)), the index extraction rule corresponding to the document ID "0002" in Fig. 9B is newly registered.

In step S711, the CPU 111 changes the attribute of the currently selected image group from an unconfirmed attribute to a confirmed attribute.
In step S712, the CPU 111 determines whether the confirmed attribute has been assigned to all the image groups determined in step S303. If the confirmed attribute has been assigned to all image groups, the process ends; otherwise, the process returns to step S701.
A case will be described in which the images shown in FIGS. 8(B) and 8(C) are successively processed through the above-described confirmation/correction process.
First, when the document shown in FIG. 8(B) is selected in step S702, the updated index extraction rule shown in FIG. 9B is acquired in step S703. In step S704, it is determined that the document is a registered document (document ID "0002"), and the file name of the recommendation result is obtained. The confirmation/correction screen displayed in step S705 in this case is shown in FIG. 12(A). In this way, even if documents of the same type that were unregistered before scanning are scanned at the same time, the user can specify the file name for one document on the confirmation/edit screen, and the other document can be scanned at the same time. can recommend file names.

Next, a case will be described in which the document shown in FIG. 8(C) is selected in step S702, and the updated index extraction rule shown in FIG. 9B is acquired in step S703. In step S704, it is determined that the selected document is a registered document (document ID "0001"), and a file name is recommended based on the previously registered index extraction rule. The confirmation/correction screen displayed in step S705 in this case is shown in FIG. 12(B). In this way, even if different types of files are scanned at the same time, it is possible to recommend file names according to each document type based on previously registered index extraction rules.
As described above, according to the first embodiment, the user's effort can be reduced in the function of dividing files when a plurality of documents are scanned and recommending index items such as file names for each document. Furthermore, even if there are multiple documents of the same type in the same scan, rules registered and modified on the spot can be applied. Furthermore, by first performing OCR-related processing on all documents at the backend where the user is checking and making corrections, the user's waiting time can be greatly reduced.

In the first embodiment, the back end of the confirmation/correction process executes the OCR-related process of step S306 in FIG. 3 for all image groups, and before displaying the confirmation/correction screen for each image group, the OCR-related process shown in step S704 of FIG. 7 is executed. An example of executing recommendation content generation processing has been described. In the second embodiment, the back end of the confirmation/correction process not only performs OCR-related processing but also generates recommendation contents. Then, before displaying the confirmation/correction screen for each image group, it is determined whether or not the recommendation content needs to be regenerated, and only when regeneration is necessary, the index extraction rule regeneration process is performed. explain. In addition, below, only the portions that are different from the first embodiment will be explained.
FIG. 13 is a flowchart of the processing procedure according to this embodiment. First, the difference between FIG. 13 and the flowchart of the first embodiment shown in FIG. 3 will be explained.
After starting the confirmation/correction process in parallel in step S304, in step S1301, the CPU 111 acquires the first index extraction rule from the HDD 120, develops it in the RAM 119, and acquires it. In this embodiment, the first index extraction rule is read only once and applied once to each of the image groups divided in step S303. This is an index extraction rule used in the recommendation content generation process in step S1302, which will be described later.
After OCR-related processing is performed in steps S305 and S306, the CPU 111 executes recommendation content generation processing in step S1302. Here, the CPU 111 generates a file name to be recommended for the image group selected in step S305 based on the first index extraction rule obtained in step S1301. The details of the recommendation content generation process are the same as those explained using FIG. 10 in the first embodiment.

In step S1303 of parallel processing, the CPU 111 recommends file names to the user for all image groups divided in step S303, and performs confirmation/correction processing to determine the file names based on the user's confirmation/correction instructions. . Details of the confirmation/correction process will be described later.
<Detailed explanation of confirmation/correction process>
Details of the confirmation/correction process shown in S1303 in FIG. 13 will be explained using the flowchart in FIG. 14. In this embodiment, the confirmation/correction process is started in step S304 of FIG. 13 and operates in parallel with the process flow of FIG. 13. Note that only the differences between FIG. 14 and the flowchart of the first embodiment shown in FIG. 7 will be explained.
In step S1401, the CPU 111 creates a second index extraction rule and deploys it in the RAM 119. The second index extraction rule means that when the first index extraction rule obtained in step S1301 of FIG. 13 is updated or newly registered through confirmation/correction processing, the updated state is temporarily This is an index extraction rule that is maintained. In addition to the first index extraction rule, the second index extraction rule has update information such as whether or not it has been updated and whether it has been newly registered as an attribute. The initial state of the second index extraction rule is the same as the document ID, document identification information, and recommendation information that are the contents of the first index extraction rule obtained in step S1301, and the initial update information is the one without update. state. Examples of the second index extraction rule are shown in FIGS. 15A and 15B. FIG. 15A shows a state generated based on the first index extraction rule in which one type of document with document ID "0001" is registered. FIG. 15B shows a state in which an index extraction rule corresponding to document ID "0002" has been newly registered.

In step S1402, the CPU 111 obtains the second index extraction rule developed in the RAM 119.
In step S1403, the CPU 111 executes a second recommendation content generation process based on the second index extraction rule acquired in step S1402. Details of the second recommendation content generation process will be described later.
After steps S705 and S706, if the second index extraction rule is modified (S707) and there is a document ID (S708), proceed to step S1404, and if there is no document ID, proceed to S1405. Proceed.
In step S1404, the CPU 111 updates the document identification information and recommendation information of the corresponding document ID based on the second index extraction rule created in step S706, and holds the updated information as updated.
In step S1405, the CPU 111 issues a new document ID, combines it with the document identification information and recommendation information created in step S706, newly registers it in the second index extraction rule, and maintains the updated status as new registration.
When the confirmation/correction process for all groups is completed, in the subsequent step S1406, the CPU 111 reflects the second index extraction rule in the first index extraction rule and stores it in the HDD 120.

<Detailed explanation of the second recommendation content generation process>
FIG. 16 shows a flowchart of the second recommendation content generation process.
In step S1601, the CPU 111 determines whether the image group selected in step S702 in FIG. 14 has been determined as a registered document in the recommendation content generation process in step S1302 in FIG. 13 and has been assigned a document ID. do. If no document ID is assigned, that is, if the document image is not registered in the second index extraction rule, the process advances to step S1602, and if a document ID is assigned, the process advances to step S1607.
In step S1602, the CPU 111 determines whether there is a new rule in the update information attribute among the second index extraction rules acquired in step S1402 of FIG. If there is a new rule, the process advances to step S1603, and if there is no new rule, the process ends. As a result, if there is no new rule, the result of the recommendation content generation process in step S1302 in FIG. 13 is recommended in step S705 in FIG. 14, and the time required to display the confirmation/correction screen can be reduced.
In step S1603, the CPU 111 executes document matching between the currently selected document and only the newly registered document of the second index extraction rule. Generally, the processing time for document matching increases depending on the number of registered documents to be matched, so by limiting the matching to only newly registered documents, processing time can be reduced. can.
In step S1604, if the input document is a newly registered document as a result of the document matching executed in step S1603, the CPU 111 advances the process to step S1605; otherwise, the process proceeds to step S1605. end. If the document is not a newly registered document, the file name that is the recommendation content is not generated, and the user's instruction for the file name is accepted on the confirmation/display screen.

In step S1605, the CPU 111 gives the input document the same document ID as the document determined to be of the same type as a result of the document matching in step S1603.
In the following step S1606, the CPU 111 obtains recommendation information associated with the document type determined in step S1603 from the second index extraction rule, and generates a file name as recommendation content based on the rule. do.
On the other hand, if there is a document ID in the image group, in step S1607, the CPU 111 refers to the update information linked to the document ID given to the input document in the second index extraction rule, and determines whether there is an update. judge. If there is an update, the process advances to step S1608, and if there is no update, the process ends. As a result, if there is no update, the result of the recommendation content generation process in step S1302 in FIG. 13 is recommended in step S705 in FIG. 14, and the time required to display the confirmation/correction screen can be reduced. Furthermore, even if there is an update, there is no need to perform document matching, and only the recommendation content needs to be generated in step S1608, which will be described later, so the time required to display the confirmation/correction screen can be reduced.
In step S1608, the CPU 111 acquires recommendation information linked to the document ID of the input document from the second index extraction rule, and generates a file name as recommendation content based on the recommendation information.
As described above, according to the second embodiment, the back end that performs confirmation and correction by the user first executes the process of generating the recommendation content once for all documents, and then regenerates the recommendation content as necessary. . As a result, by reducing the time required to display the confirmation/correction screen for each document, the user's waiting time can be further reduced significantly compared to the first embodiment.

(Other examples)
The present invention provides a system or device with a program that implements one or more of the functions of the embodiments described above via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. This can also be achieved by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Claims (33)

When a series of multiple originals set by a user is continuously scanned by a scanner , the image data consisting of multiple pages obtained by the scanning is processed without waiting for the user's operation on the image data. a control means that starts character recognition processing in order from the first page of the plurality of pages ;
A screen that displays a character string corresponding to the result of the character recognition process for the image data of the first page of the image data, the screen being capable of accepting correction instructions from the user for the character string. Providing means for providing the screen;
A processing program that causes a computer to function as
While the correction screen for the image data of the first page provided by the providing means is displayed, the character recognition process for the image data of the page subsequent to the first page is executed by the control means.
A processing program characterized by: 2. The character recognition process for the image data of the subsequent page is executed by the control means regardless of whether an operation from the user is accepted on the correction screen for the image data of the first page . The processing program described in . 3. The processing program according to claim 1, wherein the providing means provides the modified screen for the first page in response to completion of the character recognition process for the first page. In response to the completion of the character recognition process for the first page, the providing means may perform the modification for the first page while there is a page for which the character recognition process has not been performed at all among the plurality of pages. 3. The processing program according to claim 1, wherein the processing program provides a screen. The providing means provides the corrected screen for the first page before the character recognition process for the remaining pages of the plurality of pages is completed in response to completion of the character recognition process for the first page. The processing program according to claim 1 or 2, characterized in that: 6. The processing program according to claim 1, wherein the image data of the first page and the image data of the subsequent page are input by one action by the user. 7. The processing program according to claim 1 , wherein the providing means provides the modified screen on a display of the computer. 7. The processing program according to claim 1 , wherein the correction screen is displayed on a display of a device other than the computer. The character string corresponding to the result of the character recognition process displayed on the correction screen includes a character string indicating the file name of the file related to the image data of the first page , a character string indicating the folder in which the file is saved, and , at least one of a character string indicating a result of the character recognition process. 10. The processing program according to claim 1, wherein the correction instruction target is displayed on the correction screen alongside a preview image of the image data of the first page . a management unit that manages positional information of a character area corresponding to the result of the character recognition process used for the character string displayed on the correction screen in association with the type of document;
Identification means for identifying the type of manuscript of each page based on the result of image processing on the image data consisting of the plurality of pages;
It further has
On the correction screen, the character string is displayed based on the result of character recognition processing corresponding to a character area located at a position indicated by the position information that is managed in association with the type of document on the first page . The processing program according to any one of claims 1 to 10 , characterized in that: An information processing device that executes a processing program,
When a series of multiple originals set by a user is continuously scanned by a scanner, the processing program processes the image data made up of multiple pages obtained by the scanning by the user . a control means that starts character recognition processing sequentially from the first page of the plurality of pages without waiting for an operation ;
A screen that displays a character string corresponding to the result of the character recognition process for the image data of the first page of the image data, the screen being capable of accepting correction instructions from the user for the character string. providing means for providing a screen using the processing program;
has
While the correction screen for the image data of the first page provided by the providing means is displayed, the character recognition process for the image data of the page subsequent to the first page is executed by the control means.
An information processing device characterized by: 12. The character recognition process for the image data of the subsequent page is executed by the control means regardless of acceptance of an operation from the user on the correction screen for the image data of the first page. The information processing device described in . 14. The information processing apparatus according to claim 12 , wherein the providing means provides the modified screen for the first page in response to completion of the character recognition process for the first page. In response to the completion of the character recognition process for the first page, the providing means may perform the modification for the first page while there is a page for which the character recognition process has not been performed at all among the plurality of pages. The information processing device according to claim 12 or 13 , wherein the information processing device provides a screen. The providing means provides the corrected screen for the first page before the character recognition process for the remaining pages of the plurality of pages is completed in response to completion of the character recognition process for the first page. The information processing device according to claim 12 or 13 , characterized in that: 17. The information processing apparatus according to claim 12 , wherein the image data of the first page and the image data of the subsequent page are input by one action by the user. 18. The information processing apparatus according to claim 12 , wherein the providing means provides the modified screen on a display of the information processing apparatus. The information processing device according to claim 12 , wherein the correction screen is displayed on a display of a device other than the information processing device. The character string corresponding to the result of the character recognition process displayed on the correction screen includes a character string indicating the file name of the file related to the image data of the first page , a character string indicating the folder in which the file is saved, and , at least one of a character string indicating a result of the character recognition process. 21. The information processing apparatus according to claim 12 , wherein the correction instruction target is displayed on the correction screen alongside a preview image of the image data of the first page . a management unit that manages positional information of a character area corresponding to the result of the character recognition process used for the character string displayed on the correction screen in association with the type of document;
Identification means for identifying the type of manuscript of each page based on the result of image processing on the image data consisting of the plurality of pages;
It further has
On the correction screen, the character string is displayed based on the result of character recognition processing corresponding to a character area located at a position indicated by the position information that is managed in association with the type of document on the first page . The information processing device according to any one of claims 12 to 21 . A method for controlling an information processing device that executes a processing program, the method comprising:
When a series of multiple originals set by a user is continuously scanned by a scanner, the processing program processes the image data made up of multiple pages obtained by the scanning by the user . a control step that starts character recognition processing sequentially from the first page of the plurality of pages without waiting for the operation ;
A screen that displays a character string corresponding to the result of the character recognition process for the image data of the first page of the image data, the screen being capable of accepting correction instructions from the user for the character string. a providing step of providing the screen by the processing program;
has
While the correction screen for the image data of the first page provided in the providing step is displayed, the character recognition process for the image data of the page subsequent to the first page is executed by the control step.
A method for controlling an information processing device, characterized in that: 23. The character recognition process for the image data of the subsequent page is executed by the control step regardless of acceptance of an operation from the user on the correction screen for the image data of the first page. A method for controlling an information processing device according to . Control of the information processing apparatus according to claim 23 or 24, wherein in the providing step, the correction screen for the first page is provided in response to completion of the character recognition process for the first page. Method. In the providing step, in response to the completion of the character recognition process for the first page, while there is a page for which the character recognition process has not been performed at all among the plurality of pages, the correction for the first page is performed. 25. The method for controlling an information processing apparatus according to claim 23 or 24 , further comprising providing a screen. In the providing step, in response to completion of the character recognition process for the first page, and before the character recognition process for the remaining pages of the plurality of pages is completed, the modified screen for the first page is provided. 25. The method of controlling an information processing apparatus according to claim 23 or 24 , characterized in that: 28. The method of controlling an information processing apparatus according to claim 23 , wherein inputting the image data of the first page and the image data of the subsequent page is performed by one operation by the user. . 29. The method for controlling an information processing apparatus according to claim 23 , wherein in the providing step , the modified screen is provided on a display of the information processing apparatus. 29. The method for controlling an information processing device according to claim 23 , wherein the correction screen is displayed on a display of a device other than the information processing device. The character string corresponding to the result of the character recognition process displayed on the correction screen includes a character string indicating the file name of the file related to the image data of the first page , a character string indicating the folder in which the file is saved, and , at least one of a character string indicating a result of the character recognition process. 32. The control of the information processing apparatus according to claim 23 , wherein the modification screen displays the target of the modification instruction alongside a preview image of the image data of the first page. Method. a management step of managing position information of a character area corresponding to the result of the character recognition process used for the character string displayed on the correction screen in association with the type of manuscript;
an identification step of identifying the type of manuscript of each page based on the result of image processing on the image data consisting of the plurality of pages;
It further has
On the correction screen, the character string is displayed based on the result of character recognition processing corresponding to a character area located at a position indicated by the position information that is managed in association with the type of document on the first page . The method for controlling an information processing apparatus according to any one of claims 23 to 32 .

Images