JP2024040612A - Information processing device, method for controlling information processing device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enabling facilitating troublesome correction of an automatically extracted character string when scanning and computerizing a sheet document and setting a file name of a document image computerized by using a naming rule of a preliminarily set file name and the character string automatically extracted from written contents of the document on the basis of the naming rule.

SOLUTION: According to the invention, when correcting a file name generated from a scanned document, a method and program enable facilitating confirmation and selection of a character area to be used for correcting the file name from a preview image of the document by controlling an information processing device to: display the preview image together with the file name (Fig. 6); and perform highlighting of a candidate character string corresponding to an attribute of a correction place and a candidate character string display of a character recognition processing result of the character string (S901 to S907).

SELECTED DRAWING: Figure 9

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to an information processing device, a method of controlling the information processing device, and a program.

2. Description of the Related Art Conventionally, there is a system for managing a form by scanning a paper form, digitizing it, and setting a file name of the digitized form based on the written contents of the form.
Patent Document 1 discloses a technique for automatically extracting a character string from a scanned form and automatically generating a file name according to a file naming rule set in advance.

For example, assume that the attribute "document type", the delimiter "underbar", and the attribute "company name" are set as the file naming rule of Patent Document 1. In this case, from the results of character recognition processing performed on the scanned image data, "estimate" and "AACompany" are extracted as character strings related to the attribute "document type" and "company name", respectively, and the file name is Generate “Quote_AACompany” and display it on the screen. When modifying the automatically extracted file name, if you select the part you want to modify, other character string candidates in the form that correspond to the attributes defined in the naming rule for that part will be displayed, and you can select and modify them. It becomes. For example, if you select "AACompany", the attribute is "company name" according to the naming rule, so other character strings "BBCompany" and "CCCompany" corresponding to "company name" in the form are displayed.

JP 2021-140328 Publication

However, in the technique of Patent Document 1, only character strings are displayed when displaying candidates, so it is not possible to easily confirm which character area in the form has been used. Furthermore, if there is an error in the character recognition processing result, it will be necessary to correct it again after selecting a candidate. As described above, in the conventional technology, operations for checking and selecting character areas when modifying a file name generated from a scanned form are complicated.

The present invention has been made to solve the above problems. An object of the present invention is to provide a mechanism that allows easy confirmation and selection of character areas to be used when modifying a file name generated from a scanned form.

The present invention performs character recognition on a character region included in a document image, and combines character strings corresponding to attributes specified by a predetermined rule from among the character strings resulting from the character recognition. a first display means for displaying information regarding a file together with a preview display of the document image; a second display means for highlighting, in the preview display, an area in the document image corresponding to another character string having the same attribute as the attribute corresponding to the attribute; and a plurality of candidate characters of character recognition results for the highlighted area. a third display means for displaying a list of columns; and replacing the character string selected in the displayed information regarding the file with a candidate character string selected by the user from among the plurality of candidate character strings displayed in the list. and fourth display means for displaying.

According to the present invention, when modifying a file name generated from a scanned form, it is possible to easily confirm and select the character area to be used.

FIG. 1 is a diagram showing the overall configuration of an image processing system according to an embodiment. FIG. 1 is a diagram showing the hardware configuration of an MFP according to the present embodiment. FIG. 2 is a diagram showing a hardware configuration of an information processing device that can configure a client PC and an MFP cooperation service according to the present embodiment. FIG. 1 is a diagram showing a software configuration of an image processing system according to an embodiment. FIG. 3 is a sequence diagram showing the flow of processing between devices in this embodiment. The figure which shows the file name setting screen in this embodiment. The figure which shows the file name naming rule setting screen in this embodiment. 5 is a flowchart showing image analysis processing according to the present embodiment. 7 is a flowchart showing file name modification processing in the first embodiment. FIG. 3 is a diagram showing a screen for file name modification processing in the first embodiment. 7 is a flowchart showing file name modification processing in the second embodiment. FIG. 7 is a diagram showing a screen for file name modification processing in the second embodiment.

EMBODIMENT OF THE INVENTION Hereinafter, the form for implementing this invention is demonstrated using drawings. Note that the following embodiments do not limit the claimed invention, and not all combinations of features described in the embodiments are essential to the solution of the invention.

[First embodiment]
<Overall configuration>
FIG. 1 is a diagram showing an example of the overall configuration of an image processing system according to an embodiment of the present invention.
The image processing system of this embodiment includes an MFP (Multifunction Peripheral) 110, a client PC 111, an MFP cooperation service 120, and a cloud storage service (hereinafter referred to as "cloud storage") 130.

The MFP 110 and the client PC 111 are communicably connected to a server that provides various services on the Internet via a LAN (Local Area Network).
The MFP 110 is a multifunction device having multiple functions such as a scanner and a printer, and is an example of an image processing device.
The client PC 111 is a computer or an application program that receives services requested from the MFP cooperation service 120.

The MFP cooperation service 120 is an example of a service that has a function of storing an image file scanned by the MFP 110 on its own server or transferring it to a service capable of storing files, such as another storage service.
Cloud storage 130 is a service that allows files to be saved via the Internet and files to be obtained using a web browser.

The image processing system of this embodiment has a configuration including an MFP 110, a client PC 111, an MFP cooperation service 120, and a cloud storage 130, but is not limited thereto. For example, the MFP 110 may also serve as the client PC 111 and the MFP cooperation service 120. Furthermore, the MFP cooperation service 120 may be connected to a server on a LAN instead of on the Internet. Further, the cloud storage 130 may be replaced with a mail server or the like, and the scanned image may be attached to an e-mail and sent.

<MFP hardware configuration>
FIG. 2 is a diagram illustrating an example of the hardware configuration of the MFP 110.
MFP 110 includes a control section 210, an operation section 220, a printer section 221, a scanner section 222, and a modem 223.

The control unit 210 includes the following units 211 to 219, and controls the overall operation of the MFP 110. The CPU 211 reads a control program stored in the ROM 212 and executes and controls various functions of the MFP 110 such as reading/printing/communication. The RAM 213 is used as a main memory of the CPU 211, a temporary storage area such as a work area, etc. In this embodiment, one CPU 211 uses one memory (RAM 213 or HDD 214) to execute each process shown in a flowchart described later, but the present invention is not limited to this. For example, each process may be executed by making multiple CPUs, multiple RAMs, or HDDs work together. The HDD 214 is a large capacity storage unit that stores image data and various programs. Note that the configuration may include other storage devices such as an SSD (Solid State Drive) instead of or in combination with the HDD (Hard Disk Drive).

The operation unit I/F 215 is an interface that connects the operation unit 220 and the control unit 210. The operation unit 220 includes a touch panel, a keyboard, and the like, and receives operations/inputs/instructions from the user.
The printer I/F 216 is an interface that connects the printer section 221 and the control section 210. Image data for printing is transferred from the control unit 210 to the printer unit 221 via the printer I/F 216 and printed on a recording medium. The scanner I/F 217 is an interface that connects the scanner section 222 and the control section 210. The scanner unit 222 reads a document set on a document table (not shown) or an ADF (Auto Document Feeder), generates image data, and inputs the image data to the control unit 210 via the scanner I/F 217 . The MFP 110 can print out (copy) the image data generated by the scanner unit 222 from the printer unit 221, and can also send a file or email.

Modem I/F 218 is an interface that connects modem 223 and control unit 210. The modem 223 performs facsimile communication of image data with a facsimile device on the PSTN.
The network I/F 219 is an interface that connects the control unit 210 (MFP 110) to the LAN. The MFP 110 uses the network I/F 219 to transmit image data and information to various services on the Internet, and to receive various information.

<Hardware configuration of client PC and MFP cooperation service>
FIG. 3 is a diagram illustrating an example of the hardware configuration of an information processing device that can configure the client PC 111 and the MFP cooperation service 120.
The information processing device that can configure the client PC 111 and the MFP cooperation service 120 includes a CPU 311, a ROM 312, a RAM 313, an HDD 314, and a network I/F 315.

The CPU 311 controls the overall operation by reading control programs stored in the ROM 312 and HDD 314 and executing various processes. The RAM 313 is used as a temporary storage area such as the main memory and work area of the CPU 311. The HDD 314 is a large capacity storage unit that stores image data and various programs. Note that a configuration may also be provided in which other storage devices such as an SSD are used instead of or in combination with the HDD.

Network I/F 315 is an interface that connects MFP cooperation service 120 to the Internet. The MFP cooperation service 120 and the cloud storage 130 receive processing requests from other devices (such as the MFP 110) via the network I/F 315, and transmit and receive various information.
In addition to the above configuration, the client PC 111 includes an output device such as a display, an input device such as a keyboard or a pointing device, and an interface unit for connecting these devices. Note that a device serving as both an input device and an output device, such as a display with a touch panel, may be provided. The client PC 111 may be a personal computer (PC), a tablet terminal, a smartphone, or the like.

Furthermore, the MFP cooperation service 120 may be realized by one computer or by multiple computers. For example, the MFP cooperation service 120 may be implemented using cloud computing technology. Further, the cloud storage 130 can also be realized using the same hardware configuration as the MFP cooperation service 120.

<Software configuration of image processing system>
FIG. 4 is a diagram showing an example of the software configuration of the image processing system according to this embodiment.
The MFP 110 is roughly divided into two parts: a native function section 410 and an additional function section 420.

Each unit included in the native function unit 410 is provided as standard in the MFP 110. On the other hand, additional function unit 420 is an application program (hereinafter referred to as "application") additionally installed in MFP 110. The additional function unit 420 is an application based on Java (registered trademark), and can easily add functions to the MFP 110. Note that other additional applications (not shown) may be installed in the MFP 110.

The native function section 410 includes a scan execution section 411 and an image data storage section 412. The additional function section 420 includes a display control section 421, a scan instruction section 422, and a cooperation service request section 423.

The display control unit 421 displays a UI screen for accepting user operations on a liquid crystal display unit having a touch panel function of the operation unit 220 of the MFP 110. For example, it displays UI screens such as input of authentication information for accessing the MFP cooperation service 120, scan settings, scan start operations, and preview screens.
The scan instruction unit 422 requests the scan execution unit 411 to perform a scan process along with scan settings according to a user instruction input via the UI screen.

The scan execution unit 411 receives a scan request including scan settings from the scan instruction unit 422. The scan execution unit 411 generates scan image data by reading the document placed on the document platen glass with the scanner unit 222 via the scanner I/F 217 in accordance with the scan request. The generated scan image data is sent to the image data storage unit 412. The scan execution unit 411 sends a scan image identifier that uniquely indicates the saved scan image data to the scan instruction unit 422. The scanned image identifier is a number, symbol, alphabet, or the like (not shown) for uniquely identifying an image scanned by the MFP 110.

The image data storage unit 412 stores the scanned image data received from the scan execution unit 411 in the HDD 214.
The scan instruction unit 422 acquires scan image data corresponding to the scan image identifier received from the scan execution unit 411 from the image data storage unit 412. The scan instruction unit 422 requests the cooperation service request unit 423 to instruct the MFP cooperation service 120 to process the acquired scan image data.

The cooperation service request unit 423 requests various processes to the MFP cooperation service 120. For example, requests such as login, analysis of scanned images, and transmission of scanned images are made. Exchanges with the MFP cooperation service 120 use protocols such as REST and SOAP, but other communication means may also be used.

The MFP cooperation service 120 includes a request control section 431, an image processing section 432, a cloud storage access section 433, a data management section 434, and a display control section 435.

The request control unit 431 is on standby in a state where it can receive a request from an external device. Upon receiving a processing request, the request control unit 431 instructs the image processing unit 432, cloud storage access unit 433, and data management unit 434 to perform processing as appropriate according to the request.

The image processing unit 432 performs image recognition such as character area analysis, OCR (Optical Character Recognition), automatic extraction processing based on set attributes (details will be described in S510), and image rotation and tilt correction. Perform processing and image processing.

The cloud storage access unit 433 requests processing from cloud storage. Cloud services generally expose various interfaces for saving files to and retrieving files from cloud storage using protocols such as REST and SOAP. The cloud storage access unit 433 operates the cloud storage using the published cloud storage interface.

The data management unit 434 holds user information, various setting data, etc. managed by the MFP cooperation service 120.
The display control unit 435 receives a request from a web browser running on another terminal (not shown) such as a PC or mobile connected via the Internet, and generates screen configuration information (HTML, CSS, etc.) necessary for screen display. )return it. The user checks user information registered in the MFP cooperation service 120 and changes settings for scanning via a screen displayed on a web browser.

Although FIG. 4 describes an example of a configuration in which the additional function unit 420 is installed in the MFP 110, the present invention is not limited to this configuration, and the function of the additional function unit 420 may be included in the client PC 111.

<Overall processing flow>
FIG. 5 is a sequence diagram illustrating the flow of processing between devices when an image scanned by the MFP 110 is converted into a file and transmitted to cloud storage. Here, we will mainly explain the exchange between each device. Although FIG. 5 shows that the MFP 110 interacts with the MFP cooperation service 120, the client PC 111 may perform the acquisition of analysis results, screen display, learning instructions, etc., which will be described later, instead of the MFP 110. Hereinafter, regarding processes executed by "MFP 110 or client PC 111," client PC 111 will be omitted and only "MFP 110" will be described.

In a normal state, the MFP 110 displays a main screen on the touch panel in which buttons for implementing each function provided are arranged.
By installing an additional application (hereinafter referred to as a "scan application") on the MFP 110 for transmitting scanned forms to cloud storage, buttons for using the functions of the application are displayed on the main screen of the MFP 110. Note that the scan application is an application that corresponds to the additional function section 420. When you press the button to use the function of the scan app, a screen for sending the scanned form to cloud storage is displayed, and the process shown in the sequence shown in FIG. 5 is performed.

In S501, the MFP 110 displays a login screen (not shown) for entering authentication information for accessing the MFP cooperation service 120. When the user enters authentication information (user name and password) on this login screen and presses the OK button, the MFP 110 transmits a login request including the user name and password to the MFP cooperation service 120 (S502).

In S503, the MFP cooperation service 120 verifies whether the user name and password included in the login request are correct, and if they are correct, returns an access token to the MFP 110. Various subsequent requests made from the MFP 110 to the MFP cooperation service 120 are made together with this access token, and the user to be processed can be specified using this information. User authentication is performed using generally known methods (Basic authentication, Digest authentication, authorization using OAuth, etc.).

When the login process is completed, the MFP 110 displays a scan setting screen (not shown) in S504. On this scan setting screen, the user makes settings related to reading various scans. Furthermore, when the user places a paper form to be scanned on the platen glass or ADF and presses the "Start Scan" button, the MFP 110 executes the scan in S505 and generates image data that digitizes the paper form. do.

Then, in S506, the MFP 110 transmits an image generated by the scanning process (scanned image) and a scanned image analysis request to the MFP cooperation service 120. All scanned images may be transmitted at this time, or only the analysis target may be transmitted first, and all scanned images may be transmitted later.

When the MFP cooperation service 120 receives the scan image analysis request, the image processing unit 432 of the MFP cooperation service 120 starts image analysis in S507. Thereafter, the MFP cooperation service 120 returns "processId", which is an identifier uniquely indicating the analysis requested to the MFP cooperation service 120, to the MFP 110 without waiting for the end of the image analysis process.

Furthermore, in S507, after receiving the request, the MFP cooperation service 120 performs image analysis processing in the image processing unit 432. In the image analysis process, the image processing unit 432 first analyzes a character area existing in the image in S508. Thereafter, in S509, the image processing unit 432 performs character recognition processing on the area analyzed in S508. Further, in S510, the image processing unit 432 automatically extracts a file name from the character string extracted by the character recognition process in S509, according to the attributes of a predetermined rule (a preset file name naming rule). . Note that details of the processing in S508 to S510 will be explained later with reference to FIG. 8.

In S511, the MFP 110 uses the "processId" received in the response in S506 above to periodically (for example, every several hundred milliseconds to several milliseconds) request image analysis for the "processId". Check the processing status. Although not shown in the figure, the process of S511 is continued until a response indicating the completion of image processing from the MFP cooperation service 120 is obtained (until the timing of S512).

When the MFP cooperation service 120 receives the request to confirm the processing status in step S511, it confirms the processing status of “processId” and returns a response to the MFP 110. In the response, a character string indicating the current processing status is stored in "status". For example, when "status" is "processing", it indicates that processing is being performed by the MFP cooperation service 120, and when "status" is "completed", it indicates that processing has been completed. Note that if the process fails, other statuses such as "failed" may be returned. Furthermore, the response when the process is completed (when the status is completed) includes information such as the results of analyzing the scanned image and scan settings as well as the status.
In S512, the MFP cooperation service 120 returns a processing completion response to the MFP 110.

When the MFP 110 detects a response indicating completion of processing, in step S513, the MFP 110 acquires result information from the URL where the result information included in the response is stored.
Next, in S514, the MFP 110 displays the file name setting screen 600 (FIG. 6) using the result information obtained in S513 above. For example, the MFP 110 accesses the URL where the above-mentioned result information is stored, obtains the file name setting screen 600 provided by the display control unit 435, and displays the file name setting screen 600. Details of the file name setting screen will be described later.

When the user checks the file name on the scanned form on the file name setting screen 600, performs file name setting operations as necessary, and presses the "Send" button 606, which will be described later, the MFP 110 sends a transmission request in S515. Detect.
Upon detecting the transmission request, the MFP 110 transmits the file name set as input information to the MFP cooperation service 120 in S516.

Upon receiving the file name, the request control unit 431 of the MFP cooperation service 120 requests the image processing unit 432 to generate a file in S517. The image processing unit 432 obtains information on the file format to be sent to the cloud storage 130 from the scan settings registered in the MFP cooperation service, and generates a file from the scanned image based on the settings.

In S518, the MFP cooperation service 120 transmits the file generated from the scanned image and its file name to the cloud storage 130.
The MFP 110 ends the process upon receiving the response for transmitting the input information.

Note that if there are multiple documents to be scanned, the user can repeatedly scan multiple copies of the document (repeatedly perform the processes of S504 to S513). Additionally, by instructing the user to set a fixed number of pages or dividing a scanned document using partition paper before scanning, multiple documents can be scanned at once and divided into multiple documents.

Further, the processing of S514 to S516 is performed by, for example, a user logging into the MFP cooperation service 120 from a web browser of the client PC 111 or a dedicated application. Then, a list of documents for which files have not been generated among the scanned documents corresponding to the logged-in user as described above is displayed. When the user selects one of the documents from this list of documents, the MFP cooperation service 120 sends a file name setting screen 600 to the web browser or dedicated application of the client PC 111, and selects the location where the file name is to be corrected. Selection shall be accepted.

FIG. 6 is a diagram showing an example of a file name setting screen 600.
A file name area 601 is an area for displaying file names, and displays file names generated based on preset file name naming rules and analysis results of scanned images. Note that the file name naming rules are set on a file name naming rule setting screen as shown in FIG. Here, the file name naming rule will be explained using FIG. 7.

FIG. 7 is a diagram showing an example of a file name naming rule setting screen in this embodiment.
For example, when the user logs into the MFP cooperation service 120 from a web browser of the MFP 110 or the client PC 111 and selects file name naming rule setting, the display control unit 435 returns the file name naming rule setting screen 700 shown in FIG. As a result, a file name naming rule setting screen 700 is displayed on the web browser of the MFP 110 or the client PC 111.

A file name naming rule setting screen 700 in FIG. 7 shows a display example of the screen after the rule is set. The file name naming rule setting screen 700 has four areas: a rule setting area 701 , a system token area 702 , a delimiter token area 703 , and an automatic extraction token area 704 . Each of the token areas 702 to 704 displays a plurality of tokens (components) that can be used in combination when setting a file name naming rule. The system token 702 displays tokens related to information obtainable from the system (user name, device name, time, date, etc.). The delimiter token area 703 displays tokens related to delimiters (underscore, hyphen, space) to be inserted between each token. The automatic extraction token 704 contains desired attributes (document type, document creation date, document number, issuer's company name, issuer's name, issuer's name) from the character string resulting from character recognition processing of the scanned image. Tokens are displayed for selecting character strings related to company name, name of issuer, amount, etc.).

On the file name naming rule setting screen 700, the user can set naming rules by dragging desired tokens from among the plurality of tokens displayed in the token areas 702 to 704 and dropping them onto the rule setting area 701 to arrange them. Can be set. A file name is generated by combining information corresponding to the tokens arranged in the rule setting area 701 in the order in which they are arranged. Further, the file name naming rule setting screen 700 is provided with a "save" button 705 for confirming and saving the set rule. When the "Save" button 705 is pressed, the file name naming rule set on the file name naming rule setting screen 700 is finalized, transmitted to the MFP cooperation service 120, and saved.
The MFP cooperation service 120 generates a file name using the file name naming rule set using such a file name naming rule setting screen.

As shown in the example in Figure 7, if the naming rule includes the automatically extracted token attribute "document type", the delimiter token "underscore", and the automatically extracted token attribute "document number", , the MFP cooperation service 120 selects a character string corresponding to the attribute information of the automatically extracted token from the character recognition result of the scanned image and generates a file name.
For example, if "estimate" is extracted as a string corresponding to the attribute "document type" and "132343500" is extracted as a string corresponding to the attribute "document number", it will be combined with the delimiter "underscore". As a result, the file name “Quote_132343500” is generated. The generated file name is then displayed in the file name area 601 of the file name setting screen 600 in FIG.

The description of FIG. 6 will now be returned to.
If the character string automatically extracted based on the attribute information of the automatic extraction token is incorrectly extracted, the user first selects the incorrectly extracted character string displayed in the file name area 601 in order to correct the file name. and then select another string on the scanned image. For example, if you want to convert "132343500", which was automatically extracted as a document number, to a character string in a different area, proceed as follows. First, the user selects “132343500” to be corrected in the file name area 601, and then selects another character string image “E20019-01” on the form image (scanned image) displayed in the “preview area” 602. When selected, "132343500" in the file name area 601 can be modified to the character recognition result character string "E20019-01" corresponding to the selected character string image. Note that the details of the process when a modified character string is selected will be described later.
Furthermore, when a blank space on the right side of the file name area 601 is touched, a soft keyboard is displayed, and it is possible to modify the entire file name displayed in the file name area 601 and input additional arbitrary characters.

A "preview area" 602 displays a preview of the scanned image (form image). Furthermore, by touching a character area within this scanned image, a character recognition result character string corresponding to the character area corresponding to the touched position can be added to the file name. The character string used in the file name may be highlighted by adding a shape such as a line or frame line or color to the character area so that it can be seen that it has been selected. If multiple text areas are selected, each text area may be set to a different color. The character area selected on the corresponding preview area 602 and the character string in the file name area 601 may each be displayed in the same color or the same shaded frame to make the correspondence easier to understand. In the example of FIG. 6, the correspondence is displayed in an easy-to-understand manner by using the same shaded frame for the selected character area on the preview and the file name display area.

Furthermore, the preview display position may be changed or the enlargement ratio may be changed so that the selected character area is centered. Furthermore, when there are multiple character areas, the preview display position may be calculated so that a preset number of character areas are displayed. For example, change the display position and magnification so that the center of the top and bottom areas of the area used for the file name becomes the vertical center of the preview area, and then display the preview. I do. If you touch the selected text area again, the selection is canceled, the corresponding file name string is deleted, and the lines and colors added to the text area are returned to the state where they are not displayed. In the example, the text area is not displayed on the preview when the text string is not selected. However, in order to show the user which areas can be touched, the character areas may be displayed using colors or frames. Further, the display that makes the character area visible may be switched between display and non-display using a button or the like. By performing a swipe operation on the preview area, you can move the position of the image displayed in the preview area.

A "delete file name" button 603 deletes the characters corresponding to the character area added at the end of the file name. The "enlarge preview" button 604 increases the magnification of the image displayed in the preview area. A "preview reduction" button 605 reduces the magnification of the image displayed in the preview area. When enlarging and reducing, the display position is adjusted so that the coordinates of the center of the preview area are the same as before enlarging and reducing.
Furthermore, by pressing the "Send" button 606, the set file name is sent to the MFP cooperation service 120, and the file is generated and the file with the file name is sent. When the transmission is complete, the screen returns to the one before scanning.

FIG. 8 is a flowchart showing details of image analysis processing performed by the image processing unit 432 in this system. This flow corresponds to S507 to S510 in FIG.

First, in S801, the image processing unit 432 analyzes the character area of the input image and obtains a group of character areas in the form.
Next, in S802, the image processing unit 432 performs character recognition processing on the entire character area of the form acquired in S801 above, and extracts a character string.

After that, in S803, the image processing unit 432 automatically extracts a character string corresponding to the attribute information based on the attribute information of the automatic extraction token included in the preset file name naming rule.
For example, if "document type" is defined as attribute information, character strings corresponding to the document type, such as "estimate" or "invoice", are automatically extracted. Furthermore, if "company name (issuer)" is defined, a character string corresponding to the company name of the issuer in the form is automatically extracted.
In character recognition (OCR) processing, multiple candidate character strings are ranked and extracted for one character string image, and the highest ranked one is used as the character recognition result for that character string image. . Furthermore, if multiple character string candidates corresponding to the attribute information specified by the automatic extraction token are extracted, they are ranked and the highest ranked one is used as the file name. The second and lower candidates of the character recognition processing results and the second and lower candidates of the automatic extraction processing results corresponding to the attribute information are used when modifying the file name, which will be described later. Details will be described later.

From here on, the process of correcting automatically extracted file names performed on the file name setting screen of FIG. 6 will be explained in detail using FIGS. 9 and 10.
FIG. 9 is a flowchart showing details of the file name modification process in the first embodiment. The processing in this flowchart may be executed by the MFP 110 (or client PC 111) or by the MFP cooperation service 120. When executed by the MFP cooperation service 120, processing results in the MFP cooperation service 120 are sent to and displayed on the MFP 110, and operation information on the MFP 110 is sent from the MFP 110 to the MFP cooperation service 120.
First, a case where the MFP 110 executes the process shown in FIG. 9 will be described.

FIG. 10 is a diagram showing an example of a screen for file name modification processing in the first embodiment.
A flow of modifying the file name by changing the file name "132343500" in FIG. 10 to "E20019-01", another character string candidate in the form automatically extracted based on attribute information, will be specifically described.

When the user selects a character string to be modified from among the character strings automatically extracted from the file name displayed in the file name area 601, the MFP 110 accepts the selection operation in S901. For example, at this point, in the file name area 601 of the file name setting screen 600 in FIG. Displayed. Assume that the user selects the character string "132343500" in the file name area 601 on the file name setting screen 600 in FIG. 10 as the character string to be modified.

In S902, the MFP 110 checks from which attribute information the character string selected in S901 is extracted, and extracts the attribute information. For example, when the character string "132343500" is selected in FIG. 10, the character string is extracted based on the attribute "document number", so "document number" is acquired as the attribute information.

In S903, the MFP 110 checks whether there are any other automatically extracted candidates in the form based on the attribute information acquired in S902. For example, in S803 of FIG. 8, character strings in a form are ranked based on the attribute "document number" and from among the automatically extracted character string candidates, those other than the highest rank are determined as correction candidates. In the form of FIG. 10, it is assumed that the order of automatically extracted character strings based on the attribute "document number" is tax registration number 1003 and estimate number 1004. In this case, since the tax registration number 1003 is at the highest level and is used as a file name, the estimate number 1004 will be treated as a correction candidate.
If there are no other automatically extracted candidates (No in S903), the MFP 110 ends the process according to the selection of the file name modification target location.
On the other hand, if there are other automatically extracted candidates (Yes in S903), the MFP 110 advances the process to S904.

In S904, the MFP 110 controls to highlight the character string region of the automatic extraction candidate confirmed in S903 above. If there are multiple candidates, each is highlighted. Each may be highlighted with a different color or background to make it easier to recognize visually. If the area to be highlighted is outside the preview display, the preview display area may be reduced to highlight the candidate. At the same time, it controls highlighting of other character strings used in file names to be turned off.

FIG. 10 is an example in which quotation number 1004 "E20019-91", which is a candidate for automatic extraction, is highlighted. Note that at this time, the highlighting of the other character strings (“estimate”) forming the file name displayed in the file name area 601 is automatically turned off. What is highlighted is the character string to be corrected in the file name and character string candidates automatically extracted based on the attribute information of that character string. This makes it possible to easily grasp and select the positions of other candidates that are automatically extracted based on the attributes set in the file name naming rules.

In S905, the MFP 110 checks whether there are any other candidate character strings as a result of character recognition processing for the automatic extraction candidate character string highlighted in S904. In checking the candidates, it is checked whether there are other candidates as a result of the character recognition process for the highlighted character string image of the automatically extracted candidate from among the character recognition process results in S802 in FIG. For example, assume that "E20079-01,""E20019-01," and "E20079-07" are obtained as candidate character recognition processing results for estimate number 1004 "E20019-91."
If there are no candidates for character recognition processing for the character string that is an automatic extraction candidate (No in S905), the MFP 110 ends the processing according to the selection of the modification target portion of the file name.
On the other hand, if there are candidates for character recognition processing for the character string that is an automatic extraction candidate (Yes in S905), the MFP 110 advances the process to S906.

In S906, the MFP 110 displays the candidates for character recognition processing confirmed in S905 above in the region of the automatic extraction candidate character string highlighted in S904 above. As shown in FIG. 10, a candidate character string 1005 for character recognition processing is displayed for an estimate number 1004 of a candidate character string for automatic extraction. In FIG. 10, as an example, "E20079-01", "E20019-01", and "E20079-07" are displayed as a list as candidates for character recognition processing. If it overlaps the character area of the form, it may be displayed at a position that does not overlap the character area. The scanned user can select the appropriate string from the displayed list.

In the example of FIG. 10, the character string image is "E20019-01" on the previewed scanned image, but the first candidate as a result of character recognition processing is "E20079-01". The character recognition processing result is incorrect. If the first candidate of the character recognition result for the area of estimate number 1004 selected by the user is used as the file name when modifying the file name, "E20079-01" with the wrong character recognition result will be selected. It will be done. Therefore, in this embodiment, when the user selects a character string to be modified in the file name area 601, multiple character recognition results are displayed for character string images having the same attribute information as that of the selected character string. Display candidate strings. This allows the user to appropriately select the correct character string from among the displayed candidate character strings of the plurality of character recognition processing results. In this example, by selecting the second candidate "E20019-01" as the correct character string, it becomes possible to select an appropriate character string without having to modify it by direct editing. In addition, the area for the character string used for the file name is appropriate, but the character recognition processing result of the automatically extracted character string is incorrect, and in case you want to change it to another candidate, select it for the file name. Candidates for character recognition processing of the modified character string may be displayed. For example, in FIG. 10, "732343500" may be displayed as a candidate for character recognition processing in place of "132343500" in the previewed scanned image (not shown).

In S907, the MFP 110 reflects the character string selected by the user from the candidates displayed in S906 above in the file name. In this example, the file name "Estimate_132343500" becomes "Estimate_E20019-01." At this time, the highlighting of other character strings used in the file name (estimate) was turned off in step S904, but the highlighting is returned to the original highlighting after the file name is set.

Note that when the user performs similar operations as necessary on the file name setting screen 600 and presses the "send" button 606, the MFP 110 sends the file name set on the file name setting screen 600 to the MFP cooperation service 120. (corresponding to S516 in FIG. 5).

Next, a case will be described in which the process of the flowchart in FIG. 9 is executed by the MFP cooperation service 120. That is, the MFP cooperation service 120 transmits display information to the MFP 110, and the MFP 110 performs display based on the display information. It is also assumed that information on the operation received by the MFP 110 is transmitted to the MFP cooperation service 120 and processed by the MFP cooperation service 120.

When the user selects a character string to be modified from among the character strings automatically extracted from the file name displayed in the file name area 601, information on the operation is sent to the MFP cooperation service 120, and in S901, The MFP cooperation service 120 accepts the selection operation.
In S902, the MFP cooperation service 120 checks from which attribute information the character string selected in S901 is extracted, and extracts the attribute information.

In S903, the MFP cooperation service 120 checks whether there are any other automatically extracted candidates in the form based on the attribute information acquired in S902. If there are no other automatically extracted candidates (No in S903), the MFP cooperation service 120 ends the process for selecting the location to be modified in the file name.
On the other hand, if there are other automatically extracted candidates (Yes in S903), the MFP cooperation service 120 advances the process to S904.

In S904, the MFP cooperation service 120 controls to highlight the automatic extraction candidate character string area confirmed in S903 above.
In S905, the MFP cooperation service 120 checks whether there are other candidate character strings as a result of character recognition processing for the automatic extraction candidate character string highlighted in S904. If there are no candidates for character recognition processing for the character string that is an automatic extraction candidate (No in S905), the MFP cooperation service 120 ends the process for selecting the correction target portion of the file name.
On the other hand, if there are candidates for character recognition processing for the character string that is an automatic extraction candidate (Yes in S905), the MFP cooperation service 120 advances the process to S906.

In S906, the MFP cooperation service 120 displays the candidates for character recognition processing confirmed in S905 above in the area of the automatic extraction candidate character string highlighted in S904 above. The results of these controls are sent from the MFP cooperation service 120 to the MFP 110 and reflected on the file name setting screen.

In S907, the MFP cooperation service 120 reflects the character string selected by the user from among the candidates displayed in S906 above in the file name. The results of these controls are sent from the MFP cooperation service 120 to the MFP 110 and reflected on the file name setting screen.

When the user performs similar operations as necessary on the file name setting screen 600 and presses the "send" button 606, the MFP 110 sends the file name set on the file name setting screen 600 to the MFP cooperation service 120. (Corresponding to S516 in FIG. 5).

As described above, when you want to modify a character string that has been automatically extracted based on a preset file name naming rule, you can highlight the automatically extracted character string candidates and display the character recognition processing result candidates. This allows you to easily select and modify appropriate strings.

[Second embodiment]
In the first embodiment, character string candidates that are automatically extracted based on naming rules are highlighted, candidates for character recognition processing are displayed, and those with correct character recognition results are selected from among the candidates and automatically extracted. The configuration for modifying extracted character strings was explained. In the second embodiment, a configuration for correcting a character string with fewer steps when there is no correct candidate among the candidates for character recognition processing will be described with reference to FIGS. 11 and 12. Note that descriptions of processes similar to those in the first embodiment will be omitted.

FIG. 11 is a flowchart showing details of file name modification processing in the second embodiment. Similar to the flowchart shown in FIG. 9, the configuration may be executed by the MFP 110 (or client PC 111) or the MFP cooperation service 120.
Here, the description will be made assuming that the MFP 110 executes the process shown in FIG.
FIG. 12 is a diagram showing an example of a screen for file name modification processing in the second embodiment.

Processes from S1101 to S1105 in FIG. 11 are similar to those in the first embodiment, and when the character string "132343500" to be corrected in the file name in FIG. Highlight "E20019-91". At the same time, the MFP 110 turns off highlighting of other character strings used in the file name. Note that here, it is assumed that "E20079-01", "E20019-01", and "E20079-07" exist as candidates for character recognition processing of "E20019-91".

In S1106, the MFP 110 displays the character recognition processing candidates confirmed in S1105 and a button for further directly editing the character recognition candidates in the region of the automatic extraction candidate character string highlighted in S1104. conduct. For example, as shown in FIG. 12, a candidate character string 1205 for character recognition processing and a direct edit button 1206 are displayed for an estimate number 1204 of an automatic extraction candidate, thereby controlling the character recognition candidate to be further directly editable.

In FIG. 12, as an example, "E20079-01", "E20019-07", and "E20079-07" are displayed in a list as candidates for character recognition processing, and a direct edit button 1206 is displayed on the right side of each candidate. There is. In this example, none of the character strings resulting from the character recognition process is correct, so editing is required after selection, but by using the direct edit button, it is possible to reduce the steps required to edit. . The user can select the direct edit button and edit the candidate character string for character recognition processing to the left of the button. This makes it possible to directly modify each candidate for character recognition processing by selecting each button. Also, when displaying candidates for character recognition processing of a modified character string selected by file name, a direct edit button is similarly displayed.

The processing when a candidate for character recognition processing is selected in S1107 to S1108 is the same as in the first embodiment.
In S1109, if you want to further directly edit character recognition candidates, it is confirmed whether the direct edit button is selected. If the direct edit button is not pressed (No in S1109), the MFP 110 ends the process according to the selection of the file name modification target location.
On the other hand, if the direct edit button is pressed (Yes in S1109), the MFP 110 displays a candidate edit screen (not shown) for editing candidates for character recognition processing corresponding to the selected direct edit button, and edits the candidates. control possible. When editing is performed using the software keyboard displayed on the candidate editing screen and a confirmation operation is performed, the MFP 110 advances the process to S1110.

In S1110, the MFP 110 reflects the editing results on the candidate editing screen in the file name.
In the example of FIG. 12, if the user selects the edit button 1206 for "E20079-01" in the character recognition processing candidates 1205 and directly edits it to "E20019-01", the editing result is reflected and the file is The name "Estimate_132343500" is changed to "Estimate_E20019-01".

As a result, even if the candidates resulting from character recognition processing are not appropriate, by directly editing them using the edit button, it is possible to easily set an appropriate character string with fewer steps.
Note that a description of the case where the process of the flowchart shown in FIG. 11 is executed by the MFP cooperation service 120 will be omitted.
Further, in each of the above embodiments, a configuration has been described in which a file name of a document image is set on the screen of FIG. 6, but the configuration is not limited to setting a file, and a configuration in which file properties are set may be used.

In each of the embodiments described above, character regions included in a document image are recognized, character strings are extracted and selected for each attribute from the result, and a file name for saving the generated document image is set based on the file name naming rule. Displayed along with a preview of the document image. Then, in response to any character string being selected from this file name display, an area in the document image where another character string extracted with an attribute corresponding to the selected character string exists is detected. Highlighted in the preview image. Furthermore, candidates obtained by character recognition in the highlighted area are displayed in a list, and the candidates selected from the list are replaced with the character strings selected from the file name display and reflected in the file name display. As a result, when modifying a file name automatically generated from a scanned form, a preview image of the form is displayed, candidate character strings corresponding to the attributes of the modified part are highlighted, and candidates for character recognition processing for that character string are displayed. Displays strings. In this way, by displaying not only automatically extracted character string candidates for attribute information but also character string candidates resulting from character recognition processing, you can easily check and confirm character areas used when modifying file names from preview images. It becomes possible to make a selection.

It goes without saying that the configurations and contents of the various data described above are not limited to these, and may be configured in various configurations and contents depending on the use and purpose.
Although one embodiment has been described above, the present invention can be implemented as, for example, a system, an apparatus, a method, a program, a storage medium, or the like. Specifically, the present invention may be applied to a system consisting of a plurality of devices, or may be applied to a device consisting of a single device.
Moreover, all configurations that combine the above embodiments are also included in the present invention.

(Other embodiments)
The present invention provides a system or device with a program that implements one or more of the functions of the embodiments described above via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. This can also be achieved by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
Furthermore, the present invention may be applied to a system made up of a plurality of devices, or to a device made up of one device.
The present invention is not limited to the above embodiments, and various modifications (including organic combinations of each embodiment) are possible based on the spirit of the present invention, and these are excluded from the scope of the present invention. isn't it. That is, all configurations in which the above-described embodiments and their modifications are combined are also included in the present invention.

110 MFP
111 Client PC
120 MFP cooperation service 421 Display control unit 431 Request control unit 432 Image processing unit 434 Data management unit 435 Display control unit

Claims (6)

Information about the file of the document image is generated by character-recognizing a character area included in the document image, and combining character strings corresponding to attributes specified by a predetermined rule among the character strings resulting from the character recognition. , a first display means for displaying the document image along with a preview display;
In response to any character string being selected from the displayed information regarding the file, the character string in the document image corresponding to another character string having the same attribute as the attribute corresponding to the selected character string is selected. a second display means for highlighting a region in the preview display;
a third display means for displaying a list of a plurality of candidate character strings as character recognition results for the highlighted area;
a fourth display means for replacing and displaying the character string selected in the displayed information regarding the file with a candidate character string selected by the user from among the plurality of candidate character strings displayed in the list;
An information processing device comprising: comprising editing means for editing the plurality of candidate character strings displayed in the list;
The information processing according to claim 1, wherein the fourth display means replaces and displays the character string selected in the displayed information regarding the file with a candidate character string edited by the editing means. Device. 2. The information processing apparatus according to claim 1, further comprising control means for controlling the document image to be saved based on information regarding the displayed file. 4. The information processing apparatus according to claim 1, wherein the information regarding the file is a file name. A method for controlling an information processing device, the method comprising:
Information about the file of the document image is generated by character-recognizing a character area included in the document image, and combining character strings corresponding to attributes specified by a predetermined rule among the character strings resulting from the character recognition. , a first display step of displaying the document image together with a preview display;
In response to any character string being selected from the displayed information regarding the file, the character string in the document image corresponding to another character string having the same attribute as the attribute corresponding to the selected character string is selected. a second display step of highlighting the region in the preview display;
a third display step of displaying a list of a plurality of candidate character strings as character recognition results for the highlighted area;
a fourth display step of replacing and displaying the character string selected in the displayed information regarding the file with a candidate character string selected by the user from among the plurality of candidate character strings displayed in the list;
A method for controlling an information processing device, comprising: A program for causing a computer to execute the control method according to claim 5.

Images