JP2021027520A - Information processing device, information processing method, and program - Google Patents

Abstract

To provide an information processing device, an information processing method, and a program that more preferably achieve procedures using digitized documents.SOLUTION: An information processing device (MFP) of an image processing system includes: a scan execution unit that acquires an image of a scanned document such as a receipt; a determination unit that determines whether the image acquired by the scan execution unit contains information related to a user's signature; and a display control unit that displays a confirmation screen according to the result by the determination unit of determining whether the image contains information related to the signature.SELECTED DRAWING: Figure 4

Description

Information processing equipment, information processing methods, and programs.

Various techniques have been proposed for managing an image of a document such as a receipt as an electronic document by having a scanning device such as a scanner read the original. In particular, in recent years, with the development of image analysis technology, various technologies have been studied that enable information processing devices to recognize the contents described in this manuscript by analyzing the image according to the reading result of the manuscript. It has been For example, in Patent Document 1, by using a technique such as OCR (Optical Character Recognition), information on the date is extracted from the image according to the reading result of the manuscript and used for managing the electronic data of the manuscript. The technology is disclosed.

JP-A-2007-281719

Especially in recent years, by using the technology for managing the image of the manuscript as an electronic document as described above, the procedure using the actual document such as a paper medium has been changed to the procedure using the digitized document. The technology to be realized by is being studied. On the other hand, some of these procedures may require the applicant to sign in order to treat the subject document as a formal document. Against this background, the realization of technology that makes it possible to meet the requirements required for procedures using actual documents in a more suitable manner even in procedures using digitized documents is required. There is.

In view of the above problems, an object of the present invention is to support the realization of a procedure using digitized documents in a more preferable manner.

The information processing apparatus of the present invention includes an output control means for outputting to an output unit information according to whether or not the image obtained by reading the original contains information related to the user's signature.

According to the present invention, it is possible to support the realization of a procedure using an electronic document in a more preferable manner.

It is a figure which showed an example of the system structure of the image processing system which concerns on this embodiment. It is a figure which showed an example of the hardware configuration of the MFP. It is a figure which showed an example of the hardware configuration of an information processing apparatus. It is a figure which showed an example of the functional structure of an image processing system. It is a sequence diagram which showed an example of the processing of an image processing system. It is a sequence diagram which showed an example of the processing of an image processing system. It is a figure which showed an example of the main screen. It is a figure which showed an example of the login screen. It is a figure which showed an example of the format of the proxy application target person information. It is a figure which showed an example of the proxy application screen. It is a figure which showed an example of the scan setting screen. An example of the format of the response related to the return of the request ID is shown. An example of the format of the response related to the notification of the status of the analysis process is shown. An example of the format of the response related to the return of metadata information is shown. It is a figure which showed an example of the metadata setting screen. An example of the format of the request related to the metadata setting is shown. It is a flowchart which showed an example of the processing of the MFP. It is a flowchart which showed an example of the processing of the MFP. It is a flowchart which showed an example of the processing of the MFP. It is a flowchart which showed an example of the processing of the MFP. It is a figure which showed an example of the screen for presenting whether or not a receipt is valid. It is a flowchart which showed an example of the processing of the MFP. It is a flowchart which showed an example of the processing of the MFP. It is a figure which showed an example of the screen for presenting whether or not a receipt is valid.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. It should be noted that the following embodiments do not limit the invention according to the claims, and not all combinations of features described in the embodiments are essential for the means for solving the invention.

<First Embodiment>
The image processing system according to the first embodiment will be described below.

(System configuration)
FIG. 1 is a diagram showing an example of a system configuration of an image processing system according to the present embodiment. The image processing system according to the present embodiment includes an MFP (Multifunction Peripheral) 110, a service providing device 120, and an expense reimbursement device 130. The MFP 110 is connected to the Internet via a LAN (Local Area Network) and can communicate with a server that provides various services such as a service providing device 120 and an expense reimbursement device 130 connected to the Internet.

The MFP 110 is a multifunction device having a plurality of functions such as a scanner function and a printer function, and is an example of an information processing device according to the present embodiment. The service providing device 120 provides a function related to saving and transferring image data according to the reading result of the original by the MFP 110. For example, the service providing device 120 stores the image data transmitted from the MFP 110. Further, the service providing device 120 transfers the image data transmitted from the MFP 110 to another service capable of storing a file, such as a so-called storage service. The expense reimbursement device 130 provides a function of streamlining the work related to the expense reimbursement of a company. For example, the expense reimbursement device 130 stores the image data of the read receipt in association with additional information about the receipt as metadata. In addition, the expense settlement device 130 manages a user other than the person who permits the reading of the receipt (hereinafter, also referred to as a “proxy applicant”).

In the following, as shown in FIG. 1, the image processing system according to the present embodiment includes the MFP 110, the service providing device 120, and the expense settlement device 130, but the configuration of the image processing system according to the present embodiment is not necessarily the same. Is not limited to. For example, the MFP 110 may also play the role of the service providing device 120. Further, in FIG. 1, each device is connected by LAN and the Internet, but the configuration of the network connecting each device is not necessarily limited. For example, each device shown in FIG. 1 may be connected by a LAN, or may be connected by another network such as WAN (Wide Area Network). Further, the network connecting each device may be a wired network or a wireless network. Further, each device may be connected via a plurality of networks.

(Hardware configuration of MFP)
FIG. 2 is a diagram showing an example of the hardware configuration of the MFP. The MFP 110 includes a control unit 210, an operation unit 220, a printer 221, a scanner 222, and a modem 223. The control unit 210 includes a CPU (Central Processing Unit) 211, a ROM 212 (Read Only Memory), and a RAM 213 (Random Access Memory). Further, the control unit 210 includes an HDD (Hard Disk Drive) 214, an operation unit I / F215, a printer I / F216, a scanner I / F217, a modem I / F218, and a network I / F219.

The CPU 211 controls various operations of the MFP 110. For example, the CPU 211 may control the operation of the entire MFP 110. The ROM 212 stores a control program, a boot program, and the like that can be executed by the CPU 211. The RAM 213 is the main storage memory of the CPU 211, and is used as a work area or a temporary storage area for developing various programs. HDD 214 stores various data such as image data and various programs. As another example, the control unit 210 may include a storage device other than the HDD 214, such as a non-volatile memory represented by an SSD (Solid State Drive), in place of the HDD 214 or together with the HDD 214. May be good.

The CPU 211 expands the program stored in the ROM 212 or the HDD 214 into the RAM 213, and executes this program to perform various controls such as reading, printing, and communicating the original. Further, when the CPU 211 expands the program stored in the ROM 212 or the HDD 214 into the RAM 213 and executes the program, the functions of the MFP 110 shown in FIG. 4 and the functions of FIGS. 5A, 5B, 16A, 16B, 17A, and 17B The processing shown in is realized.

The operation unit I / F 215 is an interface for connecting the operation unit 220 to the control unit 210. The operation unit 220 plays a role as a reception unit that receives operations, instructions, and inputs from the user. In the present embodiment, the operation unit 220 includes a liquid crystal display unit having a touch panel function. However, as another example, the operation unit 220 may include another device such as a keyboard instead of the liquid crystal display unit having the touch panel function, or the keyboard or the like in addition to the liquid crystal display unit having the touch panel function. Other devices may be included.

The printer I / F 216 is an interface for connecting the printer 221 to the control unit 210. The control unit 210 transfers the image data to be printed by the printer 221 to the printer 221 via the printer I / F 216. The printer 221 prints the image data transferred from the control unit 210 on a print medium such as paper.

The scanner I / F 217 is an interface for connecting the scanner 222 to the control unit 210. The scanner 222 reads an image on the document to generate image data, and inputs the image data to the control unit 210 via the scanner I / F 217. The control unit 210 can transmit the image data generated by the scanner 222 to another device by file transmission or mail transmission.

The modem I / F 218 is an interface for connecting the modem 223 to the control unit 210. The modem 223 transmits / receives image data (facsimile communication) to and from a facsimile machine (not shown) via a PSTN (Public Switched Telephone Network).

The network I / F 219 is an interface for connecting the control unit 210 (MFP110) to the Internet via a LAN. By using the network I / F 219, the control unit 210 transmits image data and various information to an external device or system such as a service providing device 120 or an expense settlement device 130 connected to the Internet. In addition, the control unit 210 receives various data and various information transmitted from an external device or system by using the network I / F 219.

In the present embodiment, one CPU 211 uses one storage area (RAM 213 or HDD 214) to execute the processes shown in FIGS. 5A, 5B, 16A, 16B, 17A, and 17B. However, it does not necessarily limit the hardware configuration of the MFP 110. For example, each process may be executed by the cooperation of a plurality of CPUs and a plurality of storage areas.

(Hardware configuration of MFP cooperation service and expense settlement system)
FIG. 3 is a diagram showing an example of the hardware configuration of the information processing device 300 applied to realize the service providing device 120 and the expense reimbursement device 130. That is, in the present embodiment, each of the service providing device 120 and the expense settlement device 130 is implemented as an information processing device similar to the information processing device 300.

The information processing device 300 includes a control unit 310. The control unit 310 includes a CPU 311, a ROM 312, a RAM 313, an HDD 314, and a network I / F 315.

The CPU 311 controls various operations of the information processing device 300. For example, the CPU 311 may control the operation of the entire information processing device 300. The ROM 312 stores a control program, a boot program, and the like that can be executed by the CPU 311. The RAM 313 is the main storage memory of the CPU 311 and is used as a work area or a temporary storage area for developing various programs. The HDD 314 stores various data such as image data and various programs. The control unit 310 may include a storage device other than the HDD 314, such as a non-volatile memory typified by an SSD, in place of the HDD 314 or together with the HDD 314.

The CPU 311 expands the program stored in the ROM 312 or the HDD 314 into the RAM 313 and executes this program to realize various functions provided by the service providing device 120 and various processes executed by the service providing device 120.

The network I / F315 is an interface for connecting the control unit 310 (information processing device 300) to the LAN 100. By using the network I / F315, the control unit 310 transmits image data and various information to an external device such as the MFP 110 connected to the LAN 100, and receives various data and various information transmitted from the external device. Do.

In the following, it is assumed that one CPU 311 uses one storage area (RAM 313 or HDD 314) to execute processing related to various functions provided by the service providing device 120, but the hardware configuration of the information processing device 300 is not necessarily the same. Is not limited. For example, each process may be executed by the cooperation of a plurality of CPUs and a plurality of storage areas.

(Functional configuration of image processing system)
FIG. 4 is a diagram showing an example of the functional configuration of each component of the image processing system according to the present embodiment. The MFP 101 includes a native function unit 410 and an additional function unit 420. Each function included in the native function unit 410 indicates a function provided as standard in the MFP 101. On the other hand, each function included in the additional function unit 420 indicates a function realized by an application additionally installed on the MFP 101.
Each function included in the additional function unit 420 is realized by executing a program described in a program language such as Java (registered trademark). This makes it possible to easily add functions to the MFP 101 by using an API (Application Programming Interface) provided by a program language such as Java. The program language used to describe the program of the additional function unit 420 is not limited as long as the functions corresponding to the functions included in the additional function unit 420 can be realized. Further, the MFP 101 may include other functions (not shown in FIG. 4) as the additional function unit 420.

The native function unit 410 includes a scan execution unit 411 and an image data storage unit 412. The additional function unit 420 includes a display control unit 421, a scan instruction unit 422, a cooperation service request unit 423, and a determination unit 424.

The display control unit 421 causes the liquid crystal display unit having a touch panel function provided by the MFP 101 as the operation unit 220 to display a UI for receiving an operation from the user. Hereinafter, the liquid crystal display unit having the touch panel function provided by the MFP 101 as the operation unit 220 will also be referred to as a “display unit provided as the operation unit 220” for convenience.
For example, the display control unit 421 provides a UI for accepting an operation related to input of authentication information for accessing the service providing device 120 and an operation related to selection of an expense settlement target person when reading a document on behalf of the user. , May be displayed on the display unit provided as the operation unit 220. An example of this UI will be described later with reference to FIG.
Further, as another example, the display control unit 421 is provided as an operation unit 220 with a UI for receiving an operation related to designation of setting information related to reading a document, an operation related to starting reading of a document, and the like. It may be displayed in. An example of this UI will be described later with reference to FIG.
Further, the display control unit 421 is provided with a UI as the operation unit 220 for receiving an operation related to input of additional information such as a date and a type of the original, which is associated with the image data according to the reading result of the original. It may be displayed on the part. An example of this UI will be described later with reference to FIG. Further, hereinafter, the above additional information is also referred to as "metadata".
The scan instruction unit 422 requests the scan execution unit 411 to perform a process related to reading a document (hereinafter, also referred to as a "scan process") in response to an operation received from the user by the UI exemplified above. Further, at this time, the scan instruction unit 422 may transmit various setting information according to the input from the user to the scan execution unit 411 in association with the scan processing request.

Upon receiving a scan processing request from the scan instruction unit 422, the scan execution unit 411 instructs the scanner 222 to read (scan) the document via the scanner I / F 217. At this time, when the scan execution unit 411 associates the setting information related to reading the document with the request for scanning processing, the condition related to scanning the document to the scanner 222 based on this setting information. May be specified. The scan execution unit 411 acquires information according to the scanning result of the document from the scanner 222, and generates image data based on this information. The scan execution unit 411 outputs the generated image data to the image data storage unit 412. The scan execution unit 411 transmits an identifier (hereinafter, also referred to as a “scan image identifier”) for uniquely identifying the image data output to the image data storage unit 412 to the scan instruction unit 422. The image data storage unit 412 stores the image data output from the scan execution unit 411 in the HDD 214. Further, when the image data storage unit 412 receives a request for image data from the scan execution unit 411 or the like, the image data storage unit 412 acquires the image data according to the request from the HDD 214 and transmits the image data to the request source.

When the scan instruction unit 422 acquires the scan image identifier from the scan execution unit 411, the scan instruction unit 422 acquires the image data corresponding to the scan image identifier from the image data storage unit 412. The scan instruction unit 422 requests the cooperation service request unit 423 to apply the multi-crop process to the acquired image data. In the present disclosure, the multi-crop process detects a partial image of a document (an image showing a part of the image) included in the image to be processed, and detects the image from the image to be processed. It shows the process of extracting the partial image for each manuscript.

The cooperation service request unit 423 requests various processes from the service providing device 120. As a specific example, the linked service request unit 423 requests the service providing device 120 to log in, to send information about a proxy application target person who is permitted to settle expenses on behalf of the service provider 120, and multi-crop. Make various requests such as processing requests. Further, the cooperation service request unit 423 can also request the service providing device 120 to acquire information on the metadata associated with the various image data, set the metadata for the various image data, and the like.
The cooperation service request unit 423 uses a protocol such as SOAP (Simple Object Access Protocol) for transmitting and receiving information to and from the service providing device 120. Further, the cooperation service request unit 423 may realize the process related to the transmission / reception of information to / from the service providing device 120 based on an architecture such as REST (Representational State Transfer). Of course, the above is only an example, and does not necessarily limit the method or technique applied to the transmission / reception of information between the cooperation service request unit 423 and the service providing device 120. That is, as long as the linked service request unit 423 can send and receive information to and from the service providing device 120, the method and the applied technology are not particularly limited.

In the determination unit 424, the user who performs the operation related to the application for expense settlement based on the information specified by the user via the proxy application screen 900 described later with reference to FIG. 9 is the applicant and the proxy applicant. Determine which one.
In addition, the determination unit 424 determines whether or not the expiration date of this manuscript has passed, and whether or not this manuscript has been signed, based on the information contained in the image according to the reading result of the manuscript. Various judgments such as judgment of. The signature is at least one of the handwritten first and last name of the person. In this embodiment, the signature is the full name of the handwritten person. The signature is made by the corresponding person, a person acting on behalf of the person, or the like. Details of the processing related to the above determination by the determination unit 424 will be described later with reference to FIGS. 17A and 17B.

The service providing device 120 includes a request control unit 431, an image processing unit 432, an expense settlement request unit 433, a data management unit 434, and a display control unit 435.

The request control unit 431 receives requests for various processes from an external device, and depending on the content of the received requests, causes at least one of the image processing unit 432, the expense settlement request unit 433, and the data management unit 434 to relate to the request. Instruct processing.

The image processing unit 432 performs processing related to image analysis such as multi-crop processing, OCR (Optical Character Recognition), and layout analysis on the target image data.

The expense settlement request unit 433 requests various processes from the expense settlement device 130. As a specific example, the expense settlement request unit 433 obtains a list of proxy application target persons and a list of information indicating the type of expense (hereinafter, also referred to as "expense type") for the expense settlement device 130. Make a request such as acquisition of. Further, the expense settlement request unit 433 requests the expense settlement device 130 to register the image data of the receipt image according to the reading result of the receipt and to associate the metadata with the image data of the receipt image. Do.

The data management unit 434 stores information about the user managed by the service providing device 120, data related to various setting information, and the like in a storage area such as HDD 314 of the information processing device 300 that realizes the service providing device 120.

The display control unit 435 receives a request from a web browser operating on an external device (not shown) such as a PC or mobile terminal connected via the Internet, and screen configuration information used to display the screen. (HTML, CSS, etc.) is returned to this external device. With such a configuration, the user can confirm the user information registered in the service providing device 120 and instruct to change the setting information related to reading the original document through the screen displayed on the web browser.

The expense settlement device 130 includes a request control unit 441, an expense information management unit 442, and a display control unit 443.

The request control unit 441 stands by in a state where it can accept a request from an external device. When the request control unit 441 receives the processing request from the external device, the request control unit 441 instructs the expense information management unit 442 to execute the processing in response to the processing request.

The expense information management unit 442 stores the expense information registered in the expense settlement device 130 in a storage area such as HDD 314 of the information processing device 300 that realizes the expense settlement device 130.

The display control unit 443 receives a request from a web browser operating on an external device (not shown) such as a PC or mobile terminal connected via the Internet, and screen configuration information used to display the screen. (HTML, CSS, etc.) is returned to the external device of. With such a configuration, the user can confirm the expense information including the expense report information and the receipt image registered in the expense settlement device 130 and various information for the expense information through the screen displayed on the web browser. It is possible to instruct the input of.

(Overall processing flow)
An example of a series of processing flows of the image processing system according to the present embodiment will be described with reference to FIGS. 5A and 5B. 5A and 5B are sequence diagrams showing an example of processing of the image processing system according to the present embodiment. In this section, focusing on the exchange of various information between the MFP 110, the service providing device 120, and the expense reimbursement device 130, an example of the overall processing flow of the image processing system according to the present embodiment will be described.

The MFP 110 causes the display unit provided as the operation unit 220 to display the main screen 600 in which the buttons for instructing the MFP 101 to execute the processing corresponding to each function are arranged, and waits for the instruction from the user. The display unit provided as the operation unit 220 corresponds to an example of an "output unit" that outputs various information.

FIG. 6 is a diagram showing an example of the main screen 600. When an additional application for performing expense settlement (hereinafter, also referred to as “expense settlement application”) is installed, the MFP 110 displays an expense settlement button 601 for receiving an instruction related to expense settlement on the main screen 600. When the MFP 110 detects that the expense settlement button 601 is pressed, the display unit provided as the operation unit 220 displays a screen related to the start of expense settlement, which will be described later with reference to FIG. 7, and FIGS. 5A and 5B show. The processing shown is started. Each process mainly composed of the MFP 110 in FIGS. 5A and 5B is realized by executing the expense settlement application in which the MFP 110 is installed. In the present embodiment, the expense settlement application is a function including functional components 421 to 424. That is, each of the functional components 421 to 424 is a function provided by the expense settlement application.

First, FIG. 5A will be described. In S501, the MFP 110 causes the display unit provided as the operation unit 220 to display the login screen 700 for accepting the input of the authentication information for accessing the service providing device 120 from the user. FIG. 7 is a diagram showing an example of the login screen 700. The login screen 700 includes a user name input area 702, a password input area 703, and a login button 701. The user name input area 702 is an area for receiving input of a user name (identification information) for identifying a user who accesses the service providing device 120 from the user. The password input area 703 is an area for accepting input of a password used for authentication when accessing the service providing device 120 by using the user name input in the user name input area 702. The login button 701 is a button for receiving an instruction from the user to start access to the service providing device 120.

When the MFP 110 detects that the login button 701 is pressed while the user name and password are entered in the user name input area 702 and the password input area 703, the MFP 110 transmits a login request indicated as S502 to the service providing device 120. At this time, the MFP 110 associates the user name and password entered in the user name input area 702 and the password input area 703 with the login request, and then transmits the login request to the service providing device 120.
In S503, the request control unit 431 authenticates by determining whether the user name and password associated with the login request are correct, and if it is determined to be correct, returns an access token to the MFP 101. .. A known method can be used for this authentication, and examples of this known method include Basic authentication, Digest authentication, and authorization of access authority using OAuth. The access token is identification information used by the service providing device 120 to identify an authenticated user.
In the present embodiment, when making various requests to the service providing device 120 in the processing after S503, the MFP 110 transmits the acquired access token in association with the request. Further, the request control unit 431 recognizes that the request corresponds to the authenticated user based on the access token associated with the various requests transmitted from the MFP 110. Further, the MFP 110 can realize the process related to the transmission of various requests to the service providing device 120 based on an architecture such as REST (Representational State Transfer).

When the login process described as S501 to S503 is completed, the MFP 110 provides a service for obtaining information on the target person (hereinafter, also referred to as "proxy application target person information") for which the proxy application for expense settlement is approved in S504. It is transmitted to the providing device 120.
When the service providing device 120 acquires the acquisition request of the proxy application target person information from the MFP 110, the service providing device 120 identifies the user of the expense reimbursement device 130 that is previously associated with the user associated with the access token associated with the acquisition request. To do. Then, in S505, the service providing device 120 transmits a request for acquisition of proxy application target person information to the expense settlement device 130 using the specified access token of the user.
In the present embodiment, the service providing device 120 is associated with the user of the service providing device 120 and the user of the expense reimbursement device 130 in advance as follows. That is, the service providing device 120 includes the user of the service providing device 120 and the user of the expense reimbursement device 130 in response to an instruction from the user via a web browser operating on an external device connected via the Internet. It is assumed that there is a link between them. That is, if the user of the service providing device 120 can recognize the service providing device 120, the user of the expense reimbursement device 130 corresponding to this user can be uniquely identified.

When the expense settlement device 130 acquires the acquisition request of the proxy application target person information described in S505 from the service providing device 120, the expense settlement device 130 identifies the user corresponding to the access token associated with the acquisition request. Then, the expense settlement device 130 extracts the proxy application target person information corresponding to the specified user from the proxy application target person information registered in advance, and returns the proxy application target person information to the service providing device 120. .. In the present embodiment, the proxy application target person information registered in advance in the expense settlement device 130 is stored in the storage unit (for example, ROM 312, HDD 314) of the expense settlement device 130.
In the present embodiment, the expense reimbursement device 130 registers the proxy application target person information in advance in response to an instruction from the user via a web browser operating on an external device connected via the Internet. It shall be. Further, the proxy application target person information is stored in the storage unit (for example, HDD 314) of the expense settlement device 130.

FIG. 8 is a diagram showing an example of the format of the proxy application target person information returned by the expense settlement device 130 to the service providing device 120. The proxy application target person information includes the information shown as "lоgin User" and the information shown as "Delegators".
The information shown as "lоgin User" is information about the user who is logged in to the expense settlement device 130, and is information about the user corresponding to the access token used by the service providing device 120 to send the acquisition request of the proxy application target person information. Corresponds to. The information shown as "lоgin User" includes the information shown as "UserID", "UserName", and "Fullname". The information shown as "UserID" is the identification information assigned to the target user to identify this user. The information shown as "UserName" indicates information about the name of the target user. The information shown as "Fullname" indicates information about the full name of the target user.
The information shown as "Delegators" is information about the user registered as a proxy applicant for the user corresponding to the information shown as "lоgin User". The information shown as "Delegators" includes the information shown as "UserID" and "UserName". The information shown as "UserID" is the identification information assigned to the user registered as the proxy applicant to identify this user. The information shown as "UserName" indicates information about the name of the user registered as a proxy applicant.

The request control unit 431 acquires the proxy application target person information from the expense settlement device 130 as a response to the request shown in S505 of FIG. 5A. Then, the request control unit 431 transmits the proxy application target person information acquired from the expense settlement device 130 to the MFP 110 as a response to the request shown in S504 of FIG. 5A. In the present embodiment, the request control unit 431 transfers the proxy application target person information acquired from the expense settlement device 130 to the MFP 110 as it is, but the data conversion or format of the proxy application target person information is performed. The conversion may be performed.

In S506, the MFP 110 causes the proxy application screen 900 to be displayed on the display unit provided as the operation unit 220 based on the proxy application target person information acquired from the request control unit 431 of the MFP 110. FIG. 9 is a diagram showing an example of the proxy application screen 900. The proxy application screen 900 includes a user name designation area 902 and a next button 901.
The user name designation area 902 is an area for accepting the designation of the proxy applicant from the user. For example, the user name designation area 902 is realized as a drop-down list, and the MFP 110 displays a user who can be designated as a proxy applicant in this drop-down list based on the proxy application target person information acquired from the expense settlement device 130. Then, the MFP 110 designates the user designated by the operator of the proxy application screen 900 (for example, the applicant himself / herself) as the proxy applicant from the list that can be designated as the proxy applicant displayed in this drop-down list. Recognize as a user.
Then, when the MFP 110 detects that the next button 901 is pressed, the process proceeds to S507 of FIG. 5A.

In S507, the MFP 110 causes the display unit provided as the operation unit 220 to display the scan setting screen 1000 for accepting the designation of various setting information related to the reading of the document. FIG. 10 is a diagram showing an example of the scan setting screen 1000. The scan setting screen 1000 includes a color setting area 1002, a resolution setting area 1003, and a scan start button 1001. The color setting area 1002 is an area that accepts the designation of setting information regarding color when reading a document. The resolution setting area 1003 is an area for receiving the designation of setting information regarding the resolution when scanning the original.
In the present embodiment, the scan setting screen 1000 includes the setting areas 1002 and 1003, but the screen configuration of the scan setting screen 1000 is not necessarily limited. For example, the MFP 110 is an area for receiving the designation of other setting information in addition to the color setting area 1002 and the resolution setting area 1003 for the scan setting screen 1000 according to the function related to the scanning of the document included in the MFP 110. May be presented. Further, the MFP 110 does not have to present at least one of the color setting area 1002 and the resolution setting area 1003 to the scan setting screen 1000. Further, the MFP 110 may limit the setting value that can be specified to at least one of the color setting area 1002 and the resolution setting area 1003 to only the setting value required by the expense settlement device 130.

In S508, when the MFP 110 detects that the scan start button 1001 is pressed, it reads the document placed on the platen of the scanner 222 based on the setting information specified in the color setting area 1002 and the resolution setting area 1003. Such processing is started. The platen of the scanner 222 corresponds to an example of "a predetermined area in which a document to be read is arranged". The MFP 110 generates image data according to the result of scanning the original by the scanner 222.
In S509, the MFP 110 transmits the image data generated according to the reading result of the original to the service providing device 120, and transmits the analysis request of the image data to the service providing device 120.
In S510, the image processing unit 432 receives an analysis request from the MFP 110 and performs multi-crop processing on the image data transmitted from the MF 110 to extract an image for each document from the image data. In the following description, the image for each manuscript extracted from the image data is also referred to as a "manuscript image" for convenience.
Further, the request control unit 431 returns a request ID for uniquely identifying the analysis process according to the analysis request from the MFP 110 to the MFP 110 regardless of whether or not the multi-crop process is completed.
For example, FIG. 11 shows an example of the format of this response when the service providing device 120 returns the request ID as a response to the MFP 110 in response to the analysis request of the image data from the MFP 110. The information shown as "JobID" is information corresponding to the request ID for specifically specifying the analysis process executed by the service providing device 120 based on the analysis request of the image data from the MFP 110.

The MFP 110 uses this request ID to periodically contact the request control unit 431 until it confirms the completion of the requested analysis process (analysis process indicated by the request ID returned from the request control unit 431 to the MFP 110). Check the processing status of this analysis processing.
In S511, the MFP 110 confirms the processing status of the analysis processing with the request control unit 431 using the request ID. In this case, the image processing unit 432 of the MFP cooperation service 120 has not completed the analysis processing corresponding to this request ID. Therefore, the request control unit 431 notifies the MFP 110 of information indicating that processing is in progress as the processing status of the analysis processing corresponding to the request ID. The exchange of information between the MFP 110 and the MFP cooperation service 120 shown as S511 is periodically performed at predetermined intervals until the analysis process in the MFP cooperation service 120 is completed.

In S512, when the image processing unit 432 extracts individual manuscript images as a result of the multi-crop processing, the image processing unit 432 executes rotation processing, character recognition processing, and manuscript type identification processing for each of the extracted manuscript images. The rotation process corresponds to a process of rotating a document image so that when the side of the document is slanted with respect to the horizontal and vertical directions of the image, this side substantially coincides with the horizontal or vertical direction of the image. .. The character recognition process corresponds to a process of recognizing characters in a manuscript image as character data by performing an image analysis process such as OCR (Optical Character Recognition) on the manuscript image. The manuscript type identification process corresponds to the process of specifying the type of the manuscript based on the information contained in the manuscript extracted according to the result of the character recognition process or the like.
In S513, the MFP 110 confirms the processing status of the analysis process with the request control unit 431 using the request ID. In this case, the image processing unit 432 has completed the analysis process corresponding to this request ID. Therefore, the request control unit 431 notifies the MFP 110 of information indicating that the processing is completed as the processing status of the analysis processing corresponding to the request ID.

FIG. 12 shows an example of the format of the response notified from the service providing device 120 in response to the confirmation of the processing status of the analysis processing from the MFP 110.
Specifically, FIG. 12A shows that the request control unit 431 of the service providing device 120 is processing the analysis process on the MFP 110 in response to the confirmation of the processing status from the MFP 110 shown as S511. Shows the format of the response for notifying information. The information shown as "Status" indicates the processing status of the analysis process. In the example shown in FIG. 12 (A), "processing" indicating that processing is being performed for this "Status" is set.
Further, FIG. 12B is for the request control unit 431 of the service providing device 120 to notify the MFP 110 of information indicating that the analysis process is completed in response to the confirmation of the processing status from the MFP 110 shown as S513. Shows the format of the response. The information shown as "Status" indicates the processing status of the analysis processing as in the example shown in FIG. 12 (A). In the example shown in FIG. 12B, "completed" indicating that the process is completed is set for this "Status".

Further, in the example shown in FIG. 12B, information according to the result of the analysis processing is notified as a response to the confirmation of the processing status from the MFP 110. The information shown as "Images" indicates information according to the result of the analysis process for each original image.
The "Image ID" is identification information for uniquely identifying each original image.
"Contents Type" indicates a specific result of the type of manuscript. As a specific example, when the type of the manuscript is "receipt", "Receipt" indicating that the manuscript is a receipt is set in "ContentsType". When the type of the manuscript is "business card", "Business Card" indicating that it is a business card is set in "Contents Type". When the type of the manuscript is "license", "Driver License" indicating that the manuscript is a license is set in "Contents Type".
When "Receipt" indicating a receipt is set for "ContentsType", "ReceiptData" is set in the response for notifying the completion of the analysis process. Information according to the analysis result of the receipt, which is the manuscript, is set in "ReceiptData". Specifically, "Date" is information indicating the date when the receipt was issued. "Amount" is the information presented on the receipt as the total amount.
"Points" is information indicating a region in which the original image is presented in the image indicated by the image data to be analyzed, which is transmitted from the MFP 110 to the service providing device 120 in S509. In "Points", the X coordinate value and the Y coordinate are in this order for the upper left, upper right, lower right, and lower left vertices of the area in which the original image is presented in the image indicated by the image data to be analyzed. The values are presented sequentially. The X coordinate value and the Y coordinate value here are the image indicated by the image data to be analyzed, or the coordinate values in the XY coordinate system set on the platen.

Subsequently, FIG. 5B will be described. In S514, the MFP 110 specifies the "Image ID" of the original image to be displayed first, and requests the transmission of the original image corresponding to the service providing device 120. The request control unit 431 receives a request from the MFP 110 and returns the original image corresponding to the designated "Image ID" to the MFP 110. At this time, the original image returned from the service providing device 120 to the MFP 110 becomes an image whose orientation has been adjusted by performing rotation processing.

In S515, the MFP 110 requests the request control unit 431 to transmit information related to the metadata related to the original image (hereinafter, also referred to as “metadata information”) associated with the original image.
In S516, the request control unit 431 requests the expense settlement device 130 to transmit the metadata information about the user corresponding to the request based on the request from the MFP 110. In response to the request from the request control unit 431 shown in S516, the expense reimbursement device 130 returns information previously associated with the corresponding user as metadata information corresponding to this user. Further, the request control unit 431 transmits the metadata information corresponding to the user returned from the expense settlement device 130 to the MFP 110 as a response to the request from the MFP 110 shown as S515.
In the present embodiment, the request control unit 431 transfers the metadata information acquired from the expense settlement device 130 to the MFP 110 as it is. However, the processing related to the transmission of metadata information by the service providing device 120 is not necessarily limited. For example, when the request control unit 431 is requested to transmit a plurality of metadata information by the MFP 110, the request control unit 431 requests the expense settlement device 130 to transmit each metadata information, and the plurality of metadata acquired from the expense settlement device 130. Information may be transmitted to the MFP 110. Further, in this case, the request control unit 431 may combine the plurality of metadata information acquired from the expense settlement device 130 as one data, and then transmit the combined data to the MFP 110. At least one of the MFP 110 and the request control unit 431 may thin out the information that is not used in the subsequent processing from the plurality of acquired metadata information.

FIG. 13 shows an example of the format of information returned by each of the request control unit 431 and the expense settlement device 130 as a response to a request for transmitting metadata information. The information shown as "Expense Type" shows a list of definition values for each use related to the manuscript indicated by the image registered in the expense settlement device 130. Specifically, "ID" and "Name" are set for each use related to the document indicated by the image. The "ID" is identification information for identifying the use related to the document indicated by the image. "Name" is information indicating the name of the use related to the manuscript indicated by the image. For example, in the example shown in FIG. 13, "ID" and "Name" are assigned for each usage of the amount of money indicated by the receipt as the usage related to the manuscript shown by the image.

In S517 of FIG. 5B, the MFP 110 determines whether or not the conditions required for digitizing each original image are satisfied based on the result of the character recognition processing for each original image. The details of this determination process will be described later with reference to FIGS. 16A and 16B.
In S530, the MFP 110 displays the effective condition confirmation screen 1800 shown in FIG. 18 on the display unit provided as the operation unit 220 based on the result of determination as to whether or not the conditions required for digitizing each original image in S517 are satisfied. Let me. The details of the valid condition confirmation screen 1800 will be described later with reference to FIG.
In S518, the MFP 110 includes a metadata setting screen 1400 as an operation unit 220 for accepting from the user the designation of the metadata associated with the original image based on the original image acquired in S513 and the result of the determination in S517. Display on the display.

FIG. 14 is a diagram showing an example of the metadata setting screen 1400. The metadata setting screen 1400 has a manuscript image presentation area 1410, an expense type setting area 1404, a transaction date setting area 1405, an amount setting area 1406, a position information setting area 1407, a send button 1401, and a next button 1402. And the back button 1403.

The manuscript image presentation area 1410 is an area in which the manuscript image to be processed is displayed.

The expense type setting area 1404 is an area that accepts the designation of information regarding the type of expense to be settled. Based on the metadata information acquired from the service providing device 120 as a response to the request shown in S515 of FIG. 5B, the MFP 110 causes the expense type setting area 1404 to display information regarding the type of expense to be cleared as a list. Specifically, the MFP 110 causes the expense type setting area 1404 to display a list of information indicated as “Name” among the data for each expense type indicated as “Expense Type” in FIG.

The trading date setting area 1405 is an area for receiving the designation of information regarding the date on which the expense to be settled is incurred. The MFP 110 may input an initial value in the transaction date setting area 1405 based on the information included in the response returned from the service providing device 120 in response to the confirmation of the processing status shown in S513 of FIG. 5B. Specifically, in FIG. 12B, the MFP 110 may input information included as “Amount” of “Recept Data” for the manuscript image to be processed as an initial value in the transaction date setting area 1405. ..

The amount setting area 1406 is an area for receiving input of information regarding the amount of expenses to be settled. An arbitrary numerical value can be input to the amount setting area 1406.
Further, the MFP 110 relates to the user inputting information to the amount setting area 1406 by displaying a keyboard (not shown) on the screen when the amount setting area 1406 is designated as the information input target. The operation may be supported.

The position information setting area 1407 is an area for receiving input of information regarding the place where the expense to be settled is incurred (in other words, the place where the manuscript to be processed is issued). Any character can be input to the position information setting area 1407. Further, when the position information setting area 1407 is designated as an information input target, the MFP 110 displays a keyboard (not shown) on the screen so that the user can input the information to the position information setting area 1407. The operation related to the above may be supported. Further, as another example, the MFP 110 acquires information on the candidate for the above location from the service providing device 120 as a response to the request shown in S515 of FIG. 5B, and displays a list of information on the candidate in the location information setting area 1407. It may be displayed.

The MFP 110 associates the information specified or input in the expense type setting area 1404, the transaction date setting area 1405, the amount setting area 1406, and the position information setting area 1407 with the manuscript image to be processed as metadata.

The next button 1402 is a button that receives an instruction relating to a change in the original image to be processed. Each time the MFP 110 detects that the next button 1402 is pressed, the MFP 110 sequentially changes the original image to be processed. At this time, the MFP 110 displays the original image selected as the processing target in the original image presentation area 1410.
The back button 1403 is a button that receives an instruction relating to a change in the original image to be processed. When the MFP 110 detects that the back button 1403 is pressed, the MFP 110 again sets another manuscript image selected as the processing target immediately before the manuscript image selected as the processing target at that time. .. At this time, the MFP 110 displays the original image selected as the processing target again in the original image presentation area 1410.
The MFP 110 may invalidate the next button 1402 and the back button 1403 when there is only one original image as a candidate for processing. Further, the MFP 110 may invalidate the back button 1403 when the first candidate among the candidates for the original image to be processed is selected as the processing target. Further, the MFP 110 may invalidate the Next button 1402 when the last candidate among the candidates for the original image to be processed is selected as the processing target.
The transmission button 1401 is a button that receives an instruction relating to transmission of metadata associated with the original image to be processed to the service providing device 120. When the transmission button 1401 is pressed, the MFP 110 transmits each information designated or input to each area indicated as 1404 to 1407 for each original image to the service providing device 120 as metadata of the original image. The MFP 110 may disable the transmission button 1401 until the designation or input of the information in each area shown as 1404 to 1407 is completed for all the original images that are candidates for processing.

Here, the description returns to FIG. 5B. In S519, the MFP 110 associates various information specified or input for each original image with the metadata setting request as the metadata of the original image, and transmits the metadata setting request to the request control unit 431. The request control unit 431 receives the metadata setting request from the MFP 110, and associates the metadata associated with the metadata setting request with the image data of the original image corresponding to the metadata held by the request control unit 431.
In S520, the request control unit 431 transmits a request for image registration and metadata setting in which the image data of the original image and the metadata associated with the image data are associated with the expense settlement device 130.

FIG. 15 shows an example of the format of the request for metadata setting for the original image transmitted by the request control unit 431 to the expense settlement device 130.
The "Expense Type ID" is information indicating the type of expense to be settled, and identification information associated with each type is set.
The "Transaction Date" is information indicating the transaction date, and information indicating the date when the expense to be settled is incurred is set.
In "Amount", information about the amount to be settled as an expense is set.
In "Location", information about the place where the expense to be settled is incurred is set.

In response to the request for metadata setting, the expense settlement device 130 registers the image data associated with this request in the expense information managed by itself, and the metadata associated with the image data is registered in the expense information. Associate with the image data of the octopus.
As a response to the request shown in S520, the expense reimbursement device 130 notifies the service providing device 120 of information indicating that the registration of the image data and the setting of the metadata have been completed.
The service providing device 120 receives the response to the request shown in S520 from the expense reimbursement device 130, and notifies the MFP 110 of information indicating that the metadata setting is completed as a response to the request shown in S519.
As a result, the series of processes shown with reference to FIGS. 5A and 5B is completed.

(Details of MFP processing)
The details of the processing of the MFP 110 according to the present embodiment will be described with reference to FIGS. 16A and 16B, paying particular attention to the processing related to the settlement of expenses. 16A and 16B are flowcharts showing an example of the processing flow of the MFP 110. The series of processes shown in FIGS. 16A and 16B is realized by the CPU 211 expanding the program stored in the HDD 214 or the ROM 212 into a storage area such as the RAM 213 and executing the program. Further, in the present embodiment, when the MFP 110 detects that the expense settlement button 601 of the main screen 600 shown in FIG. 6 is pressed, the MFP 110 starts executing the processes shown in FIGS. 16A and 16B.

First, FIG. 16A will be described.
In step S1601, the display control unit 421 causes the display unit provided as the operation unit 220 to display the login screen 700 shown in FIG. 7. When the display control unit 421 detects that the login button 701 is pressed while the user name and password are entered in the user name input area 702 and the password input area 703, the display control unit 421 makes a login request to the cooperation service request unit 423. At this time, the display control unit 421 notifies the cooperation service request unit 423 of the user name and password entered in the user name input area 702 and the password input area 703.

In S1602, the cooperation service request unit 423 receives the request from the display control unit 421 and transmits a login request to the request control unit 431 based on the notified user name and password. The request control unit 431 accesses the data management unit 434 and confirms whether or not the user name and password indicated by the transmitted login request are correct. When the request control unit 431 confirms that the user name and password indicated by this login request are correct, the request control unit 431 returns the access token to the cooperation service request unit 423.

In S1603, the cooperation service request unit 423 associates the acquired access token with the request related to the acquisition of the proxy application target person information, and then transmits this request to the request control unit 431.
When the request control unit 431 receives the request for acquiring the proxy application target person information, the request control unit 431 requests the expense settlement request unit 433 to acquire the proxy application information. Then, the expense settlement request unit 433 acquires the information about the user of the expense settlement device 130 linked to the access token, and transmits the request related to the acquisition of the proxy application target person information of the target user to the request control unit 441. ..
When the request control unit 441 receives the request for acquiring the proxy application target person information, the request control unit 441 requests the expense information management unit 442 for the proxy application target person information of the target user. The expense information management unit 442 acquires the proxy application target person information corresponding to the request from the storage unit of the expense settlement device 130, and transmits it to the request control unit 441. Then, the request control unit 441 returns the acquired proxy application target person information to the expense settlement request unit 433.
The expense settlement request unit 433 returns the proxy application target person information returned from the request control unit 441 to the cooperation service request unit 423 via the request control unit 431.
The cooperation service request unit 423 outputs the proxy application target person information returned from the request control unit 431 to the display control unit 421.

In S1604, the display control unit 421 displays the proxy application screen 900 shown in FIG. 9 on the display unit included in the operation unit 220. Further, the display control unit 421 displays a list of users who can be designated as the proxy applicant in the user name designation area 902 of the proxy application screen 900 based on the acquired proxy application target person information. At this time, the display control unit 421 may display information indicating the user who is the operator of the proxy application screen 900 as an initial value for the user name designation area 902.
The display control unit 421 recognizes the user designated in the user name designation area 902 as the user designated as the applicant. When the user himself / herself who is the operator of the proxy application screen 900 is designated for the user name designation area 902, the display control unit 421 recognizes this user himself / herself as an applicant for expense settlement. Become. That is, when the user himself / herself who is the operator is the applicant himself / herself, the display control unit 421 recognizes that the applicant himself / herself is performing various operations related to the expense settlement.
Then, when the display control unit 421 detects that the next button 901 is pressed, the process proceeds to S1605.
If the proxy application target person is not registered in the proxy application target person information acquired in the processing of S1603, the display control unit 421 skips the processing of S1604 and executes the processing of S1605. You may.

In S1605, the display control unit 421 displays the scan setting screen 1000 shown in FIG. 10 on the display unit included in the operation unit 220. When the display control unit 421 detects that the scan start button 1001 has been pressed, the display control unit 421 proceeds to S1606.
In S1606, the display control unit 421 requests the scan instruction unit 422 to instruct the scanning instruction unit 422 to read the document based on the setting information designated in each of the color setting area 1002 and the resolution setting area 1003. The scan instruction unit 422 instructs the scan execution unit 411 to read the document, so that the scan execution unit 411 reads the document. The scan execution unit 411 reads the document, instructs the image data storage unit 412 to save the image data according to the scanning result of the document, and transmits the scan image identifier corresponding to the image data to the scan instruction unit 422. .. The scan instruction unit 422 acquires the image data corresponding to the scan image identifier from the image data storage unit 412, and outputs the acquired image data to the cooperation service request unit 423.
In S1606, the display control unit 421 may display a screen indicating that the document is being read on the display unit included in the operation unit 220 during the period in which the document is being read.

In S1607, the cooperation service request unit 423 transmits the image data acquired from the scan instruction unit 422 to the request control unit 431, and transmits an analysis request for this image data to the request control unit 431. In response to this analysis request, the request control unit 431 requests the image processing unit 432 to apply the multi-crop processing to the acquired image data. When the multi-crop processing by the image processing unit 432 is completed, the request control unit 431 performs rotation processing, character recognition processing, and document type identification processing on the original image extracted by the multi-crop processing for the image processing unit 432. Request application.
Further, the request control unit 431 returns a request ID for uniquely identifying these various processes to the cooperation service request unit 423 regardless of whether or not the various processes described above by the image processing unit 432 are completed. ..

In S1608, when the cooperation service request unit 423 acquires the request ID, the cooperation service request unit 423 periodically transmits a processing status acquisition request to the request control unit 431 using this request ID.
When the request control unit 431 receives the processing status acquisition request, the request control unit 431 inquires the image processing unit 432 about the processing status corresponding to the request ID associated with the request. When the process corresponding to the request ID is being processed, the image processing unit 432 returns information indicating that the process is being processed to the request control unit 431. Further, when the processing corresponding to the request ID is completed, the image processing unit 432 includes information indicating that the processing is completed and information according to the results of the multi-crop processing and various image analyzes. Is returned to the request control unit 431. The request control unit 431 creates a response to the processing status acquisition request based on the information returned from the image processing unit 432, and returns this response to the cooperation service request unit 423.

In S1609, the cooperation service request unit 423 confirms the information regarding the processing status included in the response returned from the request control unit 431. When the information regarding the processing status indicates that the processing is in progress (that is, when the processing is incomplete), the cooperation service request unit 423 waits for a predetermined period and then re-executes the processing of S1608. Further, when the information regarding the processing status indicates completion, the cooperation service request unit 423 extracts the information according to the analysis result of the image data from the above response.

Subsequently, FIG. 16B will be described.
In S1610, the cooperation service request unit 423 associates the access token with the request for acquiring the metadata information, and then transmits this request to the request control unit 431.
When the request control unit 431 receives the request for acquiring the metadata information, the request control unit 431 requests the expense settlement request unit 433 to acquire the metadata information. The expense reimbursement request unit 433 identifies the user of the expense reimbursement device 130, which is previously associated with the user associated with the access token associated with the metadata information acquisition request. The expense settlement request unit 433 transmits a request for acquiring the metadata information of the specified user to the request control unit 441.
When the request control unit 441 receives the request for acquisition of the metadata information, the request control unit 441 acquires the metadata information from the expense information management unit 442 and returns the metadata information to the expense settlement request unit 433.
The expense settlement request unit 433 returns the metadata information returned in response to the request for acquisition of the metadata information to the cooperation service request unit 423 via the request control unit 431.

In S1611, the determination unit 424 determines whether or not the manuscript corresponding to each manuscript image has passed the valid period based on the result of character recognition applied to each manuscript image. FIG. 17A is a flowchart showing an example of a process for determining whether or not the original document corresponding to the original image is valid. In particular, it is determined whether or not the original document corresponding to the original image has passed the valid period. An example of the case is shown.

In S1701, the determination unit 424 determines whether or not the proxy applicant has been selected based on the instruction received from the user by the display control unit 421 in S1604 of FIG. 16A via the proxy application screen 800. If the determination unit 424 determines that the proxy applicant has not been selected and the applicant himself / herself makes the application, the determination unit 424 proceeds to S1702.
In S1702, the determination unit 424 includes information on the date included in the manuscript image (hereinafter, also referred to as “date information”) among the information on the application result of the character recognition process on the manuscript image from the response acquired in S1608 of FIG. ) Is extracted. Based on the extracted date information, the determination unit 424 determines whether or not the expiration date of the manuscript (receipt) corresponding to the manuscript image from which the date information is extracted has passed.
In the present embodiment, the image processing system targets the manuscript that is the receipt, but as another example, among the manuscripts other than the receipt, the manuscript that is particularly related to the settlement of expenses. May be the target of processing. Further, the image processing system may process a manuscript different from the manuscript related to expense settlement.

For example, in the case of the example shown in FIG. 12B, the information included as “Date” in “Recept Data” corresponds to the date information extracted from the corresponding manuscript image. Specifically, "20180313" is set as the date information in the manuscript image in which "image0001" is set as the "ImageID". Further, "20180311" is set as the date information in the manuscript image in which "image0002" is set as the "ImageID". Similarly, "20180312" is set as the date information in the manuscript image in which "image0003" is set as the "ImageID".
The determination unit 424 sets the expiration date to "3 days from the day after the manuscript is issued", and when the date when the receipt is read is "2018/03/15", "image0002" is set. It will be determined that the receipt corresponding to the original image has passed the expiration date.

If the determination unit 424 determines in S1702 that there is a receipt that has passed the expiration date, the determination unit 424 proceeds to process in S1703.
In S1703, the determination unit 424 holds information regarding the receipt determined in S1702 that the expiration date has passed. The information regarding this receipt corresponds to the information indicated as "Date" and "Amount" included in the information indicated as "ReceiptData" in FIG. 12 (B). Further, at this time, the determination unit 424 holds the information indicating the area where the manuscript image corresponding to the receipt is presented in association with the information related to the receipt. In the present embodiment, the determination unit 424 stores the information regarding the receipt and the information indicating the area presented by the manuscript image in the RAM 213 or the HDD 214. The information indicating the area in which the manuscript image is presented corresponds to the information indicated as "Points" in FIG. 12 (B).

On the other hand, when the determination unit 424 determines in S1702 that there is no manuscript whose expiration date has passed, the process proceeds to S1704.
In S1704, the determination unit 424 sets a flag (hereinafter, also referred to as “signature confirmation flag”) indicating that signature confirmation (confirmation of whether or not there is a signature in the receipt) is required to ON. The determination unit 424 stores information indicating the signature confirmation flag in the RAM 213 or the HDD 214.

In S1630 of FIG. 16B, the display control unit 421 executes a process related to the presentation of the conditions required for digitizing the receipt based on the information held by the determination unit 424 in S1611. FIG. 17B is a flowchart showing an example of processing related to the presentation of conditions required for digitizing receipts.

In S1751, the determination unit 424 confirms whether or not the information regarding the receipt that has passed the expiration date is retained. When the determination unit 424 holds the information regarding the receipt that has passed the expiration date in S1751, it determines that the receipt that has passed the expiration date exists, and proceeds to the process in S1752.
In S1752, the display control unit 421 specifies where on the platen of the scanner 222 the receipt that has passed the expiration date is placed, based on the information about the receipt held by the determination unit 424. Then, the display control unit 421 presents the user by displaying the information indicating the position on the platen on which the receipt that has passed the expiration date is arranged on the display unit provided in the operation unit 220.

FIG. 18 shows an example of a screen for presenting whether or not the read receipt is valid. Specifically, the valid condition confirmation screen 1800 indicates the position on the platen on which the receipt that has passed the expiration date is arranged, which the display control unit 421 displays on the display unit provided in the operation unit 220 in S1752. An example of the screen is shown.
The valid condition confirmation screen 1800 includes a notification information presentation area 1860, a platen information presentation area 1861, a rescan button 1810, and a next button 1811.
The notification information presentation area 1860 is an area in which information to be notified to the user is presented. The display control unit 421 determines that there is a receipt that has passed the expiration date in the notification information presentation area 1860 of the valid condition confirmation screen 1800, and that an application by a proxy applicant is required for this receipt. The notification information to be shown is presented.
The platen information presentation area 1861 is an area in which information indicating the position of the read receipt is arranged on the platen is presented. The display control unit 421 presents the position on the platen on which each receipt is arranged by displaying an image corresponding to the reading result of the document in the platen information presentation area 1861. At this time, the display control unit 421 inverts the left and right of the image according to the reading result of the original in order to make the position of each receipt on the platen easier to understand intuitively, and after this inversion, The image may be displayed in the platen information presentation area 1861. Further, at this time, the display control unit 421 presents the existence of the receipt to the user by controlling so that the area where the receipt that has passed the expiration date is presented is emphasized. For example, in the example shown in FIG. 18, the display control unit 421 shades the area where the receipt that has passed the expiration date is presented among the images presented in the document stand information presentation area 1861 of the valid condition confirmation screen 1800. The image of is superimposed. By applying such processing, the manuscript image of the receipt that has passed the expiration date and the manuscript image of the receipt that has not passed the expiration date are presented in different modes. The area corresponding to the manuscript image of the receipt passed over is emphasized more.

If the determination unit 424 does not hold the information regarding the receipt that has passed the expiration date in S1751, it determines that there is no receipt that has passed the expiration date, and proceeds to the process in S1753.
In S1753, the determination unit 424 determines whether or not the signature confirmation flag is set to ON in S1704 of FIG. 17A.
When the determination unit 424 determines in S1753 that the signature confirmation flag is not set to ON, the determination unit 424 ends the process shown in FIG. 17B (that is, the process of S1630 in FIG. 16B), and performs the process in S1612 of FIG. 16B. Proceed.
On the other hand, when the determination unit 424 determines in S1753 that the signature confirmation flag is set to ON, the determination unit 424 proceeds to the process in S1754.
In S1754, the display control unit 421 presents the user by displaying the notification information notifying that the signature (signature of the applicant himself / herself) is required for each receipt on the display unit provided in the operation unit 220. To do.

The valid condition confirmation screen 1850 shown in FIG. 18 is a notification that the display control unit 421 displays on the display unit provided in the operation unit 220 in S1754 to notify that each receipt requires the signature of the applicant himself / herself. An example of a screen on which information is presented is shown. The valid condition confirmation screen 1850 is an aspect of the valid condition confirmation screen 1800.
The display control unit 421 presents the notification information indicating that the signature of the applicant himself / herself is required for each receipt in the notification information presentation area 1860 of the valid condition confirmation screen 1850. Further, the display control unit 421 displays an image according to the reading result of the original in the platen information presentation area 1861 of the valid condition confirmation screen 1850, so that the user can set the position on the platen on which each receipt is arranged. It is presented to. At this time, the display control unit 421 inverts the left and right of the image according to the reading result of the original in order to make the position of each receipt on the platen easier to understand intuitively, and after this inversion, The image may be displayed in the platen information presentation area 1861.

In S1755 of FIG. 17B, the display control unit 421 determines which of the rescan button 1810 and the next button 1811 on the valid condition confirmation screen 1800 or 1850 is pressed.
When the display control unit 421 detects that the rescan button 1810 is pressed in S1755, the process proceeds to S1606 of FIG. 16A, and the processes after S1606 are executed again.
On the other hand, when the display control unit 421 detects that the next button 1811 is pressed in S1755, the process shown in FIG. 17B (that is, the process of S1630 in FIG. 16B) is terminated, and FIG. 16B The process proceeds to S1612 of.

As described above, by executing the process of S1630 (that is, the process shown in FIG. 17B), the applicant himself / herself (that is, the user who is the operator) receives a receipt that is not considered to be valid on the platen of the scanner 222. It is possible to grasp the position on the top.
In addition, the applicant himself / herself can confirm whether or not there is an unsigned receipt by the image presented in the manuscript stand information presentation area 1861 of the valid condition confirmation screen 1850. If there is an unsigned receipt, the applicant himself / herself signs the target receipt again and then presses the rescan button 1810 to re-sign the receipt. It is also possible to have the MFP 110 take in the data.

Subsequently, the processing after S1612 in FIG. 16B will be described.
In S1612, the display control unit 421 displays the metadata setting screen 1400 described with reference to FIG. 14 on the display unit included in the operation unit 220. At this time, the display control unit 421 includes the original image whose information is included in the response acquired in S1608 of FIG. 16A, other than the receipt determined in S1703 of FIG. 17A that the expiration date has passed. Identify the manuscript image that corresponds to the receipt. Then, the display control unit 421 sequentially presents information on the receipt corresponding to the specified original image to each area of the metadata setting screen 1400. On the other hand, the display control unit 421 excludes the receipt determined that the expiration date has passed from the target of presenting the information on the metadata setting screen 1400. Details regarding the presentation of information regarding receipts that have passed the expiration date will be described later.

In S1613, the display control unit 421 determines which of the transmit button 1401, the next button 1402, and the back button 1403 on the metadata setting screen 1400 is pressed.
When the display control unit 421 detects that the next button 1402 is pressed in S1613, the display control unit 421 proceeds to S1614. In S1614, the display control unit 421 specifies the manuscript image to be presented to the metadata setting screen 1400 next to the manuscript image to be presented to the metadata setting screen 1400. Then, in S1612, the display control unit 421 presents information on the receipt corresponding to the newly specified original image in each area of the metadata setting screen 1400.
When the display control unit 421 detects that the return button 1403 is pressed in S1613, the display control unit 421 proceeds to S1615. In S1615, the display control unit 421 respecifies the manuscript image to be presented to the metadata setting screen 1400 before the manuscript image to be presented to the metadata setting screen 1400. Then, in S1612, the display control unit 421 presents information on the receipt corresponding to the newly specified original image in each area of the metadata setting screen 1400.
When the display control unit 421 detects that the transmission button 1401 is pressed in S1613, the display control unit 421 proceeds to the process in S1616.

In S1616, the cooperation service request unit 423 recognizes the information specified in each area of the metadata setting screen 1400 for each manuscript image as the metadata of the manuscript image. Then, the cooperation service request unit 423 transmits the request for metadata setting for each manuscript image described with reference to FIG. 15 to the request control unit 431.
At this time, the cooperation service request unit 423 displays information on the type of expense to be settled in the "ExpenseTypeID" shown in FIG. 15 according to the information specified in the expense type setting area 1404 of the metadata setting screen 1400. Set.
In addition, the cooperation service request unit 423 displays information on the date when the expense to be settled is incurred in the "Transaction Date" shown in FIG. 15 according to the information specified in the transaction date setting area 1405 of the metadata setting screen 1400. Set.
Further, the cooperation service request unit 423 sets information on the amount of expenses to be settled in the "Amount" shown in FIG. 15 according to the information specified in the amount setting area 1406 of the metadata setting screen 1400.
In addition, the cooperation service request unit 423 displays information on the location where the expense to be settled is incurred in the “Location” shown in FIG. 15 according to the information specified in the location information setting area 1407 of the metadata setting screen 1400. Set.
Further, the cooperation service request unit 423 sets the URL of the original image itself or the parameter associated with the URL as the identification information (ImageID) for identifying the original image.

When the request control unit 431 receives the request for metadata setting, the request control unit 431 specifies the image data of the manuscript image corresponding to the metadata indicated by the request to the expense settlement request unit 433. Then, the request control unit 431 requests the expense settlement request unit 433 to register the image data of the specified original image and set the metadata for the image data. The expense reimbursement request unit 433 identifies the user of the expense reimbursement device 130 that corresponds to the access token associated with the request for metadata settings. The expense settlement request unit 433 uses the specified access token of this user to transmit a request for image registration and metadata setting to the request control unit 441.
When the request control unit 441 receives the request for image registration and metadata setting, the request control unit 441 registers the image data of the manuscript image (receipt image) with the expense information management unit 442 and sets the metadata for the image data. To do. Then, the request control unit 441 notifies the expense settlement request unit 433 of information indicating that the registration of the image data and the setting of the metadata are completed as a response to the request for the image registration and the metadata setting.
The expense settlement request unit 433 returns information indicating that the metadata setting is completed to the cooperation service request unit 423 via the request control unit 431.

(About operation by proxy applicant)
In the example described as "Details of MFP processing", a case where electronic data of a receipt is registered in the expense settlement device 130 based on an operation by the applicant of the expense application has been described. This section describes an example of registering the electronic data of this receipt in the expense settlement device 130 based on the operation by the proxy applicant who received the receipt that has passed the expiration date from the applicant himself / herself. In this section, the description will focus on the operation different from the case based on the operation by the applicant himself / herself, and the description of the same operation as the case based on the operation by the applicant himself / herself will be omitted. That is, the operation not specified in this section shall be substantially the same as the case based on the operation by the applicant himself / herself.

When the proxy applicant applies for expenses, the user name of the proxy applicant is specified in the user name designation area 902 of the proxy application screen 900 shown in FIG. 9 in S1604 of FIG. 16A. Become. That is, the display control unit 421 recognizes that the application for expenses is made by the proxy applicant based on the user name designated in the user name designation area 902.

In S1701 of FIG. 17A, the determination unit 424 determines whether or not the proxy applicant has been selected based on the instruction received from the user via the proxy application screen 800 by the display control unit 421 in S1604 of FIG. 16A. In this case, since the proxy applicant is selected in S1604, the determination unit 424 does not execute the process related to the determination of whether the receipts shown as S1702 to S1704 are valid, and is shown in FIG. 17A. The process will be terminated.

In S1616 of FIG. 16B, the cooperation service request unit 423 recognizes the information designated in each area of the metadata setting screen 1400 for each manuscript image as the metadata of the manuscript image. Then, the cooperation service request unit 423 transmits the request for metadata setting for each manuscript image described with reference to FIG. 15 to the request control unit 431. At this time, the cooperation service request unit 423 requests the request control unit 431 with the "ImageID" which is the identification information of the manuscript image and the "UserID" which is the identification information of the proxy applicant in association with the request for the metadata setting. Send to. In this case, the service providing device 120 uses the access token of the proxy applicant to register the image data of the manuscript image (receipt image) with the expense settlement device 130 and to register the metadata for the image data. A request for setting will be made.
At this time, the request control unit 431 confirms whether or not the manuscript image read based on the operation from the proxy applicant matches the manuscript image previously read based on the operation of the applicant himself / herself. You may. As a result, even if the applicant himself / herself gives the proxy applicant a receipt different from the receipt to be applied for, the image processing system can detect it as an error.

In S1611 of FIG. 16B, since the determination unit 424 does not determine whether or not the receipt shown as S1702 to S1704 is valid, the presentation of an invalid receipt as in the valid condition confirmation screen 1800 of FIG. The processing related to the above will not be performed.
On the other hand, even when the proxy applicant performs an operation related to digitization of the receipt, there are cases where items that need to be confirmed when digitizing the receipt are stipulated. For example, in Japan, according to the Electronic Bookkeeping Law, the expiration date is a maximum of one month and one week (generally 37 days) from the day after the receipt is received, and the receipt is electronic within this validity period. Digitization is required. Therefore, in S1702 of FIG. 17A, the determination unit 424 acquires the date information included in the manuscript image among the information regarding the application result of the character recognition process to the manuscript image, and digitizes the receipt based on this date information. It may be determined whether or not the expiration date has been passed.
When the determination unit 424 determines that the expiration date related to the digitization of the receipt has passed, the determination unit 424 may determine that the target receipt is a receipt that has passed the expiration date. .. In this case, the display control unit 421 displays the valid condition confirmation screen 1800 on the display unit provided in the operation unit 220 as the process of S1752 in FIG. 17B, so that the receipt that has passed the expiration date is displayed as a proxy applicant. May be presented to. This enables the proxy applicant to grasp receipts that do not meet the conditions required for digitization.

<Second embodiment>
The image processing system according to the second embodiment will be described below. In the second embodiment, the description mainly focuses on the parts different from the first embodiment, and detailed description of the parts similar to the first embodiment will be omitted. That is, the functions, configurations, and processes that are not specified in the second embodiment are substantially the same as those in the first embodiment. Specifically, in the image processing system according to the second embodiment, the processing in S1611 and S1630 of FIG. 16B is different from the image processing system according to the first embodiment.
In the present embodiment, the CPU 211 expands and executes the program stored in the ROM 212 or the HDD 214 in the RAM 213 to execute the functions of the MFP 110 shown in FIG. 4, FIG. 5A, FIG. 5B, FIG. 16A, FIG. 16B, FIG. 19A and FIG. 19B. The processing shown in is realized.

(processing)
An example of the processing of the information processing system according to the second embodiment will be described below.
First, among the processes of the image processing system according to the second embodiment, the process corresponding to S1611 of FIG. 16B will be described. FIG. 19A is a flowchart showing an example of processing for determining whether or not the original document corresponding to the original image is valid, and corresponds to S1611 of FIG. 16B among the processing of the image processing system according to the second embodiment. An example of the processing to be performed is shown. Specifically, FIG. 19A particularly relates to an example in which it is determined whether or not the manuscript corresponding to the manuscript image has passed the valid period and whether or not the manuscript has been signed. Shown.

Since the processing of S1901 to S1903 is substantially the same as that of S1701 to S1703 in FIG. 17A, detailed description thereof will be omitted.
In S1910, the determination unit 424 determines whether or not the receipt is signed by the applicant himself / herself. Specifically, the determination unit 424 determines whether or not information matching the full name of the applicant himself / herself has been extracted as a result of the character recognition process for the original image. At this time, the determination unit 424 recognizes the full name of the applicant himself / herself based on the information set as "Fullname" among the responses shown in FIG. 8 returned to the processes of S504 and S505 of FIG. Just do it. It should be noted that the process of S1910 corresponds to an example of the determination process relating to the determination of whether or not the image corresponding to the reading result of the original contains the information related to the user's signature.

When the determination unit 424 determines in S1910 that the receipt is signed by the applicant himself / herself, the determination unit 424 proceeds to the process in S1911.
In S1911, the determination unit 424 provides information on the receipt determined to be signed by the applicant himself / herself and information (coordinates) regarding the position where the signature is made in the manuscript image of this receipt. Hold. In the present embodiment, the determination unit 424 holds the information regarding the receipt and the information regarding the position where the signature is made by storing it in the RAM 213 or the HDD 214.

On the other hand, when the determination unit 424 determines in S1910 that the receipt is not signed by the applicant himself, the process proceeds to S1912.
In S1912, the determination unit 424 holds information regarding the receipt determined that the applicant himself / herself has not been signed. In the present embodiment, the determination unit 424 holds the information regarding the receipt and the information regarding the position where the signature is made by storing it in the RAM 213 or the HDD 214.

Subsequently, among the processes of the image processing system according to the second embodiment, the processes corresponding to S1630 of FIG. 16B will be described. FIG. 19B is a flowchart showing an example of the process related to the presentation of the conditions required for digitizing the receipt. Among the processes of the image processing system according to the second embodiment, the process corresponding to S1630 of FIG. 16B. An example is shown. Specifically, FIG. 19B shows an example of a process relating to the presentation of conditions required for digitizing a receipt.

Since the processes of S1951, S1952, and S1955 are substantially the same as the processes of S1751, S1752, and S1755 in FIG. 17B, detailed description thereof will be omitted.
In S1961, the determination unit 424 retained the information regarding each of the receipt signed by the applicant and the receipt without this signature in the processing of S191 to S1912 of FIG. 19A. See information. Then, the determination unit 424 determines whether or not the applicant himself / herself has signed all the receipts from which the manuscript image has been extracted, based on the reference result of this information.

If the determination unit 424 determines that all the receipts from which the manuscript image has been extracted are signed by the applicant himself / herself in S1961, the process proceeds to S1962.
In the present embodiment, it is assumed that the process of S1962 is substantially the same as the process of S1754 in FIG. 17B. On the other hand, as another example, in S1962, the display control unit 421 includes an effective condition confirmation screen 2000 in which the areas 2031 and 2032 corresponding to the signature are emphasized, which will be described later with reference to FIG. It may be displayed on the display unit. As another example, the determination unit 424 ends the process shown in FIG. 19B (that is, the process corresponding to S1630 in FIG. 16B) without presenting the valid condition confirmation screen 2000 to the display control unit 421. You may.

On the other hand, if the determination unit 424 determines that at least one of the receipts from which the manuscript image has been extracted has not been signed by the applicant himself / herself in S1961, the process proceeds to S1963. ..
In S1963, the display control unit 421 displays a screen on which an image corresponding to the reading result of the document is presented on the display unit provided in the operation unit 220, thereby determining the position on the document table on which each receipt is arranged. Present to the user. The process of S1963 corresponds to an example of an output control process for outputting information according to the result of determination as to whether or not the image obtained by reading the original contains information related to the user's signature to the output unit. Further, whether or not the process of the display control unit 421 acquiring the information according to the judgment result by the judgment unit 424 shown as S1961 includes the information related to the user's signature in the image according to the reading result of the original. It corresponds to an example of the acquisition process for acquiring the information according to the above.

FIG. 20 shows an example of a screen for presenting whether or not the read receipt is valid. Specifically, the valid condition confirmation screen 2000 shows an example of a screen displayed on the display unit included in the operation unit 220 by the display control unit 421 in S1963 of FIG. 19B. The valid condition confirmation screen 2050 is one aspect of the valid condition confirmation screen 2000, but the details will be described later as a modification.
The valid condition confirmation screen 2000 includes a notification information presentation area 2020, a platen information presentation area 2021, a rescan button 2010, and a next button 2011. The broadcast information presentation area 2020 and the document stand information presentation area 2021 correspond to the broadcast information presentation area 1860 and the document stand information presentation area 1861 shown in FIG. Further, the rescan button 2010 and the next button 2011 correspond to the rescan button 1810 and the next button 1811 shown in FIG.

The display control unit 421 provides notification information indicating that there is a receipt not signed by the applicant himself / herself in the notification information presentation area 2020 of the valid condition confirmation screen 2000, and a suggestion of the corresponding receipt. Present.
In addition, the display control unit 421 presents the position on the platen on which each receipt is arranged by displaying an image according to the reading result of the document in the platen information presentation area 2021. At this time, the display control unit 421 inverts the left and right of the image according to the reading result of the original in order to make the position of each receipt on the platen easier to understand intuitively, and after this inversion, The image may be displayed on the above screen.

The display control unit 421 controls so that the area corresponding to this signature is emphasized among the areas where the receipt determined to be signed by the applicant himself / herself is presented. Notify the user that the signature has been made. For example, the display control unit 421 corresponds to the signature in the area where the receipt signed by the applicant himself / herself is presented among the images presented in the manuscript stand information presentation area 2021 of the valid condition confirmation screen 2000. A shaded image is superimposed on the areas 2031 and 2032. By applying such processing, of the areas where the receipt signed by the applicant himself / herself is presented, the areas 2031 and 2032 corresponding to the signature and the other areas are in different modes. Areas 2031 and 2032, which correspond to signatures, are more emphasized as they are presented. By performing such control, the display control unit 421 notifies the user of the existence of the receipt signed by the applicant himself / herself.

The display control unit 421 controls the area where the receipt determined to be unsigned by the applicant is presented so as to emphasize the position on the platen on which the receipt is placed. Present to the user. For example, the display control unit 421 shades the image presented in the manuscript stand information presentation area 2021 of the valid condition confirmation screen 2000 in the area 2033 in which the receipt unsigned by the applicant is presented. The image of is superimposed. By performing such control, the display control unit 421 emphasizes the area 2033 in which the receipt unsigned by the applicant himself / herself is presented.

By executing the above processing, the applicant himself / herself can intuitively recognize the position of the unsigned receipt on the platen of the scanner 222. .. In addition, since the signed portion of the signed receipt is emphasized, the applicant himself / herself can easily confirm whether or not each receipt has been signed.

(Modification example 1)
A modification 1 of the image processing system according to the second embodiment will be described below.
In the example described as the second embodiment, the determination unit 424 signs the receipt by the applicant himself / herself depending on whether or not the information matching the full name of the applicant himself / herself exists in the manuscript image. It was judged whether or not it was done. On the other hand, depending on the receipt, information that matches the full name of the applicant may be included in addition to the signature by the applicant. As a specific example, a receipt provided with an area for addressing may include the applicant's full name in the area for addressing, but this full name is signed by the applicant. The information will be different from.
In view of the above situation, in the image processing system according to the first modification, the determination unit 424 identifies information that matches the applicant's full name contained in the manuscript image and is different from the signature by the applicant. Then, it is excluded from the recognition target as a signature by the applicant himself / herself.

Specifically, in S1910 of FIG. 19A, the determination unit 424 uses the analysis result of the manuscript image to determine whether the information contained in the manuscript image that matches the full name of the applicant is the signature of the applicant. Is determined. Specifically, the determination unit 424 provides information indicating that the full name is the address in the vicinity of each information that matches the full name of the applicant himself / herself based on the result of the character recognition processing for the manuscript image. Determine if it exists.
For example, in Japan, information representing honorific titles such as "sama" and "middle" is displayed in the area where the address is described in the receipt. Utilizing such a characteristic, the determination unit 424 determines whether or not there is information representing a title for each information that matches the full name of the contractor. Then, the determination unit 424 excludes the information that matches the full name of the applicant himself / herself, which exists in the vicinity of the information representing the title, from the recognition target as the signature by the applicant himself / herself.
Further, as another example, not only the information representing the honorific title illustrated above but also other information indicating that the full name described in the predetermined area is the address may be displayed in the manuscript. In such a case, the determination unit 424 determines whether or not this full name is signed by the applicant himself / herself, depending on whether or not other information exists in the vicinity of the information that matches the full name of the contractor himself / herself. May be determined.

The valid condition confirmation screen 2050 of FIG. 20 shows one aspect of the valid condition confirmation screen 2000, and in particular, shows an example of a screen presented to the user in the image processing system according to the first modification.
The display control unit 421 confirms the valid conditions for the area 2062 in which the receipt unsigned by the applicant is presented among the images presented in the document stand information presentation area 2021 on the valid condition confirmation screen 2050. The shaded image is superimposed as in the example of the screen 2000.

Further, the display control unit 421 corresponds to the signature in the area where the receipt signed by the applicant himself / herself is presented among the images presented in the manuscript stand information presentation area 2021 on the valid condition confirmation screen 2050. A shaded image is superimposed on the region 2061. At this time, the display control unit 421 determines that the area corresponding to the information corresponding to the full name of the applicant himself / herself located in the area where the address is described in the receipt is based on the judgment result by the judgment unit 424. It is judged that it is not a signature by, and it is excluded from the target of shading.

By executing the above processing, the applicant himself / herself can more accurately confirm whether or not each receipt has been signed.

(Modification 2)
A modification 2 of the image processing system according to the second embodiment will be described below.
In the example described as the second embodiment, the determination unit 424 uses the result of applying the character recognition process to the manuscript image of the receipt to sign the receipt by the applicant himself / herself. It is judged whether or not. On the other hand, some signatures are difficult to recognize as characters by character recognition processing.
In view of such a situation, the judgment unit 424 holds an image of the signature of the applicant himself / herself in advance, and extracts an image similar to this image from the manuscript image of the receipt to obtain the receipt. It may be determined whether or not the applicant has signed the signature. The image of the signature of the applicant may be stored in the HDD 214.

(Modification 3)
A modification 3 of the image processing system according to the second embodiment will be described below.
In the example described as the second embodiment, the determination unit 424 determines whether or not the receipt from which the manuscript image has been extracted has passed the expiration date, and then the applicant himself / herself with respect to this receipt. It is judged whether or not the signature is made by. On the other hand, the judgment unit 424 determines whether the receipt from which the manuscript image has been extracted has passed the expiration date, and whether the receipt is signed by the applicant himself / herself. It may be determined whether or not. That is, the determination unit 424 determines whether or not the receipt from which the manuscript image has been extracted has been signed by the applicant himself / herself regardless of whether or not the receipt has passed the expiration date. You may.
As described above, which judgment is applied to the receipt from which the manuscript image is extracted by the judgment unit 424 is appropriately changed according to the use case to which the image processing system according to the present disclosure is applied. Just do it.

<Supplement>
The contents described as the first embodiment and the second embodiment are merely examples of application of the technology according to the present disclosure, and some of them are appropriately modified without departing from the technical concept of the technology according to the present disclosure. May be good.

For example, in the example described with reference to FIG. 17A, the determination unit 424 sets the expiration date of the receipt to "3 days from the day after the manuscript is issued", and whether or not the expiration date has passed. Whether or not this receipt is valid is determined according to the above. On the other hand, the determination unit 424 may determine whether or not the receipt is valid based on other conditions.
For example, the determination unit 424 may apply a condition different from the condition of "3 days from the day after the manuscript is issued" as a condition regarding the expiration date of the receipt. As a specific example, the determination unit 424 may set the expiration date of the receipt to one week or one month.
Further, the determination unit 424 may determine whether or not the expiration date has passed, not only on a daily basis but also on an hour, minute, or second basis. In this case, the determination unit 424 may determine the expiration date of the receipt based on the information regarding the timing in units of hours, minutes, or seconds extracted from the original image of the receipt.
Further, the determination unit 424 may use a condition other than the expiration date to determine whether or not the receipt is valid. As a specific example, the determination unit 424 may use the information about the country or region where the receipt was issued to determine whether the receipt is valid or not. Further, the determination unit 424 may change the conditions applied to the determination of whether the receipt is valid or not based on the instruction from the user via the operation unit 220 or the like.

Further, in the example described with reference to FIG. 19A, the determination unit 424 includes information matching the full name of the applicant himself / herself in the manuscript image as a determination as to whether or not the receipt is signed. It is judged whether or not it is. On the other hand, it may be assumed that the applicant's full name is not required as a signature. In this case, the determination unit 424 may determine whether or not the manuscript image contains information satisfying the conditions as the signature of the applicant himself / herself. As a specific example, the determination unit 424 may recognize the information indicating the surname of the applicant as a signature, or may recognize the information indicating the name of the applicant as a signature. That is, when the manuscript image contains information on at least one of the surname and first name of the user of the system such as the applicant himself / herself, the determination unit 424 recognizes this information as the signature of the target user. You may.

Further, if it is possible to realize each of the series of functions shown in FIG. 4, the system configuration of the image processing system according to the present disclosure, the functional configuration of the image processing system, and each device constituting the image processing system The hardware configuration of is not limited.
As a specific example, the MFP 110 may play the role of the service providing device 120. That is, each function of the MFP 110 and each function of the service providing device 120 may be realized by one information processing device. In this way, one device may play the role of two or more of the MFP 110, the service providing device 120, and the expense reimbursement device 130. In this case, the hardware configuration of this device may be appropriately changed according to the role played by this one device. As a specific example, the device that plays the role of the MFP 110 may include hardware included in the MFP 110, such as a printer 221 and a scanner 222.

<Other Embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or recording medium, and one or more processors in the computer of the system or device reads and executes the program. It is also possible to realize the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

110 MFP
120 MFP cooperation service 130 Expense settlement system 423 Cooperation service request unit 424 Judgment unit 432 Image processing unit 433 Expense settlement system request unit 441 Request control unit 442 Expense information management department

Claims (14)

An information processing device including an output control means for outputting information according to whether or not the image obtained by reading a manuscript contains information related to a user's signature to an output unit. The information processing apparatus according to claim 1, wherein the output control means causes the output unit to output information in which the information related to the signature included in the image is emphasized when the image contains information related to the signature. .. The information processing device according to claim 1 or 2, wherein the output control means causes the output unit to output the image in a manner depending on whether or not the image contains information related to the signature. Further provided with a determination means for determining whether or not the image contains information regarding the signature.
The output control means causes the output unit to output the information according to the result of the determination by the determination means.
The information processing device according to any one of claims 1 to 3. The information processing apparatus according to claim 4, wherein the determination means determines, as the determination, whether or not the information regarding at least one of the user's first name and last name included in the image is the information related to the signature. .. The determination means
Depending on whether or not information regarding at least one of the surname and the first name is displayed in association with the information indicating the address.
It is determined whether or not the information regarding at least one of the surname and the given name contained in the image is the information related to the signature.
The information processing device according to claim 5. The information indicating the address is information indicating the title, and is
The determination means
Depending on whether or not information regarding at least one of the surname and the given name is displayed in association with the information indicating the title.
It is determined whether or not the information regarding at least one of the surname and the given name contained in the image is the information related to the signature.
The information processing device according to claim 6. The determination means
Based on the information contained in the image, it is determined whether or not the condition regarding the expiration date of the manuscript is satisfied.
It is determined whether or not the image contains information regarding the signature according to the result of determining whether or not the condition regarding the expiration date of the manuscript is satisfied.
The information processing device according to any one of claims 4 to 7. The determination means determines whether or not the expiration date of the manuscript has passed as a determination as to whether or not the condition regarding the expiration date of the manuscript is satisfied based on the information regarding the timing at which the manuscript is issued contained in the image. The information processing apparatus according to claim 8. The information processing device according to claim 8 or 9, wherein the output control means outputs information according to a result of determination as to whether or not the condition regarding the expiration date of the manuscript is satisfied to the output unit. The output control means passes the image of the manuscript determined to have passed the expiration date and the expiration date as information according to the determination of whether or not the condition regarding the expiration date of the manuscript is satisfied. The information processing apparatus according to claim 10, wherein the information processing device according to claim 10 causes an output unit to output information presented in a mode different from that of an image of a manuscript determined not to be processed. The information processing device according to any one of claims 1 to 11, wherein the manuscript is a receipt. It is an information processing method executed by an information processing device.
An information processing method that includes an output control step that causes the output unit to output information according to whether or not the image obtained by reading the manuscript contains information related to the user's signature. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 12.

Images