JP7843384B1 - A document service system, the methods that the document service system performs, and a program that causes a computer to perform the methods that the document service system performs. - Google Patents

Abstract

[Problem] To automatically read characters from document data and make it easier to provide users with the information they intend.
[Solution] The document service system (1) comprises a model storage unit (10) and a processing unit (20). The model storage unit (10) is configured to automatically read characters in document data (4) obtained from a user terminal (3) when the document data (4) is input to it, and to extract and output a plurality of candidates that can be selected as the content of a predetermined item from the read characters. The processing unit (20) obtains a plurality of candidates that can become the content of a display item by inputting the document data (4) obtained from the user terminal (3) into the model, and displays the obtained plurality of candidates on the screen of the user terminal (3) in a manner that allows the user to select.
[Selection Diagram] Figure 1

Description

Applicable under Article 30, Paragraph 2 of the Patent Law. Published on July 31, 2024, on the website https://bakuraku.jp/news/20240731-2/. [Publications, etc.] Published on December 14, 2024, on the website https://docs.google.com/spreadsheets/d/1SONWWD0jPFqT5PjlsS2OC7x_4ati3mbv0vCnysaHlt8/edit? Published at gid=539721170#gid=539721170 [Publications, etc.] Published at the ComputerVision field trend survey and symposium on international development held on December 14, 2024 [Publications, etc.] Published in the product description attached to an email on December 3, 2024 [Publications, etc.] Published in the product description attached to an email on December 17, 2024 [Publications, etc.] Published in the product description attached to an email on December 19, 2024 [Publications, etc.] Published in the product description attached to an email on January 7, 2025 [Publications, etc.] Published in the product description attached to an email on January 23, 2025

This disclosure relates to a document service system configured to communicate with a user's terminal, a method performed by the document service system, and a program that causes a computer to perform the same method.

Japanese Patent Publication No. 2020-13281 (Patent Document 1) and Japanese Patent Publication No. 7561378 (Patent Document 2) describe a system that automatically reads character information from document data such as invoices using OCR (Optical Character Recognition/Reader), and automatically extracts a single character piece of information as the content of a predetermined item using AI (Artificial Intelligence) and provides it to the user.

Japanese Patent Publication No. 2020-13281 Patent No. 7561378

In the systems described in Patent Documents 1 and 2, when the document data to be processed is identical, the character information extracted as the content of each item is uniquely identified.

However, even if the document data being processed is identical, the information to be extracted as content for each item will differ for each user due to differences in their operational procedures and roles. Despite this, if the textual information extracted as content for each item is uniquely specified, the user will only receive information that differs from their intention. Patent documents 1 and 2 do not mention this problem or any countermeasures.

This disclosure was made to solve the aforementioned problems, and its purpose is to facilitate the automatic reading of characters in document data and the provision of information intended by the user.

(Section 1) The document service system provided by this disclosure is a document service system configured to communicate with a user's terminal, and comprises a model storage unit that stores a model configured to automatically read characters in document data obtained from the user's terminal, extract multiple selectable candidates for the content of predetermined items from the read characters, and output them; and a processing unit that performs processing using the model. The processing unit obtains multiple candidates by inputting document data obtained from the user's terminal into the model, and displays the obtained multiple candidates on the screen of the user's terminal in a manner that allows the user to select them.

(Paragraph 2) In the document service system described in Paragraph 1, the processing device displays multiple candidates and a layout image of the document data side by side on the user's terminal screen, and highlights the location of the candidate selected by the user from among the multiple candidates on the layout image.

(Section 3) In the document service system described in Section 1, the model is configured to extract multiple candidates and output the priority of the multiple candidates upon input of document data. The processing device inputs document data obtained from the user's terminal into the model, obtains multiple candidates and their priorities, and displays the obtained priorities on the user's terminal screen in a manner recognizable by the user.

(Paragraph 4) In the document service system described in Paragraph 3, the processing device displays a list of multiple candidates arranged in order of priority on the user's terminal screen.

(Paragraph 5) In the document service system described in Paragraph 4, the processing device defaults to displaying the candidate with the highest priority among multiple candidates on the user's terminal screen, and displays a list of multiple candidates arranged in descending order of priority on the user's terminal screen in response to user operations.

(Paragraph 6) In the document service system described in Paragraph 1, the processing device displays information representing the meanings of multiple candidates, in addition to the multiple candidates themselves, on the user's terminal screen.

(Section 7) The document service system described in Section 1 further comprises a database that acquires and stores candidate selected by the user from among multiple candidates as feedback information from the user's terminal, and a learning device that trains a model using the feedback information stored in the database as training data. The processing device acquires multiple candidates by inputting document data into the model trained by the learning device and displays them on the user's terminal screen.

(Section 8) In the document service system described in Section 7, the database stores feedback information in association with user transaction information to identify the user's transaction trends. The learning device learns a model based on the user transaction information. The processing device inputs document data acquired from the user's terminal and user transaction information into the model, thereby acquiring multiple candidates suitable for the user's operations and displaying them on the user's terminal screen.

(Section 9) In the document service system described in Section 7, the database stores document types indicating the type of document data, in addition to feedback information. The learning device learns a model based on the document types. The processing device inputs document data acquired from the user's terminal into the model, thereby acquiring multiple candidates suitable for the type of document data and displaying them on the user's terminal screen.

(Clause 10) In the document service system described in Clause 7, the feedback information includes semantic information representing the meaning of the user-selected candidates, in addition to the candidates selected by the user. The learning device uses the semantic information contained in the feedback information to train its model.

(Paragraph 11) The method described herein is performed by a document service system configured to communicate with a user's terminal. The document service system stores a model configured to automatically read characters in document data received from the user's terminal, extract multiple selectable candidates from the read characters, and output them as content for predetermined items. The method performed by the document service system includes the steps of: obtaining multiple candidates by inputting document data received from the user's terminal into the model; and displaying the obtained candidates on the user's terminal screen in a manner that allows the user to select them.

(Paragraph 12) The program provided herein is a program that causes a computer to perform the same actions as a document service system configured to communicate with a user's terminal. The document service system stores a model configured to automatically read characters from document data received from the user's terminal, extract multiple selectable candidates from the read characters, and output them as content for predetermined items. The program causes the computer to perform the steps of: obtaining multiple candidates by inputting document data received from the user's terminal into the model; and displaying the obtained multiple candidates on the user's terminal screen in a manner that allows the user to select them.

According to this disclosure, it is possible to automatically read characters in document data and make it easier to provide users with the information they intend.

This diagram schematically shows the overall system configuration, including the document service system. This is a flowchart illustrating an example of the processing steps performed by the document service system. This is a schematic diagram (part 1) showing an example of a user terminal image. This is a schematic diagram (part 2) illustrating an example of a user terminal image. This is a schematic diagram (part 3) illustrating an example of a user terminal image. This diagram schematically illustrates an example of a data structure stored in a database. This diagram schematically shows the AI-OCR processing results when a document service system receives the same document data from companies A, B, and C, respectively. This diagram schematically shows the AI-OCR processing results when the document type is "invoice" and when the document type is "receipt".

The embodiments of this disclosure will be described in detail with reference to the drawings. Parts identical or corresponding to those shown in the drawings are denoted by the same reference numerals, and their descriptions will not be repeated.

[System Configuration]
Figure 1 is a schematic diagram showing the overall configuration of System 1, which includes the document service system 2 according to this embodiment. This System 1 comprises the document service system 2 and a user terminal 3.

The document service system 2 is configured to communicate with user terminals 3. User terminal 3 is a terminal operated by a user utilizing the document service system 2. User terminal 3 may be a computer equipped with peripherals such as a display, keyboard, and mouse, or it may be a smartphone. Although Figure 1 shows one user terminal 3, the document service system 2 can communicate with multiple user terminals 3 used by multiple users.

The document service system 2, upon receiving document data 4 and user transaction information from the user terminal 3, automatically reads the characters in the received document data 4 using OCR and performs a process (hereinafter also referred to as "AI-OCR processing") that automatically extracts character information appropriate to the content of predetermined display items using AI. The user transaction information is information used to identify the user's transaction trends. This user transaction information includes a user ID to identify the user, a document ID to identify the document data, a trading partner ID to identify the user's trading partner, and a tenant ID to identify the tenant. The document service system 2 displays the results of the AI-OCR processing on the screen of the user terminal 3 identified by the user transaction information.

Specifically, the document service system 2 includes a model storage unit 10, a processing unit 20, a database 30, and a learning device 40.

The model storage unit 10 includes a first storage unit 12 and a second storage unit 14. The first storage unit 12 stores the first model. The second storage unit 14 stores the second model.

The first model is a model (candidate output model) that automatically reads the characters in document data 4 using OCR when document data 4 is input, and extracts and outputs multiple selectable options from the read characters as content for predetermined display items. The predetermined display items include multiple items such as "Document Type" indicating the type of document read (invoice, receipt, etc.), "Customer Name" indicating the name of the customer, "Amount" indicating the transaction amount, and "Transaction Date" indicating the transaction date.

The first model is configured to output multiple candidates for each of several items, each of which is suitable for the content of that item. For example, if document data 4 contains three dates, the first model is configured to output all three dates listed in document data 4 as candidates for the "Transaction Date" item, rather than outputting only one of the three dates.

The second model is a priority output model that takes multiple candidates for each item output from the first model as input and outputs the priority of those candidates for each item. For example, if three candidates for a certain item are input to the second model, the second model will stratify the three candidates for that item into the highest priority candidate, the second highest priority candidate, and the lowest priority candidate, and output them.

The processing unit 20 is an arithmetic circuit (arithmetic unit) that executes various processes by running various programs. The processing unit 20 includes a processor such as a CPU (Central Processing Unit), memory, and input/output buffers. The processing unit 20 may be implemented, for example, in a general-purpose computer or in a dedicated computer for document services.

The processing unit 20 performs the AI-OCR processing described above using the models (first model and second model) stored in the model storage unit 10, and displays the results of the AI-OCR processing on the screen of the user terminal 3. Specifically, the processing unit 20 inputs the document data 4 acquired from the user terminal 3 into the models stored in the model storage unit 10, thereby acquiring multiple candidates for each item and the priority of those candidates.

The processing unit 20 displays the acquired candidates on the user terminal 3 screen in a manner that allows the user to select one (see Figure 4 below). The processing unit 20 also displays the priority of the acquired candidates on the user terminal 3 screen in a manner that allows the user to recognize it (see Figure 4 below).

Database 30 stores AI-OCR processing results and feedback information from the user terminal 3. The feedback information from the user terminal 3 includes information identifying the candidate selected by the user from among multiple candidates obtained by AI-OCR processing (hereinafter also referred to as "user-selected candidate"). Database 30 stores combinations of AI-OCR processing results and feedback information related to those AI-OCR processing results, associated with user transaction information (see Figure 6 below).

The learning device 40, like the processing unit 20, is configured to include a processor such as a CPU, memory, and input/output buffers. The learning device 40 may be implemented using, for example, a general-purpose computer or a dedicated computer for machine learning. The model learning method used by the learning device 40 is not limited to a specific method, and known methods can be employed.

The learning device 40 uses the feedback information stored in the database 30 as training data to perform machine learning on the models (first model and second model) stored in the model storage unit 10. In this embodiment, the learning device 40 uses the feedback information stored in association with user transaction information to learn the models stored in the model storage unit 10 for each user transaction. This personalizes the models stored in the model storage unit 10 for each user (more specifically, for each user's transaction content). The processing unit 20 inputs the document data 4 and user transaction information acquired from the user terminal 3 into the models stored in the model storage unit 10, thereby acquiring multiple candidates suitable for each user's use of the document data for each item.

Furthermore, the learning device 40 learns the model stored in the model storage unit 10 based on the document type included in the AI-OCR processing result. This ensures that the model stored in the model storage unit 10 is adapted to each user's specific document type needs. The processing unit 20 inputs the document data 4 acquired from the user terminal 3 into the model stored in the model storage unit 10, thereby acquiring multiple candidates for each item that are suitable for each user's specific document type needs.

Figure 2 is a flowchart showing an example of the processing procedure performed by the document service system 2. Of the steps S10 to S60 shown in Figure 2, steps S10 to S30 are the "utilization phase" where the processing unit 20 utilizes the model to perform AI-OCR processing, and steps S40 to S60 are the "learning phase" where the learning device 40 learns the model. While each process shown in Figure 2 is expected to be implemented by software (programs), it may also be implemented by dedicated hardware (electronic circuits).

<Utilization Phase>
Referring to Figure 2, the processing of the utilization phase by the document service system 2 (steps S10 to S30) will be explained.

The processing unit 20 receives document data 4 and user transaction information from the user terminal 3 (step S10).

Next, the processing unit 20 performs the AI-OCR processing described above using the document data 4 and user transaction information received in step S10 (step S20).

Specifically, the processing unit 20 acquires multiple candidates for each display item by inputting the document data 4 and user transaction information received in step S10 into the first model, and sets priorities for the multiple candidates for each display item by inputting the acquired candidates and user transaction information into the second model. For example, if three candidates are acquired for a certain display item, these three candidates for that display item are stratified into the candidate with the highest priority, the candidate with the second highest priority, and the candidate with the lowest priority.

Next, the processing unit 20 transmits the result of the AI-OCR processing in step S20 to the user terminal 3, thereby displaying the result of the AI-OCR processing on the screen of the user terminal 3 in a predetermined layout (step S30). At this time, the processing unit 20 displays multiple candidates for each display item on the screen of the user terminal 3 in a manner that allows the user to select one.

Figure 3 schematically shows an example of a user terminal image 3a that is displayed by default on the screen of the user terminal 3 after processing in step S30 by the processing unit 20. Figure 3 shows an example where the document data 4 received by the processing unit 20 is a receipt from "AAA Corporation" to "XXX Corporation".

On the user terminal 3's screen, the result image 70 showing the result of the AI-OCR processing by the document service system 2 and the layout image 80 of the document data 4 are displayed side by side.

The result image 70 displays several predetermined display items. Figure 3 shows an example where multiple items are displayed, including "Document Type," "Customer Name," "Qualified Business Operator Number," "Amount," and "Transaction Date."

Directly below each display item, a display field is placed to show the content of that item. The highest priority candidate is displayed by default in each display field. Figure 3 shows an example where "Receipt" is displayed in the document type display field 71, "AAA Corporation" is displayed in the customer name display field 72, "aaaaaa" is displayed in the qualified business number display field 73, "7,630" is displayed in the amount display field 74, and "2024-05-30 (Purchase Date)" is displayed in the transaction date display field 75. Note that display fields for items for which no candidates exist are left blank.

The layout image 80 highlights the locations of the text displayed in the display fields of the result image 70. In the example shown in Figure 3, the locations of "AAA Corporation" and "aaaaaaa" displayed in display fields 72 and 73 of the result image 70 are highlighted with a frame 82 in the layout image 80. The location of "7,630" displayed in display field 74 of the result image 70 is highlighted with a frame 84 in the layout image 80, and the location of "2024-05-30 (Purchase Date)" displayed in display field 75 of the result image 70 is highlighted with a frame 85 in the layout image 80. This highlighting allows the user to easily recognize, by looking at the layout image 80, where the text displayed in the display fields of the result image 70 corresponds to the text in the document data 4.

The transaction date display field 75 does not only display the date "2024-05-30", but also includes "(Purchase Date)" after the date, which represents the meaning of that date. This allows users to easily understand the meaning of the date displayed in the transaction date display field 75 simply by looking at it. In other words, if we assume that only the date "2024-05-30" is displayed in the transaction date display field 75, users cannot understand the meaning of May 30, 2024 (the action taken on that date) simply by looking at the transaction date display field 75. In particular, in the example shown in Figure 3, the date "May 30, 2024" is listed in two places in the document data 4: the purchase date field and the boarding date field. Therefore, simply by looking at the transaction date display field 75, it is not possible to determine whether the "2024-05-30" displayed in the transaction date display field 75 is extracted from the purchase date or the boarding date in the document data 4. In contrast, in this embodiment, the transaction date display field 75 displays "2024-05-30 (Purchase Date)," allowing the user to easily recognize that the date displayed in the transaction date display field 75 is the "Purchase Date" simply by looking at it.

When a user performs a predetermined list display operation on the display field of each display item (for example, selecting the display field of a display item with cursor 76), a list containing multiple options that the user can select will be displayed as the content of that display field.

Figure 4 schematically shows an example of a user terminal image 3b when a user performs a list display operation on the transaction date display field 75. In this case, as shown in Figure 4, the candidate list 75a is displayed below the transaction date display field 75.

The candidate list 75a displays candidates in descending order of priority, from top to bottom. In the example shown in Figure 4, the candidate list 75a displays, from top to bottom, the three candidates: "2024-05-30 (Purchase Date)," which has the highest priority and was displayed by default; "2024-05-30 (Date of Travel)," which has the next highest priority; and "2024-05-31 (Display Date)," which has the lowest priority. By checking the order of the candidates in the candidate list 75a, the user can recognize the multiple candidates obtained by the AI-OCR processing by the document service system 2 and their respective priorities.

By performing a predetermined selection operation on the candidate list 75a (for example, selecting one of the candidates within the candidate list 75a using the cursor 76), the date displayed in the transaction date display field 75 can be changed to the date of the candidate selected by the user.

Furthermore, a calendar 75b is displayed below the list of suggested trading dates 75a. Users can also change the date displayed in the trading date display field 75 by selecting a date from the calendar 75b.

Figure 5 schematically shows an example of a user terminal image 3c when the user selects "2024-05-30 (Date of Travel)," which was the second item from the top of the candidate list 75a in Figure 4. In this case, the date displayed in the transaction date display field 75 changes from the default "2024-05-30 (Date of Purchase)" (see Figure 4) to "2024-05-30 (Date of Travel)" (see Figure 5), which the user selected from the candidate list 75a.

Furthermore, in conjunction with the change in the date displayed in the transaction date field 75 to "2024-05-30 (date of travel)," the location of the highlighted frame 85 on the layout image 80 changes from the location where the purchase date is listed (see Figure 4) to the location where the travel date is listed (see Figure 5). This allows the user to easily recognize where their selected option is located in the document data 4 by looking at the layout image 80.

Figures 4 and 5 show examples of displaying and selecting candidate transaction dates, but it is also possible to display and select candidates for other display items. For example, if a user wants to change the default "AAA Corporation" displayed in the customer name display field 72, the user can perform a list display operation on the customer name display field 72 to display a list of candidates for that field directly below it, and then select the desired candidate from the displayed list to change the content displayed in the customer name display field 72.

A confirmation button 90 is displayed below the result image 70. When the user selects and presses the confirmation button 90, the content displayed in the result image 70 is confirmed as the content requested by the user. The confirmed content is sent back from the user terminal 3 to the document service system 2 as feedback information indicating the content requested by the user. The feedback information includes not only the user's selection candidates, but also semantic information indicating the meaning of the user's selection candidates (for example, if the user's selection candidate is a date, information indicating the action performed on that date), and user transaction information.

<Learning Phase>
Returning to Figure 2, we will now explain the processing of the learning phase by the document service system 2 (steps S40 to S60).

After the AI-OCR processing result is displayed on the user terminal 3 screen in step S30, the learning device 40 determines whether or not feedback information has been returned from the user terminal 3 (step S40). If no feedback information has been returned (NO in step S40), the learning device 40 repeats the process in step S40 and waits for feedback information to be returned.

If feedback information is received from the user terminal 3 (YES in step S40), the learning device 40 saves the feedback information to the database 30 (step S50).

Figure 6 schematically shows an example of the data structure stored in database 30. Database 30 stores multiple pieces of feedback information. Each piece of feedback information is associated with user transaction information and AI-OCR processing results. The AI-OCR processing results include the document type and multiple candidates for each display item.

Furthermore, the feedback information includes not only user selection options but also semantic information indicating the meaning of each user selection option.

The learning device 40 uses the feedback information stored in the database 30 as training data to learn the models (first model and second model) stored in the model storage unit 10 (step S60).

In this process, the learning device 40 learns a model based on user transaction information (user ID, etc.) and AI-OCR processing results (document type, etc.). As a result, the model stored in the model storage unit 10 is personalized for each user (for each user's transaction details) and adapted to the specific use cases of each user's document type. Consequently, in subsequent AI-OCR processing, the user can be presented with multiple candidates and their priorities that best suit their needs.

Furthermore, the learning device 40 learns the model using the semantic information contained in the feedback information. Specifically, the learning device 40 grasps the user's needs from the semantic information contained in the feedback information and learns the model to satisfy the user's needs. As a result, in subsequent AI-OCR processing, the user can be presented with multiple candidates and their priorities that are more suitable to the user's needs.

As described above, in the document service system 2 according to this embodiment, the utilization phase processing (steps S10 to S30), which uses the model to present multiple candidates for each item to the user, and the learning phase processing (steps S40 to S60), which uses feedback information containing the information of the candidate selected by the user from the multiple candidates as training data to train the model, are repeated each time document data 4 is received. As a result, the model stored in the model storage unit 10 is personalized for each user, and content reflecting each user's needs is preferentially displayed on the user terminal 3 screen. Consequently, the user can be presented with AI-OCR processing results that appropriately reflect the user's needs.

In particular, in the document service system 2 according to this embodiment, during the AI-OCR processing in the utilization phase, instead of outputting one candidate for each item, multiple candidates and their priorities are output. This avoids providing the user with information that differs from their intent, compared to the case where only one candidate is output for each item. As a result, it becomes easier to provide the user with the information they intended.

Furthermore, in the document service system 2 according to this embodiment, feedback information is stored in association with user transaction information, and the model is learned based on the user transaction information. This allows for accurate and speedy model learning based on user transaction information. The results of AI-OCR processing using the model learned for each user are then presented to the user. Therefore, information that appropriately reflects each user's needs can be presented to each user.

Thus, the document service system 2 according to this embodiment can present each user with AI-OCR processing results 5B that are appropriate to each user's needs, even with the same document data 4. For example, even with the same receipt, the user's needs may differ depending on whether they want the company name or the official name extracted as the "customer" item. Similarly, the user's needs may differ whether they want the pre-tax amount or the tax-inclusive amount extracted as the "amount" item. The document service system 2 according to this embodiment can appropriately respond to such differences in user needs.

Figure 7 schematically shows the AI-OCR processing results 5A to 5C when the document service system 2 receives the same document data 4 from companies A, B, and C, respectively.

Even if the document data 4 to be processed is the same, the information to be extracted as content for each item will differ depending on the user's needs. As shown in Figure 7, the document service system 2 according to this embodiment, when receiving the same document data 4 from companies A, B, and C, can present company A with an AI-OCR processing result 5A suitable for company A, company B with an AI-OCR processing result 5B suitable for company B, and company C with an AI-OCR processing result 5C suitable for company C.

Furthermore, in the document service system 2 according to this embodiment, the model is trained based on the document type. This allows the system to learn the detailed use cases for each user's document type. The results of AI-OCR processing using the model trained based on the document type are then presented to the user. Therefore, even if the user's needs differ for each document type, the system can present the user with AI-OCR processing results that appropriately reflect each user's needs.

Figure 8 schematically shows the AI-OCR processing result 6A when the document type is "invoice" and the AI-OCR processing result 6B when the document type is "receipt". Even if the user's needs differ between invoices and receipts, as shown in Figure 8, if document data 4 is an invoice, the AI-OCR processing result 6A suitable for the invoice use case can be presented to the user, and if document data 4 is a receipt, the AI-OCR processing result 6b suitable for the receipt use case can be presented to the user.

Furthermore, in the document service system 2 according to this embodiment, the feedback information includes not only user selection candidates but also semantic information indicating the meaning of each user selection candidate. This allows the learning device 40 to learn the model while considering the user's intent in selecting each user selection candidate. Therefore, the process of learning the model for each user can be performed more accurately and quickly.

Furthermore, in the document service system 2 according to this embodiment, the models stored in the model storage unit 10 are divided into a first model (candidate output model) that outputs multiple candidates for each item, and a second model (priority output model) that outputs the priority of the multiple candidates for each item. This allows the first model to be a general-purpose model that can be shared by many users, while the second model can be a model customized for each user. This allows for a clear stratification of the learning objectives of each model. As a result, compared to the case where only one model is stored in the model storage unit 10, the complexity of the overall learning content of the models can be suppressed.

<Variation 1>
Figure 1 above shows an example in which the models stored in the model storage unit 10 are divided into two models (first model and second model). However, the number of models stored in the model storage unit 10 is not necessarily limited to two; it may be three or more, or even just one.

<Modified Example 2>
In Figure 1 above, an example is shown in which the learning device 40 is located outside the processing device 20. However, the learning device 40 may also be located inside the processing device 20 (i.e., the processing device 20 also functions as the learning device 40).

The embodiments disclosed herein should be considered in all respects as illustrative and not restrictive. The scope of this disclosure is indicated by the claims rather than the foregoing description, and all modifications within the meaning and scope of the claims are intended to be included.

1. System, 2. Document service system, 3. User terminal, 3a, 3b, 3c. User terminal images, 4. Document data, 5A, 5B, 5C, 6A, 6B. AI-OCR processing results, 10. Model storage unit, 12. First storage unit, 14. Second storage unit, 20. Processing unit, 30. Database, 40. Learning device, 70. Result image, 71-75. Display fields, 75a. Candidate list, 75b. Calendar, 76. Cursor, 80. Layout image, 82, 84, 85. Frame, 90. Confirmation button.

Claims (16)

A document service system configured to communicate with a user's terminal,
A model storage unit stores a model configured to automatically read characters in document data obtained from the user's terminal, extract multiple candidates that can be selected as content for a predetermined item from the read characters, and output them.
The apparatus comprises a processing device that performs processing using the aforementioned model,
The aforementioned processing apparatus is
The multiple candidates are obtained by inputting the document data acquired from the user's terminal into the model.
The acquired plurality of candidates are displayed on the user's terminal screen in a manner that allows the user to select one .
The processing device is a document service system that displays information representing the meaning of the multiple candidates, in addition to the multiple candidates, on the screen of the user's terminal . A document service system configured to communicate with a user's terminal,
A model storage unit stores a model configured to automatically read characters in document data obtained from the user's terminal, extract multiple candidates that can be selected as content for a predetermined item from the read characters, and output them.
The apparatus comprises a processing device that performs processing using the aforementioned model,
The aforementioned processing apparatus is
The multiple candidates are obtained by inputting the document data acquired from the user's terminal into the model.
The acquired plurality of candidates are displayed on the user's terminal screen in a manner that allows the user to select one.
A database which acquires and stores the candidate selected by the user from among the aforementioned multiple candidates as feedback information from the user's terminal,
The system further comprises a learning device that learns the model using the feedback information stored in the database as training data,
The processing device obtains the multiple candidates by inputting the document data into the model learned by the learning device and displays them on the screen of the user's terminal.
The database stores the feedback information in association with user transaction information for identifying the user's transaction trends.
The learning device learns the model based on the user transaction information,
The aforementioned processing apparatus is
A document service system that inputs document data and user transaction information obtained from the user's terminal into the model, thereby obtaining a plurality of candidates suitable for the user's operations and displaying them on the screen of the user's terminal. A document service system configured to communicate with a user's terminal,
A model storage unit stores a model configured to automatically read characters in document data obtained from the user's terminal, extract multiple candidates that can be selected as content for a predetermined item from the read characters, and output them.
The apparatus comprises a processing device that performs processing using the aforementioned model,
The aforementioned processing apparatus is
The multiple candidates are obtained by inputting the document data acquired from the user's terminal into the model.
The acquired plurality of candidates are displayed on the user's terminal screen in a manner that allows the user to select one.
A database which acquires and stores the candidate selected by the user from among the aforementioned multiple candidates as feedback information from the user's terminal,
The system further comprises a learning device that learns the model using the feedback information stored in the database as training data,
The processing device obtains the multiple candidates by inputting the document data into the model learned by the learning device and displays them on the screen of the user's terminal.
The aforementioned database stores document types indicating the type of document data, in addition to the feedback information.
The learning device learns the model based on the document type,
The aforementioned processing apparatus is
A document service system that inputs document data acquired from the user's terminal into the model, thereby acquiring a plurality of candidates suitable for the type of document data and displaying them on the screen of the user's terminal. A document service system configured to communicate with a user's terminal,
A model storage unit stores a model configured to automatically read characters in document data obtained from the user's terminal, extract multiple candidates that can be selected as content for a predetermined item from the read characters, and output them.
The apparatus comprises a processing device that performs processing using the aforementioned model,
The aforementioned processing apparatus is
The multiple candidates are obtained by inputting the document data acquired from the user's terminal into the model.
The acquired plurality of candidates are displayed on the user's terminal screen in a manner that allows the user to select one.
A database which acquires and stores the candidate selected by the user from among the aforementioned multiple candidates as feedback information from the user's terminal,
The system further comprises a learning device that learns the model using the feedback information stored in the database as training data,
The processing device obtains the multiple candidates by inputting the document data into the model learned by the learning device and displays them on the screen of the user's terminal.
The feedback information includes, in addition to the candidates selected by the user, semantic information representing the meaning of the candidates selected by the user.
The learning device is a document service system that learns the model using the semantic information contained in the feedback information. The aforementioned processing apparatus is
The aforementioned multiple candidates and the layout image of the document data are displayed side by side on the screen of the user's terminal.
A document service system according to any one of claims 1 to 4 , wherein the location of the candidate selected by the user from among the aforementioned multiple candidates is highlighted on the layout image. The model is configured to extract the multiple candidates when the document data is input and to output the priority of the multiple candidates.
The aforementioned processing apparatus is
By inputting the document data obtained from the user's terminal into the model, the plurality of candidates and the priority of the plurality of candidates are obtained.
A document service system according to any one of claims 1 to 4 , wherein the acquired priority is displayed on the screen of the user's terminal in a manner that the user can recognize. The document service system according to claim 6 , wherein the processing device displays a list of the plurality of candidates arranged in order of priority on the screen of the user's terminal. The aforementioned processing apparatus is
The candidate with the highest priority among the aforementioned multiple candidates is displayed by default on the user's terminal screen.
The document service system according to claim 7 , which displays a list of the multiple candidates arranged in descending order of priority on the screen of the user's terminal in response to the user's operation. A method performed by a document service system configured to communicate with a user's terminal,
The document service system stores a model configured to automatically read characters in document data obtained from the user's terminal, extract multiple candidates that can be selected as content for a predetermined item from the read characters, and output them.
The method performed by the aforementioned document service system is:
The steps include obtaining the multiple candidates by inputting document data obtained from the user's terminal into the model,
The process includes the step of displaying the acquired plurality of candidates on the screen of the user's terminal in a manner that allows the user to select them,
A method performed by a document service system , wherein the step of displaying the plurality of candidates on the screen includes the step of displaying information representing the meaning of the plurality of candidates on the screen of the user's terminal, in addition to the plurality of candidates. A method performed by a document service system configured to communicate with a user's terminal,
The document service system stores a model configured to automatically read characters in document data obtained from the user's terminal, extract multiple candidates that can be selected as content for a predetermined item from the read characters, and output them.
The method performed by the aforementioned document service system is:
The steps include obtaining the multiple candidates by inputting document data obtained from the user's terminal into the model,
The steps include displaying the acquired plurality of candidates on the screen of the user's terminal in a manner that allows the user to select one,
The steps include: acquiring and storing the candidate selected by the user from among the aforementioned multiple candidates as feedback information from the user's terminal;
The process includes the step of training the model using the accumulated feedback information as training data,
The step of displaying the plurality of candidates on the screen includes the step of obtaining the plurality of candidates by inputting the document data into the trained model and displaying them on the screen of the user's terminal,
The step of accumulating the candidates selected by the user as feedback information includes the step of accumulating the feedback information in association with user transaction information for identifying the user's transaction trends,
The step of training the model includes the step of training the model based on the user transaction information,
A method performed by a document service system, wherein the step of displaying the aforementioned plurality of candidates on the screen includes the step of inputting document data and user transaction information obtained from the user's terminal into the model to obtain the plurality of candidates suitable for the user's operation and display them on the screen of the user's terminal. A method performed by a document service system configured to communicate with a user's terminal,
The document service system stores a model configured to automatically read characters in document data obtained from the user's terminal, extract multiple candidates that can be selected as content for a predetermined item from the read characters, and output them.
The method performed by the aforementioned document service system is:
The steps include obtaining the multiple candidates by inputting document data obtained from the user's terminal into the model,
The steps include displaying the acquired plurality of candidates on the screen of the user's terminal in a manner that allows the user to select one,
The steps include: acquiring and storing the candidate selected by the user from among the aforementioned multiple candidates as feedback information from the user's terminal;
The process includes the step of training the model using the accumulated feedback information as training data,
The step of displaying the plurality of candidates on the screen includes the step of obtaining the plurality of candidates by inputting the document data into the trained model and displaying them on the screen of the user's terminal,
The step of storing the candidate selected by the user as feedback information includes, in addition to the feedback information, a step of storing a document type indicating the type of document data,
The step of training the aforementioned model includes the step of training the aforementioned model based on the aforementioned document type,
A method performed by a document service system, wherein the step of displaying the plurality of candidates on the screen includes the step of inputting document data obtained from the user's terminal into the model to obtain the plurality of candidates suitable for the type of document data and displaying them on the screen of the user's terminal. A method performed by a document service system configured to communicate with a user's terminal,
The document service system stores a model configured to automatically read characters in document data when document data acquired from the user's terminal is input, and to extract and output multiple candidates that can be selected as the content of a predetermined item from the read characters.
The method performed by the aforementioned document service system is:
The steps include obtaining the multiple candidates by inputting document data obtained from the user's terminal into the model,
The steps include displaying the acquired plurality of candidates on the screen of the user's terminal in a manner that allows the user to select one,
The steps include: acquiring and storing the candidate selected by the user from among the aforementioned multiple candidates as feedback information from the user's terminal;
The process includes the step of training the model using the accumulated feedback information as training data,
The step of displaying the plurality of candidates on the screen includes the step of obtaining the plurality of candidates by inputting the document data into the trained model and displaying them on the screen of the user's terminal,
The feedback information includes, in addition to the candidates selected by the user, semantic information representing the meaning of the candidates selected by the user.
A method performed by a document service system, wherein the step of learning the model includes the step of learning the model using the semantic information contained in the feedback information. A program that causes a computer to perform the same actions as a document service system configured to communicate with a user's terminal,
The document service system stores a model configured to automatically read characters in document data when document data acquired from the user's terminal is input, and to extract and output multiple candidates that can be selected as the content of a predetermined item from the read characters.
The aforementioned program is installed on the computer.
The steps include obtaining the multiple candidates by inputting document data obtained from the user's terminal into the model,
The process involves the user performing the steps of displaying the acquired plurality of candidates on the user's terminal screen in a manner that allows the user to select one ,
A program to be executed by a computer, wherein the step of displaying the plurality of candidates on the screen includes the step of displaying information representing the meaning of the plurality of candidates, in addition to the plurality of candidates, on the screen of the user's terminal . A program that causes a computer to perform the same actions as a document service system configured to communicate with a user's terminal,
The document service system stores a model configured to automatically read characters in document data when document data acquired from the user's terminal is input, and to extract and output multiple candidates that can be selected as the content of a predetermined item from the read characters.
The aforementioned program is installed on the computer.
The steps include obtaining the multiple candidates by inputting document data obtained from the user's terminal into the model,
The steps include displaying the acquired plurality of candidates on the screen of the user's terminal in a manner that allows the user to select one,
The steps include: acquiring and storing the candidate selected by the user from among the aforementioned multiple candidates as feedback information from the user's terminal;
The accumulated feedback information is used as training data to train the model, and the following steps are performed:
The step of displaying the plurality of candidates on the screen includes the step of obtaining the plurality of candidates by inputting the document data into the trained model and displaying them on the screen of the user's terminal,
The step of accumulating the candidates selected by the user as feedback information includes the step of accumulating the feedback information in association with user transaction information for identifying the user's transaction trends,
The step of training the model includes the step of training the model based on the user transaction information,
The step of displaying the aforementioned multiple candidates on the screen includes a program to be executed by a computer, which includes inputting document data and user transaction information obtained from the user's terminal into the model to obtain the aforementioned multiple candidates suitable for the user's operation and display them on the screen of the user's terminal. A program that causes a computer to perform the same actions as a document service system configured to communicate with a user's terminal,
The document service system stores a model configured to automatically read characters in document data obtained from the user's terminal, extract multiple candidates that can be selected as content for a predetermined item from the read characters, and output them.
The aforementioned program is installed on the computer.
The steps include obtaining the multiple candidates by inputting document data obtained from the user's terminal into the model,
The steps include displaying the acquired plurality of candidates on the screen of the user's terminal in a manner that allows the user to select one,
The steps include: acquiring and storing the candidate selected by the user from among the aforementioned multiple candidates as feedback information from the user's terminal;
The accumulated feedback information is used as training data to train the model, and the following steps are performed:
The step of displaying the plurality of candidates on the screen includes the step of obtaining the plurality of candidates by inputting the document data into the trained model and displaying them on the screen of the user's terminal,
The step of storing the candidate selected by the user as feedback information includes, in addition to the feedback information, a step of storing a document type indicating the type of document data,
The step of training the aforementioned model includes the step of training the aforementioned model based on the aforementioned document type,
The step of displaying the plurality of candidates on the screen is a program to be executed by a computer, which includes the step of inputting document data obtained from the user's terminal into the model to obtain the plurality of candidates suitable for the type of document data and display them on the screen of the user's terminal. A program that causes a computer to perform the same actions as a document service system configured to communicate with a user's terminal,
The document service system stores a model configured to automatically read characters in document data obtained from the user's terminal, extract multiple candidates that can be selected as content for a predetermined item from the read characters, and output them.
The aforementioned program is installed on the computer.
The steps include obtaining the multiple candidates by inputting document data obtained from the user's terminal into the model,
The steps include displaying the acquired plurality of candidates on the screen of the user's terminal in a manner that allows the user to select one,
The steps include: acquiring and storing the candidate selected by the user from among the aforementioned multiple candidates as feedback information from the user's terminal;
The accumulated feedback information is used as training data to train the model, and the following steps are performed:
The step of displaying the plurality of candidates on the screen includes the step of obtaining the plurality of candidates by inputting the document data into the trained model and displaying them on the screen of the user's terminal,
The feedback information includes, in addition to the candidates selected by the user, semantic information representing the meaning of the candidates selected by the user.
A program to be executed by a computer, the step of learning the model includes a step of learning the model using the semantic information contained in the feedback information.