JP7456580B2 - Information processing device, information processing system, and information processing method - Google Patents

Description

The present invention relates to an information processing device, an information processing system, and an information processing method.

In order to manage the daily sales at each store, POS (Point Of Sale System) registers have a function that prints out a receipt that tallys up the day's sales. Some users also manually input the information printed on the receipt into a spreadsheet program or similar.

JP 2012-221183 A

By the way, payment receipts are generally long, and many characters are printed on the paper. For this reason, if a general OCR (=Optical Character Recognition) technique is applied to recognize all the characters printed on the payment receipt, there is a problem that the processing time will be long. For example, OCR processing of one payment receipt requires several tens of seconds. Several tens of seconds is a sufficiently long time for a user to wait for the results of OCR processing, and if a user wishes to perform OCR processing on a plurality of payment receipts at once, the working time increases by the waiting time.

The present invention aims to shorten the time required to recognize necessary character information from a receipt.

The invention according to claim 1 includes: a control unit; an acquisition unit that acquires image data of a payment receipt imaged by an imaging unit from the imaging unit; The control unit includes a cutout processing unit that sequentially cuts out partial image data of the number of characters one line at a time, and a character recognition unit that recognizes character information from the cut out partial image data, and the control unit controls the output from the character recognition unit. If there is a registered word corresponding to the character information outputted from the character recognition unit, the entire corresponding line is determined to be the target of character recognition , and if there is no registered word corresponding to the character information output from the character recognition unit, the corresponding line is determined to be the target of character recognition. This is an information processing device that excludes entire lines from character recognition .
The invention according to claim 2 is the information processing apparatus according to claim 1, wherein the control section determines amount information corresponding to the character information output from the character recognition section.
The invention according to claim 3 provides that, for one piece of the partial image data, a line where the character recognition unit ends recognition of the character information is located above a line where the character recognition unit starts recognition of the character information. , an information processing apparatus according to claim 1 or 2 .
The invention according to claim 4 includes : a control unit; an imaging unit that images a payment receipt and outputs image data; an acquisition unit that acquires the image data from the imaging unit; a cutout processing unit that sequentially cuts out partial image data of a fixed number of characters line by line from the beginning of each line ; and a character recognition unit that recognizes character information from the cut out partial image data; , if there is a registered word corresponding to the character information output from the character recognition unit, the entire corresponding line is determined as a target for character recognition , and the registration word corresponding to the character information output from the character recognition unit is determined. This information processing system excludes the entire corresponding line from character recognition when a word does not exist .
The invention according to claim 5 includes a process of acquiring image data of a payment receipt, a process of sequentially cutting out partial image data of a fixed number of characters one line at a time from the beginning of each line containing characters from the image data, a process of recognizing character information from the extracted partial image data; a process of determining the entire corresponding line as a target for character recognition if there is a registered word corresponding to the recognized character information; and a character recognition unit. If there is no registered word corresponding to the character information outputted from the character information, the information processing method includes the step of excluding the entire corresponding line from character recognition targets .

According to the present invention, the time required to recognize necessary character information from a receipt can be reduced.

1 is a diagram illustrating a configuration example of an information processing system used in Embodiment 1. FIG. FIG. 3 is a diagram illustrating an example of the correspondence between words that may be used in a payment receipt and words used in an automatic input service such as sales proceeds. FIG. 2 is a diagram illustrating a configuration example of a user terminal used in Embodiment 1. FIG. FIG. 2 is a diagram illustrating a configuration example of an OCR server used in the first embodiment. 3 is a flowchart illustrating an example of processing operations executed by the OCR server used in the first embodiment. FIG. 3 is a diagram illustrating an example of an image displayed on a display unit of a user terminal. 13A and 13B are diagrams illustrating a portion of character image data corresponding to each line that is used to search for a target character. FIG. 3 is a diagram illustrating a specific example of a search result for a target character. FIG. 3 is a diagram illustrating an example of information linked to information recognized by OCR processing. It is a figure explaining an example of matching. 3 is a flowchart illustrating the content of processing used to select a value. 7 is a diagram illustrating selection of values in step 7. FIG. 12 is a flowchart illustrating the contents of a process for selecting values for each group. FIG. 3 is a diagram illustrating an example in which one is selected from two values linked to the group "cash sales." FIG. 6 is a diagram illustrating an example of selecting one from two values linked to the group "net sales." FIG. FIG. 3 is a diagram illustrating an example of a screen showing automatic input results such as sales proceeds displayed on a display unit of a user terminal. 7 is a diagram illustrating a range in which an OCR server executes OCR processing in Embodiment 2. FIG.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.
<Embodiment 1>
<System configuration>
FIG. 1 is a diagram illustrating a configuration example of an information processing system 1 used in the first embodiment.
The information processing system 1 shown in FIG. 1 performs OCR processing on a plurality of user terminals 100 and image data uploaded from the user terminals 100, and notifies the user terminal 100 of predetermined information from among the recognized characters. It is composed of an OCR server 200.
In the case of the information processing system 1 shown in FIG. 1, the user terminal 100 and the OCR server 200 are connected through the cloud network 300. Therefore, OCR server 200 is an example of a cloud server.

User terminal 100 in this embodiment uploads image data generated by capturing an image of a payment receipt to OCR server 200. The settlement receipt in this embodiment is used to mean a receipt on which information obtained by statistically processing daily sales is printed. In other words, the settlement receipt does not mean a receipt on which the amount associated with each transaction is printed individually, but a receipt on which the amount summed up by item set in advance for each business day is printed. Sales include, for example, cash sales, credit card sales, electronic money sales, and the like. In addition, the payment receipt is characterized in that the value that can be linked to one item is not limited to one. In other words, the payment receipt may include items to which multiple values are linked. On the other hand, in the receipt that the consumer receives, the content of the service that the consumer purchased or received and the corresponding amount are linked on a one-to-one basis.

The user terminal 100 shown in FIG. 1 is assumed to be a smartphone. However, the user terminal 100 may be a terminal other than a smartphone, for example, a tablet terminal.
The user terminal 100 in this embodiment uploads image data of the payment receipt captured by the camera of the main body to the OCR server 200. However, it is also possible to image the payment receipt with a digital camera or scanner that is a device different from the user terminal 100, obtain image data output from each device, and upload it to the OCR server 200.
Note that a notebook computer or a desktop computer may be used as the user terminal 100 as long as the image data captured by a digital camera or the like is uploaded to the OCR server 200.

The user terminal 100 and the OCR server 200 in this embodiment cooperate to provide an automatic input service for sales proceeds and the like. For this reason, a dedicated application program (hereinafter referred to as "app") is installed on the user terminal 100 to cooperate with the automatic input service for sales proceeds and the like. Therefore, the user takes an image of the payment receipt according to the instructions of the activated application.
Of course, if another type of application is being executed on the user terminal 100, the object to be imaged is not limited to the payment receipt.

The OCR server 200 in this embodiment provides an automatic input service for sales proceeds and the like in cooperation with an application executed on the user terminal 100. The OCR server 200 executes a function of reading numerical values related to sales etc. from the image data of the received settlement receipt and notifying the user terminal 100, which is the upload source.
Note that the expression of words related to sales etc. printed on the payment receipt differs depending on the manufacturer of the POS register. Therefore, the OCR server 200 in this embodiment is provided with a table that records the correspondence between words used in an application executed on the user terminal 100 and words that may be printed on the payment receipt. . In this embodiment, the words registered in the table among the words printed on the payment receipt are referred to as registered words.

FIG. 2 is a diagram illustrating an example of the correspondence between words that may be used in a payment receipt and words handled in an automatic input service such as sales proceeds.
In FIG. 2, words handled by the automatic input service such as sales proceeds are shown in the group column. The correspondence shown in FIG. 2 is held in the OCR server 200 as a registered word table. In the case of Figure 2, examples of registered words corresponding to "net sales" in the group column include "net sales", "net sales", "net sales total", "net sales", and "**total". There is. The fluctuation in the expression of registered words depends on the settings of the POS register manufacturer.
In addition, "cash" and "cash balance" are exemplified as registered words corresponding to "cash sales" in the group column.
Note that the words set in the group column are just examples. Further, the words set in the group column depend on the expressions used in automatic input services such as sales proceeds.

<Configuration of user terminal>
FIG. 3 is a diagram illustrating an example of the configuration of the user terminal 100 used in the first embodiment.
The user terminal 100 is a so-called computer. The user terminal 100 has a control unit 101 that controls the operation of each unit, a camera 102, a display unit 103, an operation input unit 104, a storage unit 105, and a communication unit 106.
The control unit 101 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. (not shown). The ROM stores data such as a BIOS (Basic Input Output System). The RAM is used as a working area for applications.

In this embodiment, the control unit 101 displays a screen that prompts to take an image of a payment receipt, uploads the imaged image data to the OCR server 200, and Display of a screen in which numerical values such as sales proceeds notified from the OCR server 200 are embedded is executed.
Of course, if another application is activated, the control unit 101 executes processing according to the activated application.
Camera 102 is an example of an imaging unit. In the case of this embodiment, camera 102 is provided in the main body of user terminal 100. Note that there are no restrictions on the use of the camera 102, and the camera 102 can be used for purposes other than capturing images of payment receipts.
The display unit 103 is, for example, a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 103 displays a confirmation screen for images captured by the camera 102, automatically entered sales proceeds, and the like.

The operation input unit 104 includes, for example, a touch sensor or a physical button. An operation input unit 104 as a touch sensor is arranged on the surface of the display unit 103 and constitutes a touch panel.
The storage unit 105 is composed of, for example, a semiconductor memory, and stores captured image data 105A and applications. Note that when the user terminal 100 is a notebook computer or a desktop computer, the storage unit 105 may be a hard disk device.
The communication unit 106 is a device used for communication with an external terminal connected to the cloud network 300 or the like. The communication unit 106 complies with various communication standards. The communication unit 106 complies with, for example, wireless LAN (Local Area Network) and wired LAN communication standards, and mobile communication systems such as 4G and 5G.

<OCR Server Configuration>
FIG. 4 is a diagram illustrating an example of the configuration of the OCR server 200 used in the first embodiment.
The OCR server 200 is a so-called computer. The OCR server 200 includes a control unit 201 that controls the operation of each unit, a storage unit 202, and a communication unit 203.
The control unit 201 is composed of a CPU, a ROM, a RAM, etc. (not shown). The ROM stores data such as the BIOS. The RAM is used as a working area for the application. The OCR server 200 is an example of an information processing device.

The control unit 201 in this embodiment functions as an image data acquisition unit 201A, a character extraction processing unit 201B, an OCR processing unit 201C, a matching unit 201D, a value selection unit 201E, and a group result selection unit 201F through the execution of the application. do. These functional units implement an automatic input service in cooperation with the user terminal 100.
The image data acquisition unit 201A is a functional unit that acquires the image data 105A from the user terminal 100. The image data acquisition unit 201A is an example of an acquisition unit, and stores the acquired image data 105A in the storage unit 202 as image data 202A.

The character cutout processing unit 201B is a functional unit that cuts out character image data including characters from the image data 202A. An image portion corresponding to a line containing characters is an example of character image data. Further, the character cutout processing section 201B is an example of a cutout processing section.
The character extraction processing unit 201B in this embodiment extracts a predetermined specific portion from the extracted character string image data, performs OCR processing on it, and determines whether or not a line includes a registered word. The character extraction processing unit 201B in this embodiment uses two characters appearing at the beginning of character image data as a specific portion.

The character cutout processing unit 201B starts processing the image data 202A with each character in the image data adjusted so that the top and bottom (top and bottom) are in the correct orientation.
In the case of this embodiment, two characters from the beginning of each line containing characters are extracted and subjected to OCR processing, but the characters to be extracted may be one character appearing at the beginning of each line, or three or more characters.
Note that the number of characters to be cut out from the beginning of each line is set in advance. The number of characters may be determined as long as it is possible to determine whether or not the registered word is included in the character image data. Therefore, the number of characters to be cut out is determined depending on the content of the registered word. However, it is preferable to cut out fewer characters because the OCR processing time will be shorter. The portion where the first two characters are printed is an example of the left end portion of the character image data.

In the case of this embodiment, the two read characters are part of the item section. If the two extracted characters include a part of the beginning of the registered word, the character extraction processing unit 201B determines the entire corresponding line of characters to be subjected to OCR processing. On the other hand, if the two extracted characters do not include the first part of the registered word, the character extraction processing unit 201B moves on to determination of the character image data of the next line.
Because it OCR-processes only the first few characters of the character image data and determines whether or not to OCR-process the entire line of characters, compared to the case where all the characters on all lines are OCR-processed, The number of characters to be subjected to OCR processing can be reduced. This leads to a reduction in the time required for OCR processing.

The character cutout processing unit 201B in this embodiment subjects character image data to determination in order from the lower end to the upper end of the image data 202A. This is because the characters such as sales proceeds to be automatically input in this embodiment appear at the bottom of the payment receipt regardless of the difference in POS register. In other words, the character cutout processing unit 201B scans the image data 202A from the bottom end to the top end.
When character image data in which no characters are printed in a specific part is found, the character extraction processing unit 201B converts the line above it in which a character first appears in the specific part to the parent character image data corresponding to the processing. Link as the row (parent row). In other words, the numeric value part of a line that has a parent line is linked to the item part of the parent line. A line that has a parent line is also set to a line where the characters of the entire line are subjected to OCR processing.

The OCR processing unit 201C performs OCR processing on character image data of a specific portion or a line set as a target for OCR processing to recognize character information. The OCR processing unit 201C is an example of a character recognition unit.
The OCR processing unit 201C in this embodiment decomposes character image data corresponding to a line set for OCR processing into an item part and a numerical part, and applies a dedicated OCR model to each part to correspond to each part. Recognize text information.
The OCR model here is generated using deep learning technology. When the OCR model receives character image data identified as a single character, it outputs likely character information (for example, a character code). A confidence level is calculated for each character, indicating how likely it is to be recognized. Note that the reliability of a recognized word is given, for example, as the average value of the reliability of each character composing the word.

The matching unit 201D refers to the registered word table 202B and links the character information recognized as a result of OCR processing (hereinafter referred to as "raw OCR result") to the words of the corresponding group according to the registered word table 202B. This is a function to perform.
The value selection section 201E has a function of selecting one value to be associated with the corresponding item when one item (item section) has a plurality of values. If one item has multiple values, a parent row may exist. For example, when two values are associated with one item, the value selection unit 201E selects the value that can be converted into Currency type. However, if both of the two values are convertible to the Currency type, the value selection unit 201E selects the larger of the two values.
The group-by-group result selection unit 201F has a function of selecting one value to be associated with the corresponding word when one word in the group column has a plurality of values. For example, when two values are associated with one word of a group, the group-by-group result selection unit 201F selects the value with higher reliability as a result of OCR processing the item part.

The storage unit 202 stores the uploaded image data 202A, the registered word table 202B, and an application that cooperates with the application on the user terminal 100 side to realize an automatic input service for sales proceeds and the like.
The storage unit 202 is composed of, for example, a hard disk device or a semiconductor memory. Note that the storage unit 202 may be connected to the cloud network 300 as a cloud storage server.
The communication unit 203 is a device used for communication with an external terminal connected to the cloud network 300 or the like. The communication unit 203 complies with various communication standards. The communication unit 203 complies with, for example, communication standards of wireless LAN and wired LAN, and mobile communication systems such as 4G and 5G.

<Processing operation>
Next, an example of the processing operation executed by the OCR server 200 will be described.
FIG. 5 is a flowchart illustrating an example of processing operations performed by the OCR server 200 used in the first embodiment. The symbol S shown in the figure means a step. The processing operation shown in FIG. 5 is an example of an information processing method.
Note that the series of operations executed by the OCR server 200 starts when the image of the payment receipt is updated from the user terminal 100.
First, on the user terminal 100, the user starts an application that cooperates with an automatic input service for sales proceeds, etc. (step 1).

When the application is started, an image prompting the camera 102 (see FIG. 3) to take an image of the payment receipt is displayed on the display unit 103 (see FIG. 3) of the user terminal 100, and a live image output from the camera 102 is displayed. Is displayed.
FIG. 6 is a diagram showing an example of an image displayed on the display unit 103 of the user terminal 100. The display unit 103 displays an image 110 of the payment receipt captured by the camera 102 (see FIG. 3) and a shutter button 120. In the case of FIG. 6, the upper end of the image 110 of the payment receipt is omitted.

When the user taps the shutter button 120, the data of the image displayed on the display unit 103 (image data 105A) is stored in the storage unit 105 (see FIG. 3). Note that the number of images 110 of the payment receipt is not limited to one, but it is also possible to capture a plurality of images.
Returning to the explanation of FIG. 5.
When the user operating the user terminal 100 selects the image data 105A to be uploaded, the user terminal 100 uploads the image data 105A of the payment receipt (step 2). Note that if the image data 105A stored by operating the shutter button 120 (see FIG. 6) is set to be automatically uploaded, the user does not need to select the image data to be uploaded.

The OCR server 200 stores the uploaded image data 105A in the storage unit 202 (see FIG. 4) as image data 202A (see FIG. 4).
When the image data 202A is stored in the storage unit 202, the OCR server 200 cuts out character image data included in the image data 202A (step 3). Therefore, background portions and margin portions that do not include characters are excluded from the extraction target.
Next, the OCR server 200 performs OCR processing on only a specific portion of the character image data corresponding to each line to search for target characters (step 4). In this embodiment, the target characters are the first two characters of the registered word.

FIG. 7 is a diagram illustrating a portion of character image data used for searching for a target character. In the case of this embodiment, the first two characters of the character image data are cut out. As shown in FIG. 7, the characters are cut out line by line from the bottom of the image data 202A of the payment receipt (see FIG. 4) upwards. In FIG. 7, the direction in which this cutting process is executed is shown as the scan direction.
For example, "Cash balance \37,462" is printed on the bottom line of the settlement receipt, but "Cash" located at the beginning of the line is cut out as the specific part. Incidentally, "cash balance" is the item part, and "\37,462" is the numerical part.
“Cash” is the first two characters of the registered word “cash balance” and is an example of target characters. Therefore, the bottom line is determined to be a target for OCR processing of the entire line of characters.

Furthermore, in the second row from the bottom, "Number of transactions: 4" is printed, but "Transaction" located at the beginning of the line is cut out as the specific part. Incidentally, "Number of transactions" is the item part, and "Number 4" is the numerical part. "Transaction" is neither a registered word nor a part of it. For this reason, the OCR server 200 does not set the entire line of characters in the second line from the bottom as a target for OCR processing, and moves on to determination of the next line.

FIG. 8 is a diagram illustrating a specific example of the search results for target characters. FIG. 8 shows a case where the search target characters are "cash", "**", and "net sale".
In the case of FIG. 8, four areas surrounded by broken lines are set as targets for OCR processing of the entire line of characters. Note that the third area from the top includes two lines: a line in which "Cash 4 Quantity" is printed as a parent line, and a line in which "\16,520" which has a parent line is printed.

Returning to the explanation of FIG. 5.
Next, the OCR server 200 performs OCR processing on the entire line in these four areas (step 5).
FIG. 9 is a diagram illustrating an example of information linked to information recognized by OCR processing.
The character image data corresponding to the first area from the top corresponds to the 19th line of the payment receipt. The raw OCR processing result of the item part is "** Net Sales", and the value of the numerical part is "\15,297".
The character image data corresponding to the second area from the top corresponds to the 21st line of the payment receipt. The raw OCR processing result of the item part is "**Total Total", and the value of the numerical part is "\16,520".

The character image data corresponding to the third area from the top corresponds to the 27th and 28th lines of the payment receipt.
There are no characters in the item section on the 28th line, but there are target characters in the item section on the 27th line one line above. Therefore, the 27th line is the parent line of the 28th line. The raw OCR processing result on the 28th line is also linked to the 27th line.
In the case of FIG. 9, the raw OCR processing result on the 27th line of the payment receipt is "cash" and the value of the numerical part is "4 numbers".
On the other hand, "cash", which is the result of the raw OCR processing on the 27th line, is linked to the 28th line of the payment receipt as a result of the raw OCR processing. In addition, the value of the numerical part in the same is "\16,520".
The character image data corresponding to the fourth area from the top corresponds to the 31st line of the payment receipt. The raw OCR processing result of the item part is "cash balance", and the value of the numerical part is "\37,462".

Returning to the explanation of FIG. 5.
Next, the OCR server 200 executes matching processing (step 6). In the matching process, a process of associating characters resulting from the raw OCR process with a registered word, and a process of associating the characters with a word of a group to which the registered word is associated are executed.
FIG. 10 is a diagram illustrating an example of matching. In FIG. 10, only the third and fourth areas from the top of the payment receipt are shown due to space constraints.
The characters "cash" recognized by OCR processing the item section of the 27th line of the payment receipt match the registered word "cash". Therefore, the reliability of the recognized word "cash" is 1.0. The reliability represents the probability of the raw OCR processing result. In the case of this embodiment, 1.0 is the maximum value. Note that the registered word "cash" corresponds to the group "cash sales."

The characters "cash" recognized by OCR processing the item section of the 28th line of the payment receipt also correspond to the registered word "cash". The reliability of this word is also 1.0. The registered word "cash" corresponds to the group "cash sales."
The characters "cash balance" read from the item section on the 31st line of the payment receipt correspond to the registered word "cash." The reliability is 0.5. Note that the registered word "cash" corresponds to the group "cash sales."

Returning to the explanation of FIG. 5.
Next, the OCR server 200 selects a value to be associated with one item (step 7). In the case of this embodiment, one is selected from the values on the 27th line and the value on the 28th line that are linked to "cash" in the item section.
FIG. 11 is a flowchart illustrating the contents of the process used to select values.
First, the OCR server 200 determines whether the processing target has multiple values in one item (step 11).

If a positive result is obtained in step 11, the OCR server 200 selects the value that is considered to be the amount (step 12). Specifically, select a number that can be converted to the Current type.
On the other hand, if a negative result is obtained in step 11, the OCR server 200 uses the corresponding value as is (step 13).
FIG. 12 is a diagram illustrating the selection of values in step 7.
The value "4 number" on the 27th line of the payment receipt cannot be converted to the Current type. On the other hand, the value "\16,520" on the 28th line is a value that can be converted to the Current type. Therefore, in this example, the value "\16,520" on the 28th line is selected as the value associated with the group "cash sales."

Returning to the explanation of FIG. 5.
When the selection of values for all four areas is completed, the OCR server 200 selects values for each group (step 8).
In the case of this embodiment, the value "\15,297" on the 19th line and the value "\16,520" on the 21st line of the payment receipt are both associated with the group word "net sales." Further, the value "\16,520" on the 28th line of the settlement receipt and the value "\37,462" on the 31st line are both associated with the word "cash sales" in the group. Therefore, it is necessary to select one value to be associated with each group of words.

FIG. 13 is a flowchart illustrating the process of selecting values for each group.
First, the OCR server 200 determines whether there are multiple values associated with the same group (step 21).
If a positive result is obtained in step 21, the OCR server 200 determines whether the reliability levels are the same (step 22).
If a positive result is obtained in step 22, the OCR server 200 selects the larger value (step 23). On the other hand, if a negative result is obtained in step 22, the OCR server 200 selects the value with higher reliability.
Note that if a negative result is obtained in step 21, the OCR server 200 extracts the value of the corresponding row as is. In this case, this is because there is only one value associated with one group.

FIG. 14 is a diagram illustrating an example of selecting one of two values associated with the group "cash sales."
In the case of the group "Cash Sales," the value "\16,520" on line 28 of the settlement receipt is linked to the value "\37,462" on line 31. However, the reliability of the raw OCR processing result on line 28 is 1.0, while the reliability of the raw OCR processing result on line 31 is 0.5. Therefore, the value "\16,520" on line 28, which has a relatively high reliability, is selected as the value for the group "Cash Sales."

FIG. 15 is a diagram illustrating an example of selecting one from two values linked to the group "net sales."
In the case of the group "net sales", the value "\15,297" on the 19th line of the settlement receipt and the value "\16,520" on the 21st line are linked. However, while the reliability of the raw OCR processing result on the 21st line is 0.9, the reliability of the raw OCR processing result on the 19th line is 0.4. Therefore, the value "\16,520" in the 21st line, which has a relatively high reliability, is selected as the value of the group "net sales".

Returning to the explanation of FIG. 5.
When values are selected for each group, the OCR server 200 transmits the selection results to the user terminal 100 (step 9). Specifically, "\16,520" is transmitted as the value of "cash sales" and "\16,520" is transmitted as the value of net sales.
The user terminal 100 that has received this information displays a screen in which the results are combined on the display unit 103 (see FIG. 3) (step 10).
FIG. 16 is a diagram illustrating a screen example of automatic input results such as sales proceeds displayed on the display unit 103 of the user terminal 100. The screen shown in FIG. 16 displays information on the date of the imaged payment receipt, weather information, cash sales information, non-cash sales information, net sales information, a cancel button 121, and a registration button 122. ing.

Here, the non-cash sales column displays the value obtained by subtracting cash sales from net sales. Non-cash sales include, for example, amounts paid by credit card and amounts paid by electronic money. Note that, as the date, the year, month, and day recognized from the payment receipt through OCR processing may be automatically displayed, or the year, month, and day specified by the user from the calendar function may be displayed. Furthermore, information input by the user is displayed in the weather column.
Note that if the value automatically input by OCR processing is incorrect, the numerical value can also be corrected by manual input.
When the user taps the registration button 122, the displayed amount is confirmed.

<Embodiment 2>
In the case of the first embodiment described above, as explained using FIG. 7, the first two characters of character image data are searched to see if the target character is included in the result of OCR processing, thereby identifying the target character for OCR processing. As a result, the time required for OCR processing is reduced.
However, the time required for OCR processing can also be reduced by other techniques.

FIG. 17 is a diagram illustrating the range in which the OCR server 200 executes OCR processing in the second embodiment. In this embodiment, the range in which OCR processing is executed is limited to a range in which there is a high possibility that information on net sales or cash sales will appear. In the example of FIG. 17, the range is limited to two-thirds from the bottom of the payment receipt. If the range of OCR processing becomes two-thirds of the entire payment receipt, the number of characters subject to OCR processing will also be approximately two-thirds, making it possible to shorten the time required for OCR processing. Note that two-thirds is an example, and it may be one-third, one-fourth, or the like. As long as the required information can be recognized, the narrower the range of OCR processing, the better.

In the case of Fig. 17 as well, the character image data corresponding to each line of the payment receipt is processed by OCR in order from the bottom to the top. Also, in the case of the second embodiment, the process from step 5 onwards in Fig. 5 is the same.
However, in the case of the second embodiment, it is also possible to combine a function of reading only the first two characters of each line and determining whether or not to perform OCR processing on the characters of the entire line. In this case, the number of characters to be processed by OCR is smaller than that in the first embodiment, and the OCR processing time can be further reduced.

<Other embodiments>
Although the embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the range described in the embodiments described above. It is clear from the claims that various changes or improvements made to the embodiments described above are also included within the technical scope of the present invention.

(1) For example, in the case of the above-mentioned embodiment 1, the target character is searched for in the character image data corresponding to each line from the bottom end of the payment receipt toward the top end, but the target character may also be searched for from the top end of the payment receipt toward the bottom end.
(2) In the above-described embodiments, the target of scanning is a billing receipt. However, the present invention can also be used to automatically input specific information on the receipt received by the consumer using OCR processing.

(3) In the above-described embodiment, a case has been described in which the relationship between registered words that may be printed on the payment receipt and words in the associated group is prepared as the registered word table 202B. Deep learning may be used to learn words related to printed sales and the correspondence between each word and group. By using a dictionary generated by deep learning, even when an unknown registered word that should be linked to a word registered as a group is recognized, it is detected as a target character and the entire line containing the word is recognized. It can be set as a target for OCR processing. Moreover, this unknown word can also be linked to words registered as a group.

(4) In the case of the above embodiment, the OCR server 200 (see FIG. 1) executes the function of recognizing numerical values such as sales proceeds from the image data of the settlement receipt. A part of the process may be executed by the user terminal 100 (see FIG. 1). The user terminal 100 in this case is an example of an information processing device.
(5) In the above embodiment, one OCR server 200 executes all the functions shown in FIG. 4, but some of these functions are performed by other servers connected to the cloud network 300 or It may also be executed in cooperation with the terminal.

100... User terminal, 101... Control unit, 102... Camera, 103... Display unit, 104... Operation input unit, 105... Storage unit, 105A... Image data, 106... Communication unit, 110... Image of payment receipt, 120... Shutter Button, 121...Cancel button, 122...Register button, 200...OCR server, 201...Control unit, 201A...Image data acquisition unit, 201B...Character extraction processing unit, 201C...OCR processing unit, 201D...Matching unit, 201E... Value selection section, 201F...Group result selection section, 202...Storage section, 202A...Image data, 202B...Registered word table, 203...Communication section, 300...Cloud network

Claims (5)

a control unit;
an acquisition unit that acquires image data of the payment receipt captured by the imaging unit from the imaging unit;
a cutout processing unit that sequentially cuts out partial image data of a fixed number of characters one line at a time from the beginning of each line containing characters in the image data;
a character recognition unit that recognizes character information from the cut out partial image data;
The control unit includes:
If there is a registered word corresponding to the character information output from the character recognition unit, determining the entire corresponding line as a target for character recognition ,
If there is no registered word corresponding to the character information output from the character recognition unit, excluding the entire corresponding line from character recognition targets ;
Information processing device. The information processing device according to claim 1, wherein the control unit determines amount information corresponding to the character information output from the character recognition unit. 3. The method according to claim 1, wherein for one piece of the partial image data, a line where the character recognition section finishes recognizing the character information is located above the line where the character recognition unit starts recognizing the character information. Information processing device. A control unit;
an imaging unit that captures an image of a receipt and outputs image data;
an acquisition unit that acquires the image data from the imaging unit;
a cutout processing unit that cuts out partial image data of a fixed number of characters from the beginning of each line of the image data , one line at a time, in order ;
a character recognition unit that recognizes character information from the cut-out partial image data,
the control unit, when a registered word corresponding to the character information output from the character recognition unit is present, determines an entire corresponding line as a target for character recognition ;
If there is no registered word corresponding to the character information output from the character recognition unit, the entire corresponding line is excluded from the target of character recognition .
Information processing system. Processing to obtain image data of payment receipt,
a process of sequentially cutting out partial image data of a fixed number of characters one line at a time from the beginning of each line containing characters from the image data;
a process of recognizing character information from the cut out partial image data;
If there is a registered word corresponding to the recognized character information, a process of determining the entire corresponding line as a target for character recognition ;
If there is no registered word corresponding to the character information output from the character recognition unit, excluding the entire corresponding line from character recognition;
An information processing method having the following.

Images