WO2023171456A1

WO2023171456A1 - Determination system, terminal device, determination method, and program

Info

Publication number: WO2023171456A1
Application number: PCT/JP2023/007169
Authority: WO
Inventors: 将司山本; 裕樹小林
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2022-03-11
Filing date: 2023-02-27
Publication date: 2023-09-14
Also published as: JP2023132727A

Abstract

This determination system has: an image acquisition unit for acquiring a voucher image in which a voucher is imaged; a detection unit for detecting, from the voucher image, one or more regions indicating entry columns in the voucher; a character recognition unit for performing character recognition on each of the one or more regions and specifying entered character information; an error determination unit for performing error determination on the basis of the character information specified by the character recognition unit and an entry rule defined in advance for each of the one or more regions; and a notification unit for providing notification of a region determined as an error by the determination by the error determination unit.

Description

Judgment system, terminal device, judgment method, and program

The present disclosure relates to a determination system, a terminal device, a determination method, and a program.

Patent Document 1 discloses a system in which information such as a postal code, address, name, or telephone number of a slip is registered in a correct database, and the read content is corrected using the information in the correct database. Further, Patent Document 2 discloses a configuration in which an address character string is corrected by comparing the result of reading an address on a slip with correct address data stored in advance.

Japanese Patent Application Publication No. 2020-009323 Japanese Patent Application Publication No. 3-257693

The present disclosure was devised in view of the above-mentioned conventional circumstances, and provides a determination system, a terminal device, a determination method, which recognizes character strings entered on a slip, and notifies a worker in the case of nonconformity and prompts the operator to make corrections. and programs.

The present disclosure is a determination system that determines the contents filled in a slip in the delivery of a package, and includes an image acquisition unit that obtains a slip image in which the slip is photographed, and a unit that indicates entry fields in the slip from the slip image. or a detection unit that detects a plurality of areas; a character recognition unit that performs character recognition on each of the one or more areas to identify written character information; and character information specified by the character recognition unit; an error determination unit that determines an error based on predefined entry rules for the one or more areas; and a notification unit that notifies the area determined to be an error based on the determination by the error determination unit. The present invention provides a determination system that has the following features.

The present disclosure also includes a photographing unit that photographs a voucher image of a voucher in the delivery of a package, a requesting unit that requests a server to make an error determination for the voucher image, and one or more areas included in the voucher image. a receiving unit that receives from the server the result of the error determination of the character information entered in the text information, a receiving unit that notifies the area determined to be an error by the error determination and receives correction information for the area, and the receiving unit and a transmitter that transmits correction information received at the server to the server.

Further, the present disclosure provides that the processor cooperates with the memory to acquire a slip image in which a slip for delivery of a package is photographed, and detects one or more areas indicating entry fields in the slip from the slip image; Performing character recognition on each of the one or more areas, identifying written character information, and based on the identified character information and writing rules predefined for the one or more areas, A determination method is provided in which an error determination is performed and an area determined as an error is notified by the error determination.

The present disclosure also provides a process for a computer to acquire a slip image in which a slip for delivery of a package is photographed, a process for detecting from the slip image one or more areas indicating entry fields in the slip; or a process of performing character recognition on each of a plurality of areas and specifying written character information, and based on the specified character information and writing rules predefined for the one or more areas, A program is provided for executing a process of determining an error and a process of notifying an area determined to be an error.

Note that any combination of the above components and the expressions of the present disclosure converted between methods, devices, systems, storage media, computer programs, etc. are also effective as aspects of the present disclosure.

According to the present disclosure, it is possible to improve the convenience of the user in processing slips.

Block diagram illustrating an example configuration of a system according to Embodiment 1 Block diagram showing an example of functional configuration of each device according to Embodiment 1 External perspective view for explaining the terminal device according to Embodiment 1 Conceptual diagram showing a schematic structure of a slip according to Embodiment 1 Conceptual diagram showing a configuration example of a prohibition rule DB according to Embodiment 1 Sequence diagram of processing according to Embodiment 1 Schematic diagram showing a configuration example of a UI screen according to Embodiment 1 Schematic diagram showing a configuration example of a UI screen according to Embodiment 1 Schematic diagram showing a configuration example of a UI screen according to Embodiment 1

(Circumstances leading to this disclosure)
Recognizes and corrects character strings entered in each field (hereinafter also referred to as ``items'') in delivery slips (hereinafter simply referred to as ``slips'') traditionally used to deliver packages, etc. Many techniques are known to do this. On the other hand, there are various items in a slip, and special entry rules may be prescribed depending on the item. In this case, even if the entered characters themselves are correct, they may not satisfy the entry rules prescribed for that item. Users such as delivery personnel who check the invoice check each item and decide whether the contents are correct or not, but they have to make judgments according to different entry rules for each item, which is time-consuming. was.

Hereinafter, embodiments specifically disclosing a determination system, a terminal device, a determination method, and a program according to the present disclosure will be described in detail with reference to the accompanying drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of well-known matters or redundant explanations of substantially the same configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims.

<Embodiment 1>
[System configuration]
FIG. 1 is a block diagram showing a configuration example of a determination system 1 according to the present embodiment. The determination system 1 according to this embodiment is configured to include a terminal device 100 and a server 200. The terminal device 100 and the server 200 are communicably connected via a network 300. In FIG. 1, one terminal device 100 and one server 200 are shown to simplify the explanation, but the present invention is not limited to this. A plurality of terminal devices 100 may be configured to be able to communicate with the server 200. Furthermore, the server 200 may be composed of a plurality of information processing devices from the viewpoint of load distribution, function distribution, and the like.

The terminal device 100 is a device used by a user such as a delivery person, and may be, for example, a POS (Point of Sales) terminal, a smartphone, a mobile terminal such as a tablet terminal, or the like. The terminal device 100 includes a control section 101, a memory 102, a camera 103, an input section 104, a display section 105, and a communication section 106. The control unit 101 controls the entire processing of the terminal device 100, and includes, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), and a GP. U (Graphical Processing Unit) or FPGA (Field Programmable Gate) Array).

The memory 102 is a storage area for storing and holding various data and programs, and includes, for example, ROM (Read Only Memory), which is a nonvolatile storage area, and RAM (Random Access Memory), which is a volatile storage area. Consists of etc. The camera 103 is a photographing unit for photographing the surroundings of the terminal device 100.

The input unit 104 accepts operations from the user of the terminal device 100. The display unit 105 displays a screen presenting various information. Note that a touch panel display in which the input section 104 and the display section 105 are integrated may be used. The communication unit 106 communicates with an external device via the network 300. The communication method by the communication unit 106 is not particularly limited, and may be compatible with a plurality of communication methods. For example, WAN (Wide Area Network), LAN (Local Area Network), power line communication, short-range wireless communication (eg, Bluetooth (registered trademark)), etc. may be used.

The server 200 is an information processing device that executes the processing according to this embodiment in response to a request from the terminal device 100. The server 200 includes a control section 201, a storage section 202, a UI (User Interface) section 203, and a communication section 204. The control unit 201 controls the entire processing of the server 200, and may be configured using, for example, a CPU, an MPU, a DSP, a GPU, or an FPGA.

The storage unit 202 is a storage area for storing and holding various data and programs, and includes, for example, a ROM or HDD (Hard Disk Drive) that is a nonvolatile storage area, and a RAM that is a volatile storage area. be done. The UI unit 203 accepts operations from the user of the server 200 and outputs various information. The communication unit 204 communicates with an external device via the network 300. The communication method used by the communication unit 204 is not particularly limited, and may be compatible with a plurality of communication methods. For example, WAN, LAN, power line communication, short-range wireless communication, etc. may be used.

[Functional configuration]
FIG. 2 is a block diagram showing an example of the functional configuration of the terminal device 100 and the server 200 according to the present embodiment. Note that the configuration shown in FIG. 2 is an example, and each part may be divided into more detailed parts, or a plurality of parts may be grouped together. Each part shown in FIG. 2 is realized by each device shown in FIG. 1 working together.

FIG. 2(a) shows an example of the functional configuration of the terminal device 100 according to the present embodiment. The image acquisition unit 151 acquires an image of a slip (hereinafter also referred to as a "slip image") captured by the camera 103 according to a user's operation. The image transmitter 152 transmits the slip image acquired by the image acquirer 151 to the server 200. The determination result acquisition unit 153 acquires the determination result for the slip image transmitted by the image transmission unit 152. Details of the determination process according to this embodiment will be described later. The determination result display unit 154 displays a screen showing the determination result obtained by the determination result acquisition unit 153 via the display unit 105. If the determination result acquired by the determination result acquisition unit 153 indicates an error, the character information correction unit 155 receives an input of correction information for resolving the error, and transmits it to the server 200 . The correction information may be information other than character strings, such as a correct character string that conforms to the entry rules of the slip, but an instruction to ignore portions that violate the entry rules.

FIG. 2(b) shows an example of the functional configuration of the server 200 according to the present embodiment. The image receiving unit 251 receives a slip image from the terminal device 100. The image correction unit 252 performs correction processing on the slip image received by the image reception unit 251. Although the correction processing here is not particularly limited, for example, trapezoidal correction, color correction, noise removal, etc. may be performed. Note that the correction process may be omitted if the slip image is in a sufficient state for subsequent processing. The item area detection unit 253 detects a plurality of item areas from the slip image based on predefined format information of the slip. It is assumed that the format information of the slip is information that defines the position, size, number, etc. of items to be filled in on each slip, and is defined in advance and stored in the storage unit 202 of the server 200 or the like. Therefore, the item area is an area where some information (more specifically, a character string) is entered by handwriting or the like by the user of the slip. Note that depending on the item, a single character may be input, or a character string composed of multiple characters may be input, but this is not particularly limited.

The character recognition unit 254 recognizes written characters in each of the plurality of item areas detected by the item area detection unit 253. Character recognition here is not particularly limited, and may use, for example, a well-known OCR (Optical Character Recognition) function or a learned model generated by machine learning. The error determination unit 255 performs error determination based on the character string recognized by the character recognition unit 254 and the prohibition rule set for the item area in which the character string has been entered. The prohibition rule corresponds to the entry rule set for each item of the actual slip. It is assumed that the prohibition rules according to this embodiment are defined in advance and stored in the storage unit 202 of the server 200 or the like. The prohibition rules may be managed in association with the format information described above. A specific example of the prohibition rule according to this embodiment will be described later using FIG. 5.

The determination result notification unit 256 transmits the determination result by the error determination unit 255 to the terminal device 100 as a response to the received slip image. The determination result here may include information on the characters recognized in each item area, in addition to information regarding the presence or absence of an error. The determination result aggregation unit 257 totals the determination results by the error determination unit 255. In this calculation, correction information received from the terminal device 100 in response to the determination result notified by the determination result notification unit 256 may be taken into consideration. The character information output unit 258 receives correction information from the terminal device 100 in response to the content notified by the determination result notification unit 256, and registers it as corrected information in association with the item area. Further, the character information output unit 258 outputs the recognized or corrected and registered character information. The output destination of the character information output unit 258 may be a slip management server (not shown) that manages the information written on the slip, a delivery management server (not shown) that manages the delivery status of the slip, or the like. In this case, each server may treat the received character information as correct information written on the slip, regardless of whether or not it has been corrected.

FIG. 3 is an external perspective view showing an example in which the terminal device 100 according to the present embodiment photographs a slip 400 and obtains an image of the slip. The user of the terminal device 100 photographs the slip 400 so that the entire slip 400 is included in the slip image. Note that the shooting settings of the slip 400 are not particularly limited, and may be predefined or may be adjustable by the user of the terminal device 100. Further, the terminal device 100 may be configured to present a UI screen (not shown) prompting the user to take a photograph of the slip 400.

[Voucher structure]
FIG. 4 is a diagram showing a configuration example of a slip 400 according to the present embodiment. The structure of the slip shown here is an example, and the structure is not limited to this. Furthermore, information regarding the structure of the slip 400 is defined as format information. Further, format information corresponding to each of a plurality of types of slips may be defined. Furthermore, prohibition rules corresponding to each of the plurality of item areas included in the slip 400 are defined. Here, an example of a slip used for delivery by cash on delivery is shown.

Item 401 is an entry field for entering the postal code of the delivery destination of the package. Item 402 is an entry field for entering the contact information (more specifically, the telephone number) of the delivery destination of the package. Item 403 is an entry column for entering the address and name of the delivery destination of the package.

Item 404 is an entry field for entering the postal code of the shipping source of the package. Item 405 is an entry column for entering the address and name of the destination of the package.

Management information 406 is inquiry information used when inquiring about the status of the actual delivery.

Item 407 is an entry column for entering the desired delivery date of the package. Item 408 is an entry column for entering the contents (items) of the package. Item 409 is an entry column for entering the cash on delivery amount. The prohibition information 410 is information indicating precautions and prohibitions when filling in a predetermined item.

In the example of FIG. 4, although the prohibition information 410 clearly states that the "Cash on delivery amount" corresponding to item 409 cannot be corrected ("Cash on delivery amount cannot be corrected"), A double line (=) indicates an example where a correction has been made. In other words, the example in FIG. 4 shows an entry state that violates the entry rules.

FIG. 5 is a diagram showing an example of prohibition rules according to the present embodiment. Prohibition rules serving as entry rules are set corresponding to each item (entry field) included in the slip 400. Note that the item names are just examples and are not limited to these. Note that a plurality of prohibition rules may be set for one item, or no prohibition rule may be set.

The prohibition rule "strikethrough, fill-in" is set for the item "cash on delivery". This is conditional on the condition that "strike-through" and "fill-in" entries are prohibited for the item "cash on delivery." As a result of character recognition, if "strikethrough" or "filling" is detected in the item area of the item "cash on delivery", an error occurs. Note that "strikethrough" and "filling" can be detected by treating and recognizing "strikethrough" and "filling" themselves as character information, or if the characters cannot be recognized, "strikethrough" and "filling" can be detected. It may be assumed that there is "filling". Further, corrections made by means other than "strike-through" and "fill-in" may be handled in the same way as "strike-through, fill-in".

The prohibition rule "Check two or more options" is set for the item "Delivery time zone". This is when the item "Delivery time zone" is configured to select one from multiple options (for example, "No specification", "Morning", "14:00 to 16:00", etc.). The condition is that the above selections are prohibited. As a result of character recognition, if two or more selections are detected from the item area of the item "Delivery time zone", an error occurs. Note that the check may be detected by treating and recognizing a symbol indicating "check" as character information.

The prohibition rule "Less than 7 digits" is set for the item "Postal code". This is based on the condition that if the item "zip code" has a frame so that seven digits can be entered, entry of character strings of less than seven digits is prohibited. As a result of character recognition, if a character string of less than 7 digits is detected from the item area of the item "zip code", an error occurs.

The prohibition rule "Characters other than numbers and hyphens" is set for the items "Postal code" and "Telephone number". This is on the condition that entries other than numbers and hyphens (-) are prohibited in the items "postal code" and "telephone number." As a result of character recognition, an error occurs if characters other than numbers and hyphens are detected in the item area of the items ``zip code'' and ``telephone number.''

The prohibition rule "Characters other than numbers" is set for the item "Delivery date". This is on the condition that the entry of characters other than numbers in the item "desired delivery date" is prohibited. As a result of character recognition, an error occurs if characters other than numbers are detected in the item area of the item "Delivery date".

The prohibition rule "Date and time before the current date" is set for the item "Delivery date". This is on the condition that entry of a date before the check date is prohibited in the item "desired delivery date". As a result of character recognition, if the date detected from the item area of the item "desired delivery date" is a date before the check date, an error will occur. Here, the current date set in the terminal device 100 may be used as the current date, or the current date may be obtained from the server 200. Furthermore, although days are explained here as an example, checking may be performed in even more detailed units such as hours and minutes. Alternatively, the check may be performed in units of time slots such as morning and afternoon. The unit to be checked may differ depending on the type of slip.

For the item "Delivery date", the prohibition rule "If the sender's or recipient's address is on a remote island, the current date" is set. This is based on the condition that in the item ``Desired delivery date'', if the item ``Delivery destination'' is a remote island, entry of a date before the check date is prohibited. In other words, conditions are set assuming that delivery is not realistically possible due to distance or time constraints. As a result of character recognition, if the item "Delivery destination" is a remote island (or a certain distance or more), and the date detected from the item area of the item "Delivery date" is before the check date. An error occurs if the date is a date.

For the item "Size", the prohibition rule "Size too small/too large compared to the size inferred from the product name" is set. This assumes that there is a correlation between the item "Size" and the item "Product Name", and if a size range corresponding to the product name is defined in advance, the size for the product name will be prohibited from exceeding the range. It is a condition. As a result of character recognition, an error occurs if the size detected from the item area of the item "Size" is larger or smaller than the size range set corresponding to the character string detected from the item area of the item "Product name". becomes. Note that if the correlation between "size" and "product name" cannot be estimated, such as when an unknown product name is recognized, it may not be handled as an error.

As mentioned above, conditions that combine multiple items may be used. In this case, the configuration may be such that one of the plurality of items is referred to first, and the next item is determined based on the reference result. Moreover, when using a plurality of items in combination, a configuration may be adopted in which priorities are set for the items and the items are determined in order of priority. Moreover, calendar information, map information, etc. may be used to determine the above-mentioned prohibition rule.

Furthermore, if errors occur for multiple items at the same time, all errors may be displayed on the UI screen, or the errors to be displayed may be limited according to priority. Here, the priority may be, for example, the same as the priority of the determination described above. Furthermore, when multiple errors occur in the same column, the priority of errors that are less susceptible to misrecognition of character information may be given higher than other errors. This is because if the character information is misrecognized, there may be no need to correct the slip. Specifically, if an error that the number of digits in the postal code is insufficient and an error that the character type of the postal code is incorrect occur at the same time, the error related to the number of digits is displayed preferentially. This means that if the number of digits is insufficient, it is necessary to correct the slip regardless of whether or not the characters are misrecognized. This is because there is no need to modify the slip itself. Also, if an error occurs in one column and another column at the same time, the priority of the error occurring in the farthest column in the slip layout is higher than the priority of the error occurring in the closer column. It's okay. If an error occurs in a nearby field, there is a high possibility that that field will also be visible at the same time, so there is a high possibility that you will notice it even if you do not notify it again, whereas an error that occurs in a far field will not be notified. This is because there is a high possibility that you will not notice it.

[Processing sequence]
FIG. 6 shows a processing sequence of determination processing according to this embodiment. The processing in each step of this processing sequence is realized by the control section of each device reading and executing various programs stored in the storage device. Here, in order to simplify the explanation, the processing subject is shown as a device. Note that the content and timing of data transmitted and received between the terminal device 100 and the server 200 are not limited to those shown in FIG. 6, and data can be further transmitted and received between the devices according to requests. good.

Based on the user's operation, the terminal device 100 performs an operation of photographing the slip 400 using the camera 103 (step S601). The photographing of the slip 400 may be performed after operating a predetermined application. Furthermore, the configuration may be such that the user is prompted to adjust the shooting range and shooting settings so that an appropriate slip image can be obtained for character recognition.

The terminal device 100 transmits the slip image photographed in the photographing operation of step S601 to the server 200 (step S602). At this time, various information on the terminal device 100 may be transmitted to the server 200 together with the photographed image.

The server 200 receives the slip image transmitted from the terminal device 100 in step S602 (step S603).

The server 200 performs a correction process on the slip image received in step S603 (step S604). The content of the correction process here is not particularly limited, and may be omitted depending on the quality of the received slip image.

The server 200 detects a plurality of item areas corresponding to the entry fields of the slip from the slip image (step S605). The server 200 specifies the type of slip that is photographed as a slip image, and further detects an item area based on format information predefined for the specified type of slip. The size, arrangement, number, etc. of item areas are defined depending on the type of slip. At this time, if the type of slip or item area cannot be detected from the slip image, the terminal device 100 may be notified and instructed to transmit the slip image again.

The server 200 performs character recognition processing on each of the plurality of item areas detected in step S605 (step S606). The character recognition method is not particularly limited, and a trained model generated by known OCR technology or machine learning may be used. The server 200 associates and stores item areas and character strings recognized through character recognition.

The server 200 uses the result of the character recognition in step S606 and the prohibition rule as shown in FIG. 5 to determine an error in the entry content in each item area (step S607). The error determination here may be performed sequentially starting from a predetermined item area, or may be performed according to the order in which the prohibition rules are applied.

In the error determination process in step S607, the server 200 determines whether there is at least one item area that corresponds to the prohibition rule (step S608). If there is an item area that corresponds to the prohibition rule (step S608; YES), the process of the server 200 proceeds to step 609. On the other hand, if there is no item area that corresponds to the prohibition rule (step S608; NO), the process of the server 200 proceeds to step S615.

Based on the fact that an error occurred in the error determination process in step S607, the server 200 notifies the terminal device 100 of the error contents for each item (step S609). If a plurality of errors occur due to a prohibition rule, the contents of the plurality of errors may be notified at once. Additionally, the server 200 also notifies the character information recognized in the item area corresponding to the error.

The terminal device 100 receives the error details notified from the server 200 in step S609 (step S610).

The terminal device 100 displays a UI screen according to the error content received in step S610 (step S611). A configuration example of the UI screen will be described later using FIG. 7.

The terminal device 100 presents the character information recognized in the item area corresponding to the error via the UI screen, and accepts correction information for the character information (step S612). A configuration example of the UI screen will be described later using FIG. 8.

The terminal device 100 transmits the correction information received in step S612 to the server 200 (step S613).

The server 200 receives the correction information transmitted from the terminal device 100 in step S613, and stores it as character information in the corresponding item area (step S614). Then, the server 200 returns to step S607, executes the error determination process again based on the correction information, and repeats the subsequent process.

The server 200 transfers the character information corresponding to each of the plurality of item areas to a predetermined transfer destination (step S615). The transfer destination here may be, for example, a slip management server (not shown) that manages the information written on the slip, a delivery management server (not shown) that manages the delivery status of the slip, or the like. Note that if the server 200 is configured to be integrated with these servers, this step may be omitted. Further, the server 200 may be configured to notify the terminal device 100 that the character recognition of the slip has been successfully completed together with the recognition result. Then, this processing sequence ends.

[Screen configuration example]
A configuration example of a UI screen displayed on the terminal device 100 according to the present embodiment will be explained using FIGS. 7 and 8. Note that each UI screen shown below may be configured as a web screen displayed on a web browser (not shown) included in the terminal device 100, or may be configured as a web screen that is installed on the terminal device 100 and executed by the control unit 101. It may be configured as a screen of a functioning application.

FIG. 7 is an example of the configuration of a UI screen 700 for notifying error details, for example, in the process of step S611 in FIG. 6. On the UI screen 700, as a result of character recognition performed on a plurality of items included in the slip 400, information 701 of the corresponding prohibition rule is displayed. Information 702 regarding more detailed contents of the prohibition rule is also shown. When the OK button 703 is pressed, the UI screen 700 is closed. In this case, you may transition to a UI screen 800 shown in FIG.

FIG. 8 is a configuration example of a UI screen 800 for receiving correction information, for example, in the process of step S612 in FIG. 6. FIG. 8(a) shows a display example when character recognition has failed, and FIG. 8(b) shows a display example when character recognition has succeeded. In this example, the item "zip code" will be explained as an example.

On the UI screen 800, an item area 801 detected from the slip image is displayed. In this example, "9650072" is entered. Furthermore, character information 802 obtained by performing character recognition on the item area 801 is displayed in an area that can be modified. In this example, "965A072" is recognized as the character information 802. Error determination processing is performed based on the items in item area 801 and the content of character information 802, which is the result of character recognition. In this example, since the character "A", which is not normally used in postal codes, is included, it is determined as an error that it falls under the prohibition rule.

As mentioned above, there are cases where the entry is correct but character recognition fails. Specifically, in the example of FIG. 8A, "0" is incorrectly recognized as "A", which has a similar shape. In such a case, the user can send the corrected information to the server 200 by correcting the contents of the text information 802 and pressing the OK button 804. Note that since the entry itself in the item area 801 is correct, the check box 803 is not checked. Upon pressing the OK button 804, the UI screen 800 may be closed.

Similarly, on the UI screen 810, an item area 811 detected from the slip image is displayed. In this example, "965A072" is entered. Furthermore, character information 812 obtained by performing character recognition on the item area 811 is displayed in an area that can be modified. In this example, "965A072" is recognized as the character information 802. Error determination processing is performed based on the items in item area 811 and the content of character information 812, which is the result of character recognition. In this example, since the character "A", which is not normally used in postal codes, is included, it is determined as an error that it falls under the prohibition rule.

As mentioned above, there are cases where the entry itself is incorrect and the incorrectly entered characters are correctly recognized. In such a case, the user can send the corrected information to the server 200 by correcting the content of the text information 812, checking the check box 813, and pressing the OK button 814. Upon pressing the OK button 814, the UI screen 810 may be closed.

FIG. 9 shows a configuration example of a UI screen 900 as a result of aggregation processing based on error determination results according to the present embodiment. The execution timing of the aggregation process is not particularly limited, but it may be performed based on instructions from the administrator of the server 200, or may be performed by the user of the terminal device 100 inquiring the server 200. Alternatively, the configuration may be such that the aggregation process is performed at the timing of step S615 in FIG. Further, the UI screen 900 may be displayed on the display unit 105 of the terminal device 100 or may be displayed on the UI unit 203 of the server 200.

On the UI screen 900, the total results based on the items and prohibition rules are shown in a table format. Here, explanation will be given using one particular slip as an example. Therefore, the UI screen 900 may be configured to be able to switch the tally results depending on the type of slip, or may display the tally results of multiple types of slips all at once.

Item 901 indicates each entry column in the slip 400. Item 902 indicates the priority of the entry fields included in the slip 400. For example, depending on the priority value, it indicates which items are prioritized when there is a mismatch between the entries in the entry fields. Item 903 indicates the prohibition rule that was applicable at the time of error determination. Item 904 indicates a count value of the number of times an error has occurred.

As described above, according to the present embodiment, the determination system 1 includes the image receiving unit 251 that acquires a slip image in which the slip is photographed from the terminal device 100, and detects one or more item areas indicating entry fields in the slip from the slip image. a character recognition unit 254 that performs character recognition on each of one or more areas to identify written character information; and character information identified by the character recognition unit 254, and one or more an error determination unit 255 that performs an error determination based on a prohibition rule predefined for the item area; and a determination result notification that notifies the terminal device 100 of the area determined as an error by the error determination unit 255. 256. This makes it possible to improve the convenience of the user in processing slips. In particular, it is possible to save the effort of checking individual prohibition rules for multiple entry fields included in a slip each time.

Further, when the error determination unit 255 determines that there is an error, the determination result notification unit 256 can select whether the error is caused by a mistake in filling out the slip or a recognition error by the character recognition unit 254. Display and accept. This makes it possible to perform appropriate aggregation processing even if a misrecognition occurs on the system side.

Further, the determination result notifying unit 256 receives correction information for the area determined to be an error by the determination by the error determining unit 255. Further, the determination system 1 includes a character information output unit 258 that corrects the character information of a corresponding area among one or more areas based on the received correction information. As a result, it is possible to receive correction information indicating the content of correction for an item on a slip in which an error has occurred, and furthermore, based on the correction information, it is possible to appropriately utilize information related to the slip.

Further, prohibition rules are defined for each type of slip. This makes it possible to define entry prohibition rules depending on the type of slip.

Further, the prohibition rule is defined for each of one or more item areas included in the slip. This makes it possible to define entry prohibition rules corresponding to each item area included in the slip.

It further includes a determination result aggregation unit 257 that totals the determination results by the error determination unit 255. This makes it possible to identify items that are likely to cause errors among the entry fields of the slip, and to feed back the results of the tally. The fed-back tally results can be used as hints for correcting the form to a format that is less prone to errors, such as changing the explanation of entry rules for error-prone items to easier-to-understand content. Further, the information may be displayed on the terminal device 100 so that items that are likely to cause errors are checked in a focused manner. In this case, on the UI screen that notifies the user of the error, information (for example, a thick frame, etc.) indicating items that should be checked with priority may be displayed superimposed on the slip image.

<Other embodiments>
In the above embodiment, an example was shown in which the errors that occurred are aggregated and displayed for each prohibition rule or item, as shown in FIG. In addition, for example, a configuration may be adopted in which the tally results are associated with the layout of each item to indicate the positional relationship between the slip design and the items where errors are likely to occur. Furthermore, a configuration may be adopted in which the total results of each slip and the differences in layout of each slip are clearly shown among a plurality of types of slips.

Furthermore, if multiple errors occur with respect to one item included in a slip that falls under multiple prohibition rules, information for each of the multiple errors may be listed and displayed. Additionally, if an error occurs in each of multiple items included in the slip, for example, the configuration can be configured to display the error details one by one and accept corrections based on the above-mentioned priority. Alternatively, the configuration may be such that corrections are accepted all at once.

Additionally, when an error occurs, the server may be configured to estimate the correction content. In that case, the estimation result as the correction content may be displayed on the UI screen, and the user may select whether or not to perform the correction based on the estimation result.

In addition, if the character information of a slip is successfully registered as a result of the user's correction, the system is configured to support the user's correction work by linking with a device for correcting the slip, depending on the result. There may be. The device here includes, for example, a printing device that prints documents such as stickers, and is configured to correct the target part of the slip by printing and outputting a sticker with the content based on the correction result. It's fine. As mentioned above, the corrected character information is provided to the slip management server and delivery management server, but in this state, even if the correct contents of the slip are recorded as data in the server, the characters of the slip itself are It remains as it was before the correction. Therefore, a delivery person who visually checks the characters written on the slip and makes the delivery cannot directly recognize the corrected information. Therefore, if a correction occurs, by correcting the slip itself using the sticker, the contents recorded on the server and the contents visually checked by the delivery person can be aligned. In the above example, a sticker is printed as a document for correcting a mistake in entry, but other documents may be printed, such as reprinting the entire slip. If a sticker is used, the size of the printing device can be reduced by printing the sticker only on the portion corresponding to the corrected area. However, if the user pastes the sticker in the wrong place, or if the sticker peels off during shipping, there is a risk that the corrected information will be lost. Reprinting the entire document requires a relatively large printing device, but compared to using stickers, it is less likely to make a mistake in the place where the sticker is pasted or the sticker will peel off, so it is possible to make corrections more easily. You can definitely inform the delivery person. Further, the format of the printed document may be any other format as long as the revised content is directly or indirectly communicated to the delivery person. For example, there may be a document stating that there are corrections, or a document on which a URL, two-dimensional code, etc. for accessing a server where correct data is recorded is printed.

Furthermore, in the above embodiment, a slip for delivery of a package has been described as an example. However, the idea of the above embodiments may be applied to slips and documents other than delivery of packages. For example, there are cases in which documents submitted to government offices have rules for filling out fields, such as fields in which corrections are prohibited. With respect to such a document, it is possible to check whether it violates the input rules and to request correction based on the same concept as in the above-described embodiment. Note that when the idea of this embodiment is applied to documents other than package delivery slips, it may be possible to deal with rules that do not need to be considered in package delivery. For example, if only a "strikethrough line" is detected, it will be determined that the input rules are violated, but if a "strikethrough line" and a "correction mark" are detected, it may be determined that the input rules are met. It will be done. Note that the presence or absence of a "correction mark" can be determined, for example, by determining whether or not the surrounding red characters can be detected.

Furthermore, in the above embodiment, if an error is detected, the user is prompted to input correction information. However, it may be sufficient to simply notify that an error has been detected. This is because if it is difficult to make corrections using the terminal device 100, such as in a cold region where the touch panel cannot be operated without removing gloves, it may be more efficient to create the slip again.

Furthermore, in the above embodiment, if it is difficult to determine whether or not there is an error, a check result indicating that the determination could not be made may be further output. By doing so, it is possible to prompt the user to make additional confirmations. A specific example of a case where the determination cannot be made is when the item is unknown when checking the correlation between the item and the size.

Furthermore, in the above embodiment, a configuration is shown in which the image correction section 252, the item area detection section 253, and the character recognition section 254 are provided on the server 200 side. However, the configuration is not limited to this, and a configuration in which these parts are provided on the terminal device 100 side may be used. In such a case, the content of data transmitted and received between the terminal device 100 and the server 200 may be changed depending on the division of processing by each part.

Further, programs and applications for realizing the functions of one or more embodiments described above may be supplied to a system or device using a network or a storage medium, and one or more processors in a computer of the system or device may This can also be achieved by reading and executing a program.

Alternatively, it may be realized by a circuit that realizes one or more functions (for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array)).

Additionally, the above functions may be configured on a network using cloud computing technology. In this case, the terminal device can utilize the various functions described above via the network.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear to those skilled in the art that various changes, modifications, substitutions, additions, deletions, and equivalents may be made within the scope of the claims; It is understood that it naturally falls within the technical scope of the present disclosure. Moreover, each component in the various embodiments described above may be combined arbitrarily within a range that does not depart from the spirit of the invention.

Although various embodiments have been described above, it goes without saying that the present invention is not limited to such examples. It is clear that those skilled in the art can come up with various changes or modifications within the scope of the claims, and these naturally fall within the technical scope of the present invention. Understood. Further, each of the constituent elements in the above embodiments may be arbitrarily combined without departing from the spirit of the invention.

Note that this application is based on a Japanese patent application (Japanese Patent Application No. 2022-38223) filed on March 11, 2022, the contents of which are incorporated as a reference in this application.

The present disclosure is useful as a determination system, terminal device, determination method, and program that can improve convenience in processing slips for users.

1... Judgment system 100... Terminal device 101... Control section 102... Memory 103... Camera 104... Input section 105... Display section 151... Image acquisition section 152... Image transmission section 153... Judgment result acquisition section 154... Judgment result display section 155... Character information correction unit 200...Server 201...Control unit 202...Storage unit 203...UI unit 204...Communication unit 251...Image reception unit 252...Image correction unit 253...Item area detection unit 254...Character recognition unit 255...Error determination unit 256 ...Judgment result notification section 257...Judgment result aggregation section 258...Character information output section 300...Network

Claims

A determination system for determining the contents entered in a slip for delivery of a package, the system comprising:
an image acquisition unit that acquires a voucher image in which the voucher is photographed;
a detection unit that detects one or more areas indicating entry fields in the slip from the slip image;
a character recognition unit that performs character recognition on each of the one or more areas and identifies written character information;
an error determination unit that performs an error determination based on the character information specified by the character recognition unit and a predefined entry rule for the one or more areas;
a notification unit that notifies an area determined as an error by the error determination unit;
A judgment system with
further comprising a reception unit that receives correction information for the area determined to be an error by the error determination unit;
The reception unit is configured to selectably display whether the error is caused by a mistake in filling out the slip or a recognition error by the character recognition unit when an error is determined by the error determination unit. The determination system according to claim 1, wherein the determination system accepts.
The determination system according to claim 1, wherein the entry rule is defined for each type of slip.
The determination system according to claim 1, wherein the entry rule is defined for each of the one or more areas included in the slip.
The determination system according to claim 1, further comprising a totaling unit that totals the determination results by the error determining unit.
The determination system according to claim 5, wherein the aggregation unit displays the aggregation results of each of the one or more areas included in a slip in association with the layout of the one or more areas in the slip.
a correction unit that corrects character information in a corresponding area of the one or more areas based on the correction information received by the reception unit;
an output unit that outputs the character information corrected by the correction unit to an external device;
The determination system according to claim 2, further comprising:
The determination system according to claim 7, wherein the external device is a printing device that prints a document for correcting errors in entry on the slip.
a photographing department that photographs a document image of a document in the delivery of a package;
a request unit that requests a server to make an error determination on the slip image;
a receiving unit that receives from the server the result of the error determination of the character information written in one or more areas included in the slip image;
a reception unit that notifies an area determined to be an error by the error determination and receives correction information for the area;
a transmitting unit that transmits the correction information received by the receiving unit to the server;
A terminal device with
Processor works with memory to
Obtain the document image of the document for delivery of the package,
detecting one or more areas indicating entry fields in the slip from the slip image;
Performing character recognition on each of the one or more areas, identifying the written character information,
Performing an error determination based on the specified character information and a predefined entry rule for the one or more areas,
Notifying an area determined as an error by the error determination;
Judgment method.
to the computer,
A process of acquiring a document image in which a document for delivery of a package is photographed;
a process of detecting one or more areas indicating entry fields in the slip from the slip image;
A process of performing character recognition on each of the one or more areas and identifying written character information;
A process of determining an error based on the specified character information and a predefined entry rule for the one or more areas;
A process of notifying an area determined to be an error,
A program to run.