JP2018190064A - Accounting processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an accounting processing system which determines, when performing accounting processing by reading out an image such as a receipt, whether the image is not overlapped with an image read in the past, eliminates wrong duplicate input, and supports preparation of appropriate accounting data.SOLUTION: A system for performing accounting processing by reading out an image such as a receipt comprises: character recognition means which makes character recognition of a character including a date and an amount as a text; image feature pattern extracting means which extracts an image pattern of information useful for the accounting processing including the date, the amount and a shop name, for each prescribed region; and duplication possibility determination means which combines comparison of a result of the character recognition and the comparison of the image feature pattern, and determines duplication possibility between an image such as a new receipt, and an image such as the receipt read in the past.SELECTED DRAWING: Figure 2-1

Description

In the present invention, when reading a hail image such as a receipt (hereinafter abbreviated as “image such as a receipt”) and performing an accounting process, it does not overlap with an image such as a registered receipt read in the past. A screen is displayed to confirm whether or not it is duplicated for the user, and it is possible to prevent duplicate entries from being entered by mistake. The present invention relates to improving the efficiency and certainty of accounting processing by supporting the creation of appropriate accounting data while distinguishing between them.

2. Description of the Related Art Conventionally, for example, as in the invention described in Patent Document 1, when inputting expense data from an image such as a receipt, there is a system for converting the image into text, taking out expense data, and inputting the data. there were.

JP-T-2006-511896

Receipts such as receipts and slips are important materials for accounting processing. They are managed by companies and individual business owners and used as basic materials for accounting tax processing, and they are experts in accounting tax processing. When making a request to an accounting firm, etc., the accounting firm's staff visit the advisor to read the evidence, and the consultant will hand it over to the accounting firm's staff, etc. After referring to accounting data at the office, it is operated to return it to the consultant.
At that time, if the management of the receipt or the like is inadequate at the consultant or the like, there is a possibility that the same receipt or the like is duplicated.

Therefore, conventionally, as a process of reading an image of a receipt or the like, displaying it on a system screen, and determining whether it overlaps with an image of a receipt or the like read in the past when inputting data relating to accounting processing, A method has been used in which character recognition processing is performed on information printed on an image such as a receipt, converted into text characters, and compared with text characters of an image such as a receipt read in the past.
However, the text characters of images such as receipts vary widely depending on the store and business type to be issued, and the size of the characters to be printed is quite small. It was difficult to convert the text into text by performing character recognition processing.

In this regard, in the system described in Patent Document 1, it is described that character recognition processing is performed on a plurality of pieces of print information to determine whether or not an image such as a receipt is duplicated on condition that a plurality of pieces of the information match. However, if a plurality of pieces of print information cannot be converted into text in the character recognition process, it is not possible to determine whether or not there is an overlap with an image such as a receipt read in the past.

Moreover, the result of misrecognition may be included in the text characters acquired in the character recognition process. In such a case, there is a possibility that an image such as a duplicate receipt may be missed.
In this regard, if the character string happens to be coincident when it is misrecognized, the user's decision is made by presenting the image such as the receipt to the user who is the operator of the terminal or the like. Can wait, but as a result of misrecognition, if the text characters do not match the text characters of the receipt or other image that was read in the past, it cannot be presented to the user in the first place, and duplicated input will be made by mistake There was a possibility.

In order to solve the above-mentioned problems, a first invention of the present invention is a system for reading a testimonial image and performing accounting processing. From the read testimonial image, characters including date and amount are converted into text. Character recognition means for character recognition, and image feature pattern extraction means for extracting image feature patterns of information useful for accounting processing including date, price, and store name for each predetermined area, the character recognition means Character recognition result registration means for registering the acquired character recognition result in the storage unit, and image feature pattern registration means for registering the image feature pattern extracted by the image feature pattern extraction means in the storage unit, With respect to the newly read evidence image, the character recognition means recognizes characters including date and amount as text, and extracts the feature pattern of the image. The feature pattern of the image is extracted by the means, the character recognition result of the registered testimonial image and the feature pattern of the image are read from the storage unit, and the character recognition result of the newly read testimonial image and the registered Character recognition result comparison means for comparing the character recognition result of the testimony image, and an image for comparing the feature pattern of the image extracted with respect to the newly read testimony image and the feature pattern of the image of the registered certificate And a comparison result of the character recognition result comparison means and a comparison result of the image feature pattern comparison means, the new testimony image overlaps with the registered testimony image. New duplication possibility judging means for judging the possibility of duplication Characterized in that it prevents the uptake of such testimony leopard image.
According to a second aspect of the present invention, when the image feature pattern extraction unit extracts an image pattern of information useful for accounting processing including a date, an amount of money, and a store name for each predetermined area, The feature pattern of each image includes at least a position and range pattern or bitmap pattern aspect of a region with pixels, and the image feature pattern comparison means for each predetermined region in the image feature pattern comparison means Even if there is an error in the character recognition result or when the character recognition result cannot be obtained, the date, price, and store name can be compared by comparing the position of the area with the pixel and the pattern of the range or bitmap pattern. By determining the possibility that the information useful for the accounting process that includes them matches, the possibility of duplication determination means that the comparison result by the character recognition result comparison means and the image Comprehensively the comparison result by the feature pattern comparing means, determining the possibility that the overlapping, characterized by.

According to a third aspect of the present invention, when a plurality of testimony images are read together, the possibility of overlapping each of the testimony images is determined, and a group of the plurality of testimony images is collected. , And determining and outputting the possibility that the registered evidence image overlaps with a group of predetermined units including a daily unit and a week unit.

In addition, the fourth invention of the present invention reads a testimony image, accepts input of transaction contents relating to the testimony image as accounting data, and among the acquired accounting data, at least summary information or subject information, The image feature pattern of the testimonial image is registered in association with it, and when the next accounting data is input, in addition to the comparison process by the character recognition result comparison means and the image feature pattern comparison means, The summary information or subject information associated with the image is compared with the inputted summary information or subject to improve the accuracy of accounting data input.

According to a fifth aspect of the present invention, when the possibility of duplication exceeds a threshold value by the duplication possibility determination means, the registered evidence may be duplicated with the new evidence image. By displaying the images in comparison, it is possible to prevent a new identification image that has been determined to be possibly duplicated from being captured.

Further, the sixth invention of the present invention has enlarged one of the testimony images when the registered testimonial image that may overlap with the newly read testimony image is displayed in contrast. In the case, the other is enlarged and displayed at the same rate, and when one of the testimonial images is scrolled, the other scroll is caused to follow.


Further, according to a seventh aspect of the present invention, when a plurality of evidence images are newly read collectively or continuously, the possibility of overlapping each evidence image is determined, If there is a newly scanned proof image that may be duplicated with the registered proof image, at least two or more proof images are displayed at the same time. In addition, a newly read evidence image that may possibly overlap with the registered evidence image is displayed separately from the others.

The character recognition result and the image feature pattern distinguished for each region are stored for images such as receipts read in the past, and the character recognition result and image feature pattern of the newly read image such as receipt are stored. By comparing and judging by combining both comparison results, the possibility of overlapping can be judged in stages.
This makes it possible to determine that there is a high possibility of overlapping, taking into account that the character recognition results such as date and amount of money match, and that the feature patterns of the images match. It is possible to avoid unnecessary burdens on the user because the character recognition results such as the date and the amount of money are not coincidentally presented to the user.

Even if character recognition results such as date or amount of money include misrecognition, or even if character recognition fails and all or part of the text characters cannot be obtained, the image of date, amount or store name etc. By using the comparison result with the feature pattern together, it is possible to perform the determination of the possibility of duplication without omission, so that the accuracy of preventing duplicate input can be improved.

Moreover, in addition to determining the possibility of overlapping images such as one receipt, it is possible to determine the possibility of overlapping as a whole when reading a plurality of images continuously or collectively. Even if the accuracy of duplication determination is insufficient due to fading at the time of reading one by one, it is possible to determine the possibility of duplication with what has been read in the past as a whole, so the accounting office Receipts for each unit, such as daily or weekly, due to inadequate management of receipts, etc. Etc. can be effectively determined.

FIG. 1-1 is a diagram showing a system configuration of the present invention, and is an example showing a set of a terminal device 10 and an image capturing device 50.
FIG. 1-2 is a diagram illustrating a system configuration of the present invention, and illustrates a case where a set of the terminal device 10 and the image capturing device 50 of the present invention is connected to each other via the LAN 60.
1-3 is a diagram showing a system configuration of the present invention, and shows a case where a set of the terminal device 10 and the image capturing device 50 of the present invention is connected to the server 100 via the LAN 60. FIG. .
FIG. 1-4 is a diagram showing a system configuration of the present invention. A set of the terminal device 10 and the image capturing device 50 of the present invention is connected to the end of a WAN (wide area network) 70 via a LAN or a router 60. It is a figure which shows the case where it connects to the server 100 in.
FIG. 1-5 is a diagram illustrating a system configuration when an image such as a receipt is exchanged between an accounting office and a consultant.
FIG. 1-6 is a diagram illustrating a system configuration when an image such as a receipt is exchanged between an accounting office and a home worker.
FIG. 1-7 is a diagram illustrating a system configuration for exchanging images such as receipts between a company head office (head office) and a branch office.
FIG. 2A is a functional block diagram of the terminal device 10.
FIG. 2B is a functional block diagram of the server 100.
FIG. 3A is a flowchart showing the entire processing contents of the accounting processing system of the present invention.
FIG. 3B is a diagram showing a processing flow of a duplication determination processing subroutine for receipts and the like captured in the past, in which the processing contents of step S3A-5 of the overall processing flow are described in detail.
FIG. 3C is a diagram illustrating a modification of the duplication determination processing subroutine of FIG. 3B.
FIGS. 3D and 3E are diagrams in the case where the character recognition result comparison process and the image feature pattern comparison process of FIG.
FIG. 4 is a diagram showing a processing flow of portable processing for taking out and operating the terminal device 10 from a state in which the server 100 and the terminal device 10 are connected.
FIG. 5A is an image management table in which an image ID is assigned to an image such as a receipt read for each advisor or company, or for each organization such as a head office, a branch, or a sales office in the company. FIG. 6 is a diagram showing a state in which character recognition results of images and the like, image feature patterns, accounting data, and the like are managed in association with each other.
FIG. 5B is a diagram illustrating a data configuration of accounting data such as journal data, which is data generated using an accounting input application.
FIG. 5-3 is a table for determining the possibility that each newly read image such as a receipt is duplicated with an image such as a receipt read in the past. 2 is a diagram illustrating an example of an overlap comprehensive determination table for comprehensively determining an image feature pattern comparison result. There are (1) a default pattern, (2) an example in which a threshold value (parameter) is changed, and (3) a table prepared for each advisor, for each organization of the company, or for each type of business of the company. “Duplicate: Large” indicates that the possibility of duplication is high, but instead of increasing the possibility of duplication, the possibility of duplication may be expressed by a numerical value such as probability. A numerical value such as 0.9 (90%) is assigned when the possibility is high, 0.6 (60%) is indicated when it is medium, and 0.2 (20%) is assigned when the value is small.
Fig. 5-4 shows a bundle of images such as receipts read in the past (determined for a certain period or transaction at a certain store) when multiple sheets are read together or continuously after determining the possibility of overlap for each sheet. It is a figure which shows the example of a table for comprehensively determining the possibility of overlapping with a plurality of receipts etc.). (1) Default pattern and (2) An example in which threshold values (parameters) are changed for each advisor, each company organization, or each industry.
Reading multiple sheets collectively means reading about 10 sheets at a time with a flatbed scanner or camera, or reading about 10 sheets of receipts attached to a single sheet (paper, etc.). Yes (see FIG. 8).
Alternatively, when reading about 10 sheets of receipts attached to a single sheet (paper, etc.), cut out the images of the receipts one by one, obtain the size of the receipts, etc. Register one by one separately.
FIG. 5-5 is a table showing an example of a company master for management for each accounting firm advisor or each company organization, and FIG. 5-5 (A) shows the accounting office and advisors. FIG. 5B shows an example in which the system is operated between a head office and a branch office.
FIG. 5-6 is an overlap comprehensive determination table (for determination for each sheet) for FIG. 3C. Numerical values of the possibility of overlap of determination results (i), (b), (c), (d) for each consultant or for each business type of company. On the other hand, it is a table of a plurality of determination patterns, each of which has a constant weight.
FIG. 6A is a user confirmation screen when the possibility of duplication is equal to or greater than a certain threshold, and confirms a receipt image read in the past and a newly read receipt image or the like in comparison with the left and right. It is an example of the screen for doing.
The left-right contrast is just an example, and in addition to comparing the top and bottom, images such as receipts that may overlap are displayed in a pop-up display so that they can be compared with newly read images such as receipts. It doesn't matter.
FIG. 6-2 is a user confirmation screen when the possibility of duplication is equal to or greater than a certain threshold value. When an image such as one of receipts is enlarged and displayed, the other image is displayed at the same magnification. In addition, when one image is scrolled, the other image is automatically scrolled in the same manner.
FIG. 6C is an example of a user confirmation screen when the possibility of duplication is equal to or greater than a certain threshold value, and is an example in which there are a plurality of received receipts that may overlap.
FIG. 6-4 illustrates an example of the distinguishing display when a plurality of images such as receipts newly read this time overlap with images such as receipts read in the past. If there are many duplicate candidates, the duplicate candidates may be displayed in a list format so that the operator can easily select them.
FIG. 6-5 shows the setting of which image capturing device such as a camera or scanner is used, whether or not to perform duplication determination processing, and whether or not to perform duplication determination, or whether to read an image. This is a screen for setting whether to perform after temporary registration.
FIG. 7 shows processing of images such as receipts managed by the image management table (FIG. 5A) (contents of recognition processing of texts, extraction processing of feature patterns of images such as receipts). In addition, based on this information and data acquired by accounting data input, a product name dictionary, store name dictionary, etc. are prepared separately, and information such as product name and store name, and processing for associating with account items It is a figure which shows an outline.
FIG. 8 shows an example in which a plurality of receipts and the like are pasted and managed on a single sheet (paper) and collectively read.

[Explanation of terms]
A proof is a document that proves the contents of a transaction and the fact that the transaction has occurred, such as a receipt, receipt, receipt or slip, credit card usage details, passbook copy, etc. . This includes not only paper media but also images.
The accounting data refers to transaction information, journal data in which account items are determined from the transaction information, and other information used for accounting processing. It is a concept that includes a transaction date and amount, a product name and quantity, a store name, a description showing what kind of process and content the transaction is, and account items.
The accounting data DB is a database that stores accounting data.
An accounting office is an accounting professional organization operated by a tax accountant, accountant, or staff who performs accounting operations to which specialists with accounting expertise belong.
An advisor refers to an individual or an individual business owner who receives guidance or services related to accounting tax processing or accounting tax processing by contracting or outsourcing with an accounting firm.
The head office is a concept that includes the head office and head office in one company as well as the parent company in a group company.
Branches, etc. include branch offices and branches within one company as well as subsidiaries within a group company. The concept of a branch includes a business office and a sales office. In addition, it includes concepts such as each department of a company, and is a term indicating each organization of a company. In the accounting process between head offices, accounting data such as accounting data input by persons belonging to the accounting department with accounting knowledge is performed at the head office, etc., and transaction data by persons who do not have accounting knowledge at the branch office, etc. Are often entered.
A home worker is a person who is entrusted with a part of accounting work by a teacher or staff member of an accounting office that performs work such as accounting at a place outside the accounting office such as home. This means a contractor or individual business owner.
Auditing means that the accounting firm confirms whether or not the accounting data of the client is in compliance with accounting standards and tax laws on a monthly (monthly) or quarterly or semiannual basis or in each accounting period, etc. To do. In the case of teachers and staff such as tax accountants and accountants at accounting firms, the accounting data displayed in the accounting system is verified (checked) as an accounting audit, and as a result of the verification, the accounting data is corrected as necessary. Confirm the appropriateness of the accounting data (date, subject, amount, transaction information, etc.) displayed in the accounting system.
Image management DB assigns an ID to images such as receipts, and manages them for each advisor or company, or for each organization such as a head office, branch, or sales office in the company, and results in character recognition of images such as receipts And a database for associating data acquired by inputting characteristic data and accounting data. The contents are described in the image management table of FIG.

As shown in FIG. 5-3, the threshold value for the possibility of duplication for each image is determined by the combination of the character recognition comparison result and the image feature pattern comparison result as “high, medium possibility of duplication”. In the case where it is possible to distinguish in stages, such as “degree, low, almost none”, etc., for example, when the possibility of “medium” is duplicated and exceeds this threshold, Thus, images such as receipts that may overlap are presented to the user.

As shown in FIG. 5-4, the threshold value for the possibility of duplication for determining a plurality of sheets collectively is the past of the images such as receipts that are read when a plurality of sheets are read together or continuously. This is a threshold value for determining whether or not there is a high possibility of overlapping with an image such as a receipt registered in, and for example, there is a possibility that the image is duplicated while 10 images such as a receipt are newly read. If there are two or more cards with medium or higher, such a thing should be rare, so this “two or more (or more than 20% etc.)” is set as a threshold, and when this threshold is exceeded, the whole It is determined that there is a high possibility of overlap.

If it is not actually duplicated, but it is mistakenly judged that it is duplicated, the character recognition result such as date or amount will be misrecognized. There are cases where it is determined that the feature patterns of the images are the same. Specifically, if the same taxi company is used at the destination and the movement is repeated and the number of digits of the amount is the same, or if the same item is purchased multiple times at the same store, the date, price, store name, etc. If the character recognition result is erroneously recognized due to the similarity of the feature patterns of the images, it is determined that there is a possibility of duplication.
If the character recognition result of an image such as a receipt cannot be obtained or is misrecognized, the printing status is poor and the characters or store name is faint, and the paper such as a receipt is caused by a paper jam in the scanner. There are cases where it is broken, wrinkled, or torn.
In order to accurately determine the presence or absence of duplication in such a use environment, it is possible to adjust the duplication possibility threshold for each sheet and the duplication possibility threshold for judging a plurality of sheets together (FIG. 3A). And FIG. 3C described later).

[Configuration of one embodiment of accounting system]
FIG. 1-1 is an example showing a set of the terminal device 10 and the image capturing device 50 of the present invention. The image capturing device 50 may be a camera configured in the terminal device 10 such as a mobile device. Good.
This mode mainly assumes a case where accounting staff or accounting staff such as a company inputs accounting data using a stand-alone terminal device.
For example, when accepting a receipt of an accounting firm employee advisory company and entering accounting data, bringing a terminal device (note type or mobile) to the advisory company and entering accounting data at the advisory company, This is a case where an accounting person in a company inputs accounting data such as a receipt or a slip generated in his / her transaction.

FIG. 1-2 shows that a set of the terminal device 10 and the image capturing device 50 (see FIG. 1-1) of the present invention is connected to each other via the LAN 60 and shares an image such as a read receipt. It is a figure which shows a case.
FIG. 1-3 shows that the set of the terminal device 10 and the image capturing device 50 of the present invention is connected to the server 100 via the LAN 60 and manages images such as read receipts in the storage unit of the server 100. It is a figure which shows a case. The server 100 includes a virtual server in addition to a real server.
In the case of FIG. 1-3, the terminal device 10 reads a receipt or the like while being connected to the LAN 60 or the server 100, and disconnects from the LAN 60 or the server 100 and brings it to an advisory company or a branch to read the receipt or the like. There shall be.
Note that an image such as a receipt may be uploaded by connecting the image capturing device 50 to the server 100 via an interface such as a wireless LAN or WAN.

1-4 shows a receipt in which a set of a terminal device 10 and an image capturing device 50 of the present invention is connected to a server 100 at the end of a WAN (wide area network) 70 via a LAN or router 60 and read. It is assumed that images such as are managed in the storage unit of the server 100. Alternatively, the terminal device 10 and the image capturing device 50 may be connected to the server 100 to perform duplication determination of images such as read receipts and to register images such as receipts.
In this configuration, when the LAN system configuration side is an accounting office system or when an accounting office staff is visiting an advisory company, the brought-in terminal device 10 is transferred to the server 100 ahead of the WAN 70. By connecting, images such as receipts can be managed in the storage unit of the server 100.
Although not shown, the terminal device 10 in the LAN system may be further connected to the server device and connected to the server 100 at the end of the network (WAN 70) via the server device.

Alternatively, the LAN system configuration side may be a system of a consulting company, read a receipt, etc., and register an image, such as a receipt, in the server 100 of the accounting office ahead of the WAN 70. In addition, the LAN system configuration side may be a home worker, a subcontractor, or a system of each organization in the company.
The server 100 may be a server in an accounting office, a real server of a cloud or IDC (Internet data center), or a virtual server. In this case, the staff of the accounting office can input accounting data using the terminal device 10 connected to the server 100 while displaying an image such as a receipt sent from the consulting company.

Alternatively, the LAN system configuration side may be a system such as a branch office of a company, and the server 100 at the end of the WAN 70 may be a server installed at a head office or the like. In this case, a person in charge such as a branch reads the receipt or the like with the terminal device 10 and stores it in the storage unit of the server 100. Then, an accounting person such as a company head office enters accounting data using the terminal device 10 connected to the server 100 on the head office side.

FIG. 1-5 is a diagram illustrating a system configuration when an image such as a receipt is exchanged between the accounting office and the consultant via the WAN 70. In this case, an image such as a receipt is read and stored in the storage unit of the server 100 in addition to the case of the terminal device 10 of the advisor, and the terminal device 10 of the accounting office is separated from the server 100 and sent to the advisor. After bringing it in and reading it, it may be stored in the storage unit of the server 100 by connecting to the server 100 again after returning to the accounting office.

FIG. 1-6 is a diagram illustrating a system configuration when an image such as a receipt is exchanged between the accounting office and the home worker via the WAN 70. A home worker may be an outside trader or an individual business owner in addition to the staff of an accounting firm. In the case of an external business operator or individual business owner, the accounting office temporarily stores the images such as receipts for each consultant on the server in a state where there is no duplication judgment, and duplicate judgment and accounting data input are performed at the terminal of the home worker. To do.

FIG. 1-7 is a diagram illustrating a system configuration when an image such as a receipt is exchanged via the WAN 70 between a company head office (head office) and a branch office. In this case, an image such as a receipt is read and stored in the storage unit of the server 100 in addition to the terminal device 10 such as a branch office, or the terminal device 10 of the head office (head office) is separated from the server 100 and the branch office or the like. After returning to the main store and returning to the main store, it may be connected to the server 100 again and stored in the storage unit of the server 100.

FIG. 2A is a functional block diagram of the terminal device 10.
In the terminal device 10, an image such as a receipt is read by the image capturing device 50 connected via the communication unit 16 or the image capturing unit 12 such as a built-in camera and extracted by the OCR processing unit of the control unit 14. A character recognition process is performed on the text characters thus performed, and an image feature pattern is extracted for each region such as a receipt by the image feature pattern extraction unit.
Then, the character recognition result acquired in the character recognition process and the image feature pattern acquired by the image feature pattern extraction unit are associated with an image such as a receipt and registered in the storage unit 15, and the duplication possibility determination unit Compared with the character recognition result and image feature pattern of images such as receipts captured in the past, it is possible to determine the possibility that newly read images such as receipts overlap with images registered in the past. Then, it is presented to the user via the output unit 13 by the display image control unit.

Each processing unit that executes each function of the control unit is configured such that a CPU (central processing unit) loads a program stored in a storage unit such as an HDD into a main memory including a DRAM, an SRAM, and the like, and the CPU (Central processing unit) is implemented by executing each step of the program.

FIG. 2B is a functional block diagram of the server 100.
The server 100 may be a virtual server or other cloud in addition to a real server.
In the server 100, an image such as a receipt read by the terminal device 10 is received via the communication unit 160 and stored in the storage unit 150, and the OCR processing unit of the control unit 140 is operated according to the operation of the terminal device 10. In addition to performing character recognition processing on the text characters extracted in step 1, an image feature pattern extraction unit extracts an image feature pattern for each region such as a receipt.
Then, the character recognition result acquired in the character recognition process and the image feature pattern acquired by the image feature pattern extraction unit are associated with an image such as a receipt and registered in the storage unit 150, and the duplication possibility determination unit Compared with the character recognition result and image feature pattern of images such as receipts captured in the past, it is possible to determine the possibility that newly read images such as receipts overlap with images registered in the past. The determination result of the possibility of duplication is stored in the storage unit 150.
The determination result of the possibility of duplication is displayed on the display unit of the terminal device 10 via the WAN 70 or LAN 60 network.


[Overall process flow]

FIG. 3A is a flowchart showing the entire processing contents of the accounting processing system of the present invention.
Images such as receipts are managed for each advisor for the accounting office or for each branch for the head office, and a company master is prepared for the management.
Hereinafter, the case where the accounting office provides the accounting specialist service of the consultant will be described as an example, but the same applies to the relationship between the head office and the branch.

In FIG. 3A, first, an advisor to be processed is selected (step S3A-1), and a predetermined number of receipts are read (step S3A-2).
When reading of a predetermined number of receipts and the like is completed (step S3A-3), a character recognition process (OCR process) for grasping a character string printed on the receipt or the like as text and a printed logo are performed. Image feature patterns such as store names, product names, quantities, amounts, and dates are extracted as bitmap images or coordinates of the start and end points, and information on the position and range of rectangular areas (hereinafter referred to as image feature patterns). (Step S3A-4).

It should be noted that the image reading process such as receipts in steps S3A-2 to S3A-3 and the character recognition process such as text and the image feature pattern extraction process in step S3A-4 are performed independently of each other. Of course, reading of images such as receipts, character recognition results and feature pattern registration processing of extracted images, and overlap determination processing may be performed continuously, or images of receipts and character recognition results and extracted images. The overlap determination processing may be performed after the feature pattern registration processing.
FIG. 6-5 shows the state of the setting screen whether or not to apply this duplication determination process.
According to FIG. 6-5, when setting the type of the image capturing device 50 (internal camera, external scanner, etc.) used in the terminal device 10 and whether or not to perform duplication determination processing, the duplication determination processing is performed. In addition, it is possible to make settings such as “to do now” when reading an image such as a new receipt or “reading (separately)” after temporarily reading an image such as a receipt and temporarily registering it in the storage unit. it can.
When the setting for the duplication determination process is set to “whether it will be performed later (separately)”, a flag for performing the duplication determination later is set in the image management table in FIG. At this time, the overlap determination process is automatically executed by the flag, or a screen display for prompting the execution of the overlap determination process is performed (not shown).

As shown in FIG. 7, as an example of the feature pattern of the image, in order from the top of the receipt, etc., the logo area such as the company name, the store name and telephone number, the person in charge, the date of transaction, the product name, the quantity The amount, bar code, and the like are printed for each paragraph or area, respectively, as bitmap images, coordinates of the start and end points, and information on the position and range of the rectangular area. In addition to the company name, telephone number, person in charge, date of transaction, product name, quantity, amount of money, and store name, the contents of the transaction include names of other departments, names of business partners, branch offices, sales offices, and the like.
Here, for example, the feature pattern of the image of the date and the amount of money has a feature in which the date and the number of digits of the amount are reflected in the width, and such a feature is compared in the comparison of the feature pattern of the image. Can be used as a determination index as to whether or not the transaction contents are the same.
In addition to the partial image feature pattern for each region, there are feature patterns that grasp the entire receipt, etc., and an example of the feature pattern of the image is shown on the right side of FIG. Show.

Here, the image feature pattern extraction process may be performed after grasping the store name such as logo, and the attributes such as product name, quantity, price, and date, and in particular, the feature pattern of each image. Rather than grasping the attributes of the individual, in general, in addition to a rough grasp of what is in the upper row and what is in the middle row, it is individual information in a way of grasping that it is any information useful for accounting processing such as some store name or product name The feature pattern of the image may be extracted regardless of the attribute of the feature pattern of the image.
Ideally, it would be possible to determine the duplicates by accurately grasping the store name such as individual logos and the attributes such as product name, quantity, price, date, etc., but character recognition failed or character recognition due to character blurring etc. If the accuracy is low, duplicate determination cannot be made, so in the present invention, the character recognition fails or the character recognition accuracy is low, and the store name, product name, quantity, amount, date, etc. This is intended to widen the range where duplication can be determined assuming a case where the attribute cannot be grasped.
Further, color information can also be used as an image feature pattern.

Note that the feature pattern of the image is not limited to that illustrated in the above and the image management table in FIG. 5A, and may be anything as long as it represents the feature of the image. Of course, an external size such as a receipt is also included. This is because the outer size increases correspondingly as the number of products increases, and the unique size is adopted for each store and business partner, so the features are well expressed.
Further, any one or more of the illustrated feature patterns may be used, and the accuracy of determination is improved by combining a plurality of feature patterns.

When the character recognition process and the image feature pattern extraction process in step S3A-4 in FIG. 3A are completed, a subroutine for a process for determining overlap with an image such as a receipt read in the past is called and executed (step S3A-5).
Details of the duplication determination subroutine will be described later.

When the duplication determination process is completed for a predetermined number of receipts and the like that have been read, the user is presented with a possibility of duplication exceeding a certain threshold value, and the read receipt and other images are stored in the storage unit. A screen display requesting an operation for formal registration or deletion is performed (step S3A-6).
The features of the confirmation display will be described later.

As a result of confirmation by the user, if there is no duplication (step S3A-7: no duplication), the newly read image such as a receipt is officially stored in the storage unit in association with the recognition result and the feature pattern information of the image. Register (learn) (step S3A-8).
An image such as a receipt registered in the storage unit is thereafter managed as a receipt image read in the past, and is used when determining duplication.
On the other hand, if it is determined that there is an overlap (step S3A-7: there is an overlap), a process for deleting the newly read image such as a receipt is performed by the user's operation, and the result of the overlap determination is displayed. Register in the storage unit (step S3A-9).

When the processing for determining the overlap of images such as receipts in step S3A-9 in FIG. 3A is completed, processing for inputting accounting data is performed (step S3A-10).
In the accounting data input process, for example, an application for inputting journal data or an accounting application for inputting a cash transaction is started (step S3A-10).
Of the input information, for the summary information, register a store name dictionary, product name dictionary, etc. in advance, extract the store name and product name information from the summary information, character recognition results and image features Learning, registering, and reusing can be performed in association with pattern information (step S3A-11).

For example, in the image management table (FIG. 5-1), the store name and other information are linked and sorted for each store, so that the image management table can be used as a dictionary and image management can be performed for each store. In some stores, it is possible to associate items such as daily necessities and in some stores with food and beverage expenses. When you try to enter a different account item, a warning is displayed or a recommended item that is frequently used is automatically presented. be able to.
In the image management table (FIG. 5-1), since images such as receipts are linked to accounting data such as journals, recommended courses are presented, and store names and courses such as receipts are displayed. Related journal data can also be presented.
Further, an image such as a related receipt can be searched and presented for an image such as a certain receipt. In the search process at that time, the user selects a search item or accepts input through a UI (user interface) for inputting search conditions, and the subject, description, shop name, etc. associated with the image of the receipt, etc. The registered receipt image is searched based on the information.
Alternatively, the images such as receipts can be sorted in the order in which there are many matching items in the comparison of character recognition results such as registered receipts and feature patterns of images, and listed in the order in which there are many matching items. .

As shown in FIG. 7, the character recognition result obtained from the image such as a receipt through the character recognition process and the feature pattern extraction process of the image, and the feature pattern (part) of the image for each area are finally converted into the accounting data. The data is registered in the image management table (FIG. 5-1) in association with data (date, amount, description, store name, product name, account item) acquired by input.

[Duplicate judgment processing subroutine with registered receipts imported in the past]

FIG. 3B is a diagram showing a processing flow of a duplication determination processing subroutine with a registered receipt and the like imported in the past, in which the processing contents of step S3A-5 of the entire processing flow of FIG. 3A are described in detail.
This subroutine combines a comparison process using text characters such as date and amount acquired in the character recognition process with a comparison process using information on the feature pattern of the image. A process of determining the possibility of overlapping with an image such as a receipt read in the above is performed.
Here, “I” indicates that the result of character recognition is “Yes (result was able to be obtained)” and comparison result of character recognition is “match”, and “J” is that the result of character recognition is “Yes” (result was obtained) ) ”And the character recognition comparison result is“ partial disagreement (partial match) ”,“ K ”is the character recognition result“ none (cannot obtain the result) ”, and“ L ”is the feature pattern of the image The comparison result is “match”, the comparison result of the entire image is “match”, “M” is the comparison result of the feature pattern of the image is “match”, the comparison result of the entire image is “mismatch”, and “N” is the feature of the image It is a code that is simply added as a representation of the status of each comparison result, such as a pattern “mismatch”.

In FIG. 3B, in order to speed up the processing as necessary, first, text characters and image feature pattern information of a character recognition result of an image such as a receipt read in the past are loaded into a main memory or the like (step S3B-1), HDD, SSD, flash memory and other long-term storage media may be compared.

Images such as receipts that are referred to as duplication sources are those managed for each company ID (that is, for each advisor or branch of a company) as in the image management table of FIG.
In addition, the time range includes user settings such as “from registered receipts for transactions on the first day of the fiscal year” or “from registered receipts for transactions on the first day of the current month” or system defaults. The set range is targeted.
As the default setting of the system, for the first day of the fiscal year, for example, if the settlement is at the end of March, a setting that covers receipts from April 1 onward can be considered.
Or, if the management of receipts, etc. is strict in a company such as an advisor, and the images do not mix reliably until the previous month, the scope of the images such as receipts from the first day of the current month is set by the user setting. It may be possible to change the setting to This is because it is possible to speed up the overlap determination process by narrowing the target range.
These settings are reflected and stored in FIG. 5-5 (company master).

Next, the process proceeds to duplication determination processing. In the case where images such as receipts from the first day of the current period are targeted, if there are 1,000 images such as receipts per company per month, December Since the number of images such as receipts to be referred to is nine months from April to December, that is, 9,000, it is necessary to devise the processing contents and processing order of the duplication determination processing.
In order to speed up this process, for example, the character recognition result comparison process and the image feature pattern comparison process are started as parallel processes (step S3B-2).
Then, in the character recognition result comparison process, it is confirmed whether or not the character recognition result of the newly read image such as a receipt has been acquired (step S3B-3). (“No” in step S3B-3), the status of the comparison result is recorded as “K” (step S3B-7), and if the character recognition result can be acquired (“Yes” in step S3B-3) ), Comparing the text characters, if all match, even if some match (partial match), etc., the status of each character recognition result is "I", It is recorded as “J”,... (Steps S3B-4, S3B-7).

Further, as a process in parallel with the character recognition process from step S3B-2 in FIG. 3B, a process for comparing the feature patterns of the images is performed (steps S3B-5 and S3B-6).
Specifically, the feature patterns for each area of the image are compared (step S3B-5), and if some of the feature patterns of the image include a mismatch (“mismatch” in step S3B-5), the comparison is performed. The status of the result is recorded as “N” (step S3B-8), and when all the feature patterns for each region of the image match (“match” in step S3B-5), the image of the entire image is recorded. The feature patterns are compared (step S3B-6). If they match (step S3B-6 “match”), the comparison result is recorded as “L” (step S3B-8). In the case of mismatch (“mismatch” in step S3B-6), the comparison result is recorded as “M” (step S3B-8).

In this case, in the comparison process of the feature pattern of the image, the comparison order of each region or the entire image may be changed.
In FIG. 3B of the embodiment of the present application, an example in which parallel processing is performed is described. However, the character recognition result comparison processing and the image feature pattern comparison processing may be processing in which one of them is before or after the other.
Then, synchronization processing such as adjusting the timing is performed so that both the character recognition result comparison processing and the image feature pattern comparison processing are completed (step S3B-9).
The character recognition result comparison process is comparatively light because it compares text characters, whereas the image feature pattern comparison process is comparatively heavy. Perform comparison processing first to narrow down the images of registered receipts and the like that may be duplicated as much as possible, and then perform comparison processing of the image feature patterns to speed up the processing time. May be.

Character recognition results do not match, but the image recognition pattern is the result of the combination of the comparison process using text characters such as date and amount obtained in the above character recognition process and the comparison process using image feature pattern information. The comparison results of various aspects, including the case where they match in the comparison of (for example, the amount of money is the same, but the date is misunderstood with letters included) obtain.
Obtaining such a comparison result, comprehensive determination processing is performed for comprehensively determining the possibility that the one newly read receipt image or the like overlaps with a previously read receipt image or the like ( In step S3B-10), the determination result is stored in a storage unit such as a main memory or HDD (step S3B-11).

The determination result for each sheet is read and used in the determination of a plurality of sheets at the same time in step S3B-13, and is used when displayed on the overlap confirmation screen (FIGS. 6-1 to 6-4) such as a duplicate receipt. It is done.
Note that the comparison process of the character recognition result and the comparison process of the image feature pattern tend to have a larger processing amount in the comparison process of the feature pattern of the image. Can be determined by making a determination as to whether or not to perform the operation first.

Comprehensive determination processing for each sheet can be performed using an overlap comprehensive determination table as shown in FIG. 5-3, but is not limited to this, and can also be performed in the form of a processing flow that combines cases. .
In the overlap comprehensive determination table of FIG. 5-3, the character recognition results match (the status of the character recognition comparison results is I), and the image feature pattern comparison results are also the partial image feature patterns for each region. If the comparison result also matches and the comparison result of the image feature pattern in the entire image also matches (the situation of the comparison result of the image feature pattern is L), the possibility of overlap is high Determined.

On the other hand, when the character recognition results match (the comparison result status is I), and the comparison results of the image feature patterns include a mismatched region in the partial image feature pattern comparison results for each region. (The feature pattern comparison result status is N), and it is estimated that the number of digits such as date and amount is different. Determined to be low.

Similarly, even when the character recognition result cannot be acquired (character recognition status is K), the comparison result of the image feature patterns shows that the partial image feature patterns for each region match, and the entire image features If the patterns match, the date and amount of money are presumed to be the same, the product name and quantity are presumed to be the same, and the issued stores are also presumed to be the same. Estimated that the character recognition happened to be unsuccessful by mistake, etc., and determined that the possibility of duplication was medium, and adjusted so that duplication judgment can be prevented and presented to the user just in case Judgment.
Then, the possibility of duplication is determined for all of the predetermined number of receipts read this time (step S3B-12 in FIG. 3B).

When the process for determining the possibility of duplication for each sheet is completed, the process for determining the duplication of a plurality of sheets is performed (step S3B-13).
Here, a process using the overlap overall determination table (for determination of a plurality of sheets collectively) shown in FIG. 5-4 described later is performed. However, in the overlap determination of a plurality of sheets, a plurality of sheets are read while being read. In cases where the possibility of duplication is moderate or higher, the series of receipts etc. indicates that the possibility of duplication with images such as receipts read in the past is typologically high. The duplication judgment of each sheet complements that the possibility of duplication may not always be judged with high accuracy, and comprehensively, receipts newly read this time are insufficiently managed, and the past Information that is likely to be duplicated with receipts, etc. read in a timely manner can be presented to the user in a timely manner, and the user interrupts reading of the remaining tens or hundreds of receipts and puts them in bags, etc. The remaining receipts etc. It can be re-confirmation, in that it is not necessary to wasteful read work, is a practice useful.

In addition, due to the duplication determination of a plurality of sheets, if the image completely matches the image of a receipt read in the past, if the user re-reads the same receipt, etc. It may be determined that a new receipt including the same receipt has been read.
When the above processing is completed, the subroutine is terminated and the process proceeds to the confirmation display in step S3A-6 of the overall processing flow.

[Modified example of duplication judgment processing subroutine for registered receipts imported in the past]
A modification example of the duplication determination processing subroutine for a registered receipt or the like that has been captured in the past will be described with reference to FIGS.
The difference from FIG. 3B is that the character recognition result comparison process and the image feature pattern comparison process are converted into subroutines as shown in FIGS. 3D and 3E, respectively, and the character recognition result comparison process and the image feature pattern comparison process are performed. For each of the above, a process of changing a parameter such as a duplication determination threshold value for comparison and looping a plurality of times is added.

In addition, depending on the business partners and stores, the receipts issued, etc. tend to have a certain tendency such as small size of letters or things like logos or figures. Compatibility with feature pattern comparison processing.
Therefore, by comparing the character recognition result comparison process and the image feature pattern comparison process with each other, it is possible to narrow down images such as receipts that match the character recognition result comparison process, or the image feature pattern. Multiple processes such as whether to narrow down images such as receipts that match in the comparison process, or whether both comparison processes are operated in parallel, and both comparison processes are merged instead of narrowing down one of them By running the pattern, it is possible to select a comparison method having good compatibility with each other, weight each of them, and optimize the numerical value of weighting by learning.

In FIG. 3C, for example, when the printed character is large, the image is clear, and the reliability of the character recognition result is high, first, the receipt processing read in the past by comparing the character recognition result in step S3C-3. The images that are likely to overlap are narrowed down to some extent, and subsequently, the image feature pattern comparison process is performed in step S3C-4 for those that are suspected of being duplicated in the comparison of the character recognition results. (Step S3C-5), the character recognition result comparison process and the image feature pattern comparison process can be performed again.
Alternatively, if the printed characters are small or there are a lot of blurs in an image such as a receipt, the reliability of the character recognition result is low. Character patterns similar to each other are picked up, and then a character recognition result comparison process is executed later, further narrowing down and further changing parameters such as a threshold (step S3C-5), and comparing character recognition results The process and the comparison process of the feature pattern of the image can be performed again.

In FIG. 3B, in the overlap determination process of (a) and (b) and the overlap determination process that combines c and c, (1) the default pattern of FIG. 5-3, or customized for each advisor or company type Using the determination table, each duplication determination process can be repeated by changing the threshold value (parameter).
Synchronization processing is performed so that each duplication determination process is completed (step S3C-7), and four duplication judgment results (each) are obtained as the judgment results of the duplication judgment processing. It is done.
The results of C and D are applied to the duplication determination table of FIG. 5-5, and the duplication possibility is output by combining the character recognition result comparison and the image feature pattern comparison result.
Each determination result is output with a high possibility of duplication, medium, small, etc., or 0.9 (90%) when the possibility of duplication is high, 0.6 (60%) when it is medium, and small In the case of, a numerical value such as the probability of duplication such as 0.2 (20%) may be used.

Then, after four overlap determination results (each) are obtained (i, b, c), the weight of each overlap determination process is weighted according to the compatibility. A comprehensive determination of the possibility of duplication is performed (S3C-8).
The comprehensive determination is output by applying to the overlap comprehensive determination table (for determination for each sheet) (for FIG. 3C) in FIG.
At that time, weighting (numerical values of a, b, and c) for each determination result can be optimized for each consultant or for each type of business of the company, and optimization for duplication determination according to each business condition can be performed.
In addition, adjusting the weighting for each advisory company or each industry of a company often has a certain range of business partners for each advisory or industry, and as a result, a certain amount is also applied to receipts, etc. Since there is a tendency, the order of the character recognition results and the comparison process of the image feature patterns and threshold values (parameters) in the image duplication judgment processing such as receipts for each consultant and for each industry This is because compatibility occurs with the setting.

The comparison process of character recognition results is a relatively light process because it compares text characters, whereas the comparison process of image feature patterns is a relatively heavy process. The comparison process is performed first to narrow down the images such as receipts that may be duplicated as much as possible, and thereafter the process of comparing the feature pattern of the image (that is, the process up to “A” in FIG. 3C) is weighted. It is also possible to increase the processing time (in comparison with the processing up to “B” in FIG. 3C in which the comparison of the image feature patterns is performed first) to increase the processing time.

  Although the duplication determination table of FIGS. 5-6 showed the example prepared for every determination pattern like the determination patterns 1, 2, 3, ..., the weight (a, b, c) of each determination pattern is as follows. When the possibility of duplication is above a certain level (for example, medium (0.6) or more), present it to the user, check whether it is actually duplicated, obtain the confirmation result, The error between the determination result of the possibility and the confirmation result of the user is converted by an error function such as a mean square error, and weight values (a, b, c) of each determination pattern so that the error function is minimized. Can be performed to improve the accuracy of the determination pattern.

[Mobile processing flow]
As in the case of FIG. 1-5 and the like, in the case where the accounting server 100 and the staff terminal device 10 constitute a client server system, the terminal device 10 is separated from the server 100 and brought to the advisor. Thus, when an image such as a receipt is read by the consultant, the terminal device 10 is carried by the consultant, so that it is referred to as portable processing.
A processing flow of such portable processing is shown in FIG.
In FIG. 4, after determining the target advisor (step S4T-1), the data of the advisor to bring in the terminal device 10 (image data such as a receipt, character recognition result associated therewith, image feature pattern) Information from the accounting office server 10 (step S4SV-1) and temporarily stored in the storage unit of the terminal device 10 (step S4T-2) to prepare for duplication determination at the consultant at the destination. Can do.

In this case, it is also possible to perform a process of locking all the data of the advisor to be processed so that the data of the advisor cannot be added or updated at all (in step S4SV-2). “Lock”), without locking, another staff member of the accounting office uses another terminal device 10 to continue the process of capturing images such as receipts, or the advisor directly The server 100 may be accessed so that an image such as a receipt can be uploaded ("Do not lock" in step S4SV-2).

<Common processing when locking and not locking advisor data on the server>
In FIG. 4, an image such as a receipt is read by the terminal device 10 brought into the advisor (step S <b> 4 </ b> T- 3), and the image such as a receipt stored in the storage unit of the terminal device 10 and the character recognition result associated with the image are displayed. A comparison with the image feature pattern is performed, and an image such as a receipt is taken in while performing duplication determination processing at the consultant (step S4T-4).

<When locking advisor data on the server>
In the accounting office server 100, if the data of the advisor is locked, after returning to the accounting office, the server 100 is connected to the server 100 again, and the receipt or the like read by the advisor as it is. An image is additionally registered (step S4T-8, step S4SV-5).

<If the server does not lock the advisor data>
On the other hand, in the accounting office server 100, when the data of the advisor is not locked, the duplication determination with an image such as a receipt added from another staff member or advisor during the portable processing is further performed ( In step S4T-6), additional registration is performed for those that do not overlap (step S4T-7, step S4SV-4).

[table]
FIG. 5A is an image management table in which an image ID is assigned to an image of a read receipt or the like for each advisor or company, or for each organization such as a head office, a branch, or a sales office in the company. Information on character recognition results of images such as receipts and information on image feature patterns, information on partial image feature patterns for each region, or information on feature patterns of the entire image, and information on images such as receipts by the user Store names, business partner names, product names, descriptions, and account information obtained by inputting transaction details as accounting data such as journal data are registered and managed in association with each other.
Here, for example, the company ID K0001 indicates the company B (consultant B).
The feature pattern of the entire image includes a feature pattern of the size of the outer frame of the image such as a receipt, in addition to the feature pattern of the image of the entire print area.
Also, whether or not duplication determination processing is to be performed on the reading setting screen of FIG. 6-5, a flag for saving the setting of whether to perform now or later, and whether or not duplication determination processing has been executed. It has a flag.

Here, the feature pattern (abbreviated as feature PTN) in FIG. 5A is projected from the rectangular width and height of a region with dots, bitmap, and information quantized in a histogram format (X and Y directions). And the like, and feature amounts including information vectorized by mathematical processing.
Since the feature PTN is an image related to some kind of accounting, it has attributes (number, product name, date, amount of money, etc.) in accounting processing of each area, so the position, range, width, height, etc. are important.
A feature pattern (part) is a feature pattern of an image for each region, and includes (feature 1 (feature PTN such as store name), feature 2 (position, feature PTN), feature 3 (position, feature PTN),. Can be described as:

FIG. 5-2 is a diagram showing the data structure of accounting data such as journal data, generated using an accounting input application, etc., and journal data IDs are assigned to each advisor, Information such as a summary, amount, date, and the like are associated with each other, and an image management table also associates which receipt or other accounting data is related to the image.

Fig. 5-3. Overlapping comprehensive determination table (for determination for each sheet) and Fig. 5-4. Since the duplicate comprehensive determination table (for determining a plurality of sheets) has been described above, the details are omitted, but FIG. In the duplicate comprehensive judgment table (for judgment of multiple sheets collectively), as a judgment condition, it is described that two or more sheets have been read. However, when multiple sheets are duplicated, the ratio is 20% or more. It can be replaced with a certain threshold such as overlapping.

In short, if 10 sheets are read and there is a possibility that one sheet is duplicated, the 10 sheets are insufficiently managed within the range of one week ago, the previous month, the same month last year, the same store, It can be said that there is a low possibility of overlapping with images such as receipts read in the past, but if it is determined that there is a high possibility that two or three images are overlapping, the whole There is a high possibility that images of 10 receipts and many receipts to be read subsequently are duplicated with images of receipts, etc. that have been read in the past in the past. Before reading the, you can perform operations that check carefully, so that you can perform lean accounting.

[Characteristics of screen display when it is determined that there is a possibility of duplication]
Next, a description will be given of a confirmation screen for a receipt or the like displayed when it is determined in step S3A-6 in the overall processing flowchart (FIG. 3A) that the possibility of duplication is a certain level or more (possibility is medium or more). To do.
FIG. 6A is a confirmation screen (left-right comparison) of a receipt or the like when it is determined that the possibility of overlap is a certain level or more (medium or more possibility). In this screen example, on the right side, among the newly read receipt images and the like, three images that may overlap with the previously read receipt images and the like are listed, and of these, the receipt number is ID0065. When an item is selected, the receipt image is displayed, and at the same time, the left-side screen displays a duplicate original receipt image that may overlap among previously read receipt images. It shows how it is.
In this screen example, since it can be confirmed while comparing images such as receipts that may overlap on the left and right, the work efficiency is improved.
In addition, the screen to check with the left and right contrast is an example, only the image such as a newly read receipt is displayed, and if you focus the mouse cursor etc. on the image such as a receipt, it has been registered in the past Alternatively, an image such as a receipt may be displayed in a pop-up display (the same applies to FIGS. 6-2 and 6-3).

FIG. 6-2 shows one of the receipt images in contrast to the confirmation screen (left / right comparison) of the receipt or the like when it is determined that the possibility of overlapping in FIG. 6-1 is a certain level or more (medium or more possibility). Is displayed with the same enlargement factor, and when one image is scrolled, the other image is automatically scrolled in the same manner. By applying the value of the scroll amount of the image to the other, the same portion can be enlarged and confirmed, and it is supported to surely recognize whether or not they overlap.

FIG. 6C is an example in which there are a plurality of receipts and the like that may be duplicated on the confirmation screen for duplicate receipts.
FIG. 6A and FIG. 6B illustrate a case where there is one captured receipt image that may be duplicated with respect to a newly read receipt image. On the other hand, in FIG. 6C, there is a difference that there is a plurality of images such as receipts that may be duplicated with respect to one newly read image such as a receipt.
In FIG. 6C, there is a possibility of duplication in the case where there are two images, such as already received duplicate original receipts, which may be duplicated with respect to one newly read receipt image or the like. Receipt numbers are listed in descending order, and the selected image is displayed on the left side, and it is shown that it can be confirmed in comparison with the image such as a receipt newly read this time.
As shown in the overlap overall determination table of FIG. 5-3, the level of possibility of overlap may be distinguished as high or medium, and after the user finally confirms and learns the result of overlap determination Thus, a level expressed by a probability may be used.

FIG. 6-4 illustrates an example of the distinguishing display when a plurality of images such as receipts newly read this time overlap with images such as receipts read in the past.
On the right side, five newly read receipts and other images are displayed. Of these, the background is distinguished and displayed for those that are more than a certain level that may overlap with previously received receipts and other images. An example is shown.
In this case, if you place the mouse cursor on an image of a receipt that may overlap on the right side, or touch the image of the receipt on the touch panel on the screen in the case of a mobile terminal In the screen on the left, images such as already received duplicated receipts that may be duplicated are displayed one after another, and it is possible to check whether or not they are duplicated.

In this example, it is shown that there is a possibility that images such as two receipts out of five may overlap. In such a case, management of receipts is insufficient, and an attempt is made to read from now on. Since images such as receipts may overlap in a typical manner, the top of the screen says, “This group of receipts is likely to overlap with previously received receipts. By displaying a warning message such as “Please pay attention!”, The user immediately stops reading receipts and other images, checks the management status of receipts and other images stored in bags, and takes receipts that have already been imported. Therefore, it is possible to prevent useless reading work.
Although not shown, there may be a case where there are a plurality of duplicate source receipts that have been captured. In this case, the duplicate source receipts may be displayed as shown on the left side of FIG. 6-3.

Regardless of whether or not the above-described duplication determination process is performed, information such as a character recognition result such as a receipt, an image feature pattern, and an account item, a description, or a store name is registered. On the other hand, it is possible to search for items with many hits using these pieces of information as search keys.
For an image of a certain receipt, etc., use the information such as account item, description, or store name as a search key to search and enumerate images of related receipts, etc. It is also possible to refer to accounting data such as a summary or journal entry.

Alternatively, for images such as receipts, search for similar receipt images using the character recognition results and image feature pattern information as search keys, and list them in the order of the number of hit items. It is also possible to refer to account data associated with images such as receipts and accounting data such as abstracts or journals. In this case, either one of the character recognition result and the comparison of the feature pattern of the image can be used, or both comparison processes can be combined in the same manner as the overlap determination. Sorting in the order of matching items in the comparison of character recognition results and image feature patterns, etc., and listing them in order of the number of matching items, or listing only the top 10 items is possible. .

The above search process and enumeration display can be applied to both when a new image such as a receipt is read, or when searching for an image such as an arbitrary receipt among already read receipts registered. It is.

As described above, by combining the comparison of the character recognition result and the comparison of the feature pattern of the image, and determining the possibility of overlapping, even when character recognition cannot be performed accurately due to blurring or small characters, The possibility of overlapping can be determined with high accuracy.

In addition, when the possibility of duplication is more than a certain level, by displaying a screen that makes it easy for the user to check whether there is duplication, by preventing registration of images such as duplicate receipts In addition to preventing the creation of duplicate accounting data, it is an image of a similar receipt, etc., but it is possible to enter accounting data after confirming that there were really two different transactions By supporting the creation of appropriate accounting data while distinguishing between the occurrence of double journals and the occurrence of two transactions, accounting processing can be made more efficient and reliable.
And according to this system, the auditing side of the accounting office or the head office in the company goes back to the testimonial image of the receipt etc. even at the auditing stage for the service receiving side such as the advisor or the branch of the company It is easy to give accurate guidance on the above, and it can lead to operational efficiency.

10 terminal device 11 input unit 12 image capture unit 13 output unit (display, print)
14 Control unit 15 Storage unit (main storage such as RAM, long-term storage such as SRAM and HDD)
16 Communication unit 50 Image capture device (camera, scanner)
60 LAN (Local Area Network)
70 WAN (Wide Area Network, Internet etc.)
100 servers (including virtual servers)
130 Output unit 140 Control unit 150 Storage unit 160 Communication unit

Claims (7)

A system for reading a testimony image and performing accounting processing,

From the read proof image,
Character recognition means for recognizing characters including date and amount as text,
Image feature pattern extraction means for extracting the feature pattern of the image of information useful for accounting processing including date, price, and store name for each predetermined area;

Character recognition result registration means for registering the character recognition result acquired by the character recognition means in a storage unit;

Image feature pattern registration means for registering the image feature pattern extracted by the image feature pattern extraction means in the storage unit;

For the newly read proof image,
The character recognition means recognizes characters including date and money as text,
Extracting the feature pattern of the image by the feature pattern extraction means of the image,

From the storage unit, read the character recognition result of the registered proof image and the feature pattern of the image,
A character recognition result comparing means for comparing the character recognition result of the newly read testimony image and the character recognition result of the registered certificate image, and a feature pattern of the image extracted for the new read testimony image; Image feature pattern comparison means for comparing the image with the feature pattern of the registered card image,
In combination with the comparison result by the character recognition result comparison means and the comparison result by the feature pattern comparison means of the image,
A duplication possibility determination means for determining a possibility that the new certificate image and the registered certificate image overlap;
By having

An accounting processing system for preventing a new identification image that has been judged to be possibly duplicated by the duplication possibility judging means.

In the image feature pattern extraction means, when extracting an image pattern of information useful for accounting processing including date, price, and store name for each predetermined area, at least a pixel as an image feature pattern for each area Including aspects of the location and range pattern or bitmap pattern of the region,
In the image feature pattern comparison means, there is a pixel even if there is an error in the character recognition result or the character recognition result cannot be obtained by comparing the feature pattern of the image for each predetermined region. By comparing the position of the area and the pattern of the range or bitmap pattern, it is possible to determine the possibility that the information useful for accounting including date, amount, and store name matches,
The duplication possibility determination means determines the possibility of duplication by combining the comparison result by the character recognition result comparison means and the comparison result by the feature pattern comparison means of the image;
The accounting system according to claim 1.
When a plurality of evidence images are read together, the possibility of duplication of the evidence images is determined, and the date of the registered evidence images is determined for the collection of the plural evidence images. Determine and output the possibility of overlapping with a group of predetermined units including units and weeks,
The accounting system according to any one of claims 1 and 2.
Reads the testimony image, accepts the transaction details related to the testimony image as accounting data, and associates at least summary information or subject information with the acquired accounting data and the feature pattern of the image of the testimony image In addition to comparison processing by the character recognition result comparison unit and the image feature pattern comparison unit when inputting the next accounting data, the summary information associated with the test image or Compare the subject information with the summary information or subject you entered to improve the accuracy of accounting data input,
The accounting system according to any one of claims 1 to 3, wherein
When the possibility of duplication exceeds a threshold by the duplication possibility determination means,
By displaying the registered proof image that may overlap with the new proof image, in contrast,
5. The accounting system according to claim 1, further comprising: taking in a new evidence image determined to have a possibility of overlapping.

In contrast to displaying the registered proof image that may overlap with the newly read proof image,
When one proof image is enlarged, the other is enlarged and displayed at the same rate,
When scrolling one of the testimonial images, make the other scroll follow,
The accounting system according to claim 5.
When a plurality of new proof images are read all at once or when they are read consecutively, it is determined whether each proof image is duplicated, and it overlaps with the registered proof image. If there is a possibility that a newly read testimony image exists, at least two or more of the testimonial images read together or continuously are displayed at the same time, and the registered testimonial image and Display newly read evidence images that may be duplicated separately from others,
The accounting processing system according to claim 1, wherein: