JP6507459B2 - Accounting system - Google Patents

Description

In the present invention, when a proof image such as a receipt (hereinafter referred to as “image such as receipt”) is read and accounting processing is performed, it may not overlap with an image such as a registered receipt read in the past Determine whether and display the screen to confirm whether the user is overlapping, to prevent accidental duplicate input, and the occurrence of double journaling and the fact that two transactions actually occurred To create more efficient accounting data, and to make accounting more efficient and reliable.

Conventionally, when inputting expense data from an image such as a receipt, for example, as in the invention described in Patent Document 1, a system for converting the image into text, extracting the expense data, and inputting the data there were.

Japanese Patent Application Publication No. 2006-511896

Proofs such as receipts and slips are important materials for accounting, and they are managed by companies and individual business owners and used as basic materials for accounting and tax processing, and they are specialists in accounting and tax processing. When asking the accounting office etc., staff of the accounting office visit the adviser to read the certificate, or hand over or mail to the staff of the accounting office from the adviser, accounting affairs After referring to it at the time of accounting data input at the office, it is managed to return to the adviser.
At that time, if the management of the receipts and the like is insufficient at the adviser or the like, there is a possibility that the same receipts and the like are read in duplicate.

Therefore, conventionally, an image such as a receipt is read and displayed on the screen of the system, and when performing data input relating to accounting processing, as processing to determine whether or not an image such as a receipt read in the past is overlapped. A method has been used in which character recognition processing is performed on information printed on an image such as a receipt, text conversion is performed, and comparison is made with text characters of an image such as a receipt read in the past.
However, the text characters of images such as receipts vary depending on the store and type of business, and the aspect varies widely, and the size of the characters to be printed is considerably small. It was difficult to convert all into text by performing character recognition processing.

In this respect, in the system described in Patent Document 1, it is described that by performing character recognition processing on a plurality of print information, an overlap determination of an image such as a receipt is performed on the condition that a plurality of pieces of the information match. However, if character recognition processing can not successfully convert a plurality of print information into text, it has not been possible to judge duplication with an image such as a receipt read in the past.

In addition, the text character acquired by the character recognition processing may include the result of erroneous recognition, and in such a case, there is also a possibility that an image such as a duplicate receipt is missed.
In this respect, when the character string is coincident by chance when it is erroneously recognized, the user is determined by presenting the image such as the receipt to the user who is the operator such as the terminal. It is possible to wait, but as a result of misrecognition, if the text character does not match the text character of the image such as a receipt read in the past, it can not be presented to the user in the first place, resulting in erroneous duplicate input There was a possibility.

In order to solve the above-mentioned problems, a first invention of the present invention is a system for reading a proof image and performing accounting processing, in which a character including a date and an amount is used as text from the read proof image. Character recognition means for character recognition, and image feature pattern extraction means for extracting a feature pattern of an image of information useful for accounting processing including a date, an amount, and a store name for each predetermined area; Character recognition result registration means for registering the obtained character recognition result in the storage unit, and image feature pattern registration means for registering the feature pattern of the image extracted by the feature pattern extraction means of the image in the storage unit, The character recognition means character-recognizes characters including date and amount of money as texts for the newly read out said proof image, and extracts the feature pattern of the image. The feature pattern of the image is extracted by the means, and the character recognition result of the registered proof image and the feature pattern of the image are read out from the storage unit, and the character recognition result of the new read proof image and the registered Character recognition result comparison means for comparing the character recognition result of the proof image, and an image for comparing the feature pattern of the image extracted about the new read proof image with the feature pattern of the image of the registered proof image And a combination of the comparison result by the character recognition result comparison unit and the comparison result by the feature pattern comparison unit of the image, the new proof image and the registered proof image are overlapped with each other. New possibility that is determined to have a possibility of duplication by providing a means for determining possibility of duplication Characterized in that it prevents the uptake of such testimony leopard image.
Further, according to the second aspect of the present invention, in extracting the pattern of the image of the information useful for accounting processing including the date, the amount of money, and the name of the store in each of the predetermined regions in the feature pattern extraction unit of the image The feature pattern of each image includes at least the pattern of the position and range of the region with pixels and the aspect of the bit map pattern, and the feature pattern comparison means of the image, the feature pattern aspect of the image for each predetermined region Even if there is an error in the character recognition result or the character recognition result can not be obtained by comparing the date, amount, and store name by comparing the position of the area with the pixel with the range pattern or bitmap pattern. The duplication possibility determination means determines the possibility that the information useful for the accounting process including is matched, the comparison result by the character recognition result comparison means and the image Comprehensively the comparison result by the feature pattern comparing means, determining the possibility that the overlapping, characterized by.

Further, according to the third invention of the present invention, when a plurality of proof images are read at one time, determination is made as to the possibility of duplication of each of the proof images, and a combination of the plurality of proof images is determined. And determining and outputting the possibility that the registered proof image is overlapped with a predetermined unit including daily units and weekly units.

Further, according to a fourth aspect of the present invention, at least abstract information or subject information among accounting data acquired by reading a proof image, accepting an input of contents of a transaction regarding the proof image as accounting data, In addition to the comparison process by the comparison means of the character recognition result and the comparison means of the image at the time of the input of the next accounting data, the relevant certificate is registered in association with the feature pattern of the image of the proof image. The summary information or the subject information associated with the image is compared with the input summary information or the subject to improve the accuracy of the input of the accounting data.

Further, according to a fifth aspect of the present invention, in the case where the duplication possibility determining means causes the registered evidence to possibly duplicate the new evidence image when the duplication probability exceeds a threshold. The images are displayed in contrast to prevent capture of a new proof image that is determined to have a possibility of overlapping.

The sixth invention of the present invention enlarges one proof image in the case where the registered proof image which may overlap with the newly read proof image is displayed in contrast. In this case, the other is enlarged and displayed at the same rate, and when one proof image is scrolled, the other scroll is made to follow.


In the seventh invention of the present invention, when a plurality of proof images are newly read collectively or continuously read, each duplication possibility of the proof images is determined, If there is a newly read proof image that may overlap with the registered proof image, display at least two or more proof images simultaneously read in a batch or continuously And displaying a newly read proof image that may overlap with the registered proof image separately from the others.

For images such as receipts read in the past, character recognition results and feature patterns of images distinguished for each area are stored, and character recognition results of images such as receipts newly read and feature patterns of images By comparing and judging by combining both comparison results, the possibility of overlapping can be judged stepwise.
Thereby, in addition to the fact that character recognition results such as date and amount of money match, it is possible to determine that the possibility of overlapping is high in consideration of the fact that the feature patterns of the images also match. Since it is possible and presented to the user as if it happens that the character recognition results such as date and amount of money are identical and duplicated, there is an effect that the user is not burdened unnecessarily.

In addition, even if incorrect recognition is included in the character recognition result such as date or amount, or even if the character recognition fails and all or part of the text character can not be acquired, the image of date, amount, store name, etc. By using the comparison result with the feature pattern as well, the determination of the possibility of overlapping can be performed without omission, so that the accuracy of preventing duplicate input can be improved.

Further, in addition to determining the duplication possibility of images such as one sheet of one receipt, when reading a plurality of sheets continuously or collectively, it is possible to determine the possibility of duplication as a whole. Even if the accuracy of the duplicate determination is insufficient due to blurring at the time of reading each sheet, it is possible to determine the possibility of overlapping with what was read in the past as a whole. If there is a leak in the management of receipts etc. and duplicate readings have been made between the adviser and the adviser, receipts per predetermined unit such as daily units or weekly units due to a defect in management of receipts etc. promptly Etc. can be effectively determined.

FIG. 1A is a diagram showing a system configuration of the present invention, and is an example showing a set of a terminal device 10 and an image capturing device 50. As shown in FIG.
FIG. 1B is a diagram showing a system configuration of the present invention, and shows a case where sets of the terminal device 10 and the image capturing device 50 of the present invention are mutually connected via the LAN 60.
FIG. 1-3 is a diagram showing a system configuration of the present invention, showing a case where a set of the terminal device 10 and the image capturing device 50 of the present invention is connected to the server 100 via the LAN 60. .
1-4 is a view showing a system configuration of the present invention, and a set of the terminal device 10 and the image capturing device 50 of the present invention is a point of a WAN (wide area network) 70 via a LAN or a router 60. It is a figure which shows the case where it connects to the server 100 in FIG.
FIGS. 1-5 is a figure which shows the system configuration | structure at the time of exchanging images, such as a receipt, between an accounting office and an advisor.
1-6 is a figure which shows the system configuration | structure at the time of exchanging images, such as a receipt, between an accounting office and a home worker.
FIG. 1-7 is a diagram showing a system configuration when exchanging an image such as a receipt between a head office (head office) of a company and a branch office or the like.
FIG. 2A is a functional block diagram of the terminal device 10.
FIG. 2B is a functional block diagram of the server 100.
FIG. 3A is a flowchart showing the entire processing contents of the accounting system of the present invention.
FIG. 3B is a diagram showing a processing flow of an overlap determination processing subroutine with a receipt or the like captured in the past, in which the processing content of step S3A-5 of the entire processing flow is described in detail.
FIG. 3C is a diagram showing a modification of the overlap determination processing subroutine of FIG. 3B.
FIGS. 3D and 3E are diagrams in the case where the comparison processing of the character recognition result of FIG. 3B and the comparison processing of the feature patterns of the image are respectively independent subroutines.
FIG. 4 is a diagram showing a processing flow of portable processing in which the terminal device 10 is brought out for operation from the state where the server 100 and the terminal device 10 are connected.
Fig. 5-1 shows an image management table, which assigns an image ID to an image such as a receipt read for each adviser or company, or for each organization such as a head office, branch or sales office within the company, and receives the receipt. , Etc. are shown in association with each other as a result of character recognition of an image, a feature pattern of the image, accounting data, and the like.
FIG. 5-2 is data generated using an accounting input application or the like, and is a diagram showing a data configuration of accounting data such as journal data.
FIG. 5C is a table for determining the possibility of overlapping with an image such as a receipt read in the past, for each image of a newly read receipt etc., and a comparison result of character recognition results It is a figure which shows an example of the duplication synthetic | combination determination table for combining and determining with and the comparison result of the feature pattern of an image. There are (1) default patterns, (2) an example where threshold values (parameters) are changed, and (3) tables prepared for each adviser, each organization of a company, or each industry of a company. “Duplicate: large” indicates that the possibility of duplication is large, but instead of making the possibility of duplication large, the possibility of duplication may be represented by a numerical value such as probability, or duplication If the possibility is high, a numerical value such as 0.9 (90%), medium (0.6), or 0.2 (20%) will be allocated.
Fig. 5-4 shows a bundle of images such as receipts read in the past (a transaction in a certain period or a certain store, when a plurality of sheets are read collectively or continuously after determining the possibility of duplication for each sheet) Is a diagram showing an example of a table for comprehensively determining the possibility of overlapping with a plurality of receipts etc.). An example is shown in which the threshold (parameter) is changed for (1) default pattern and (2) for each adviser, for each organization of a company, or for each type of industry.
When reading multiple sheets at once, when reading about 10 sheets at once with a flatbed scanner or camera, or when reading a sheet with about 10 receipts etc. on a sheet (paper etc.), etc. There is (see FIG. 8).
Alternatively, when reading a sheet with about 10 receipts etc. attached to one sheet (paper etc.), cut out one image each of the receipt etc. and acquire the size etc. of the receipt etc. Register one by one separately.
FIG. 5-5 is a table showing an embodiment of a company master for managing for each accounting firm adviser and company organization, and FIG. 5-5 (A) is an accounting firm and adviser FIG. 5-5 (B) shows an example in which the system is operated between the head office and the branch office.
Fig. 5-6 is a table of overlapping comprehensive judgment for Fig. 3C (for judgment for each sheet), and for each adviser or for each type of business of the company, numerical values of the judgment results of the judgment results i, b, c and d. The table is a table of a plurality of determination patterns, each of which has a certain weight.
FIG. 6-1 is a confirmation screen of the user in the case where the possibility of duplication is equal to or higher than a certain threshold value, and the image of the receipt read in the past and the image of the newly read receipt etc. It is an example of the screen for doing.
Contrasting the left and right is only an example, and in addition to contrasting in the top and bottom, a pop-up display of images such as receipts that may overlap with the newly read images such as receipts can be displayed. I don't care.
FIG. 6-2 is a confirmation screen of the user in the case where the duplication possibility is equal to or more than a certain threshold, and in the case where an image such as a receipt is enlarged, the other image is displayed at the same magnification In addition, when one image is scrolled, the other image is similarly scrolled automatically.
FIG. 6C is a confirmation screen of the user in the case where the possibility of duplication is equal to or more than a certain threshold, and is an example in the case where there are a plurality of received receipts and the like which may be duplicated.
FIG. 6-4 shows an example of a distinguishable display in the case where a plurality of sheets of a plurality of images such as receipts newly read this time overlap with an image such as a receipt read in the past. If there are many duplicate candidates, the duplicate candidates may be displayed in a list format so that the operator can easily select.
Fig. 6-5 shows which image capture device to use, such as a camera or scanner, and whether or not to perform overlap determination processing, and whether to perform overlap determination now, or to read an image. It is a screen for setting whether to perform after temporary registration.
FIG. 7 shows a process for an image such as a receipt, etc., regarding an image such as a receipt managed in the image management table (FIG. 5-1) (content of recognition process of text etc., extraction process of feature pattern of image such as receipt) And, based on these information and data acquired by input of accounting data, separately prepare item name dictionary and store name dictionary etc., and processing such as information such as item name and store name and association with account items It is a figure showing an outline.
FIG. 8 shows an example of a case where a plurality of receipts and the like are attached to one sheet (paper) and managed, and collectively read.

[Explanation of terms]
Proof is a document that proves the details of the transaction and the fact of the transaction, such as receipts, receipts, receipts and vouchers, credit card statement details, copy of passbook, etc., and may be called primitive vouchers etc. . In addition to paper media, this includes cases of image acquisition.
Accounting data refers to information on transactions, journal data in which account items are determined from information on transactions, and other information used for accounting. It is a concept that includes the date and amount of the transaction, the item name and quantity, the store name, as well as an abstract indicating what kind of process and content the transaction is, account items, and the like.
The accounting data DB is a database for storing accounting data.
An accounting firm is an accounting professional organization that is operated by a tax accountant, an accountant, and staff members, and performs accounting processing to which a specialist with accounting expertise belongs.
An adviser refers to an individual or an individual business owner who receives guidance and services related to accounting tax processing and accounting tax processing by a contract with an accounting office or outsourcing.
The head office etc. is a concept including the head office in one company and the parent company in the group company besides the head office.
Branches, etc. include branch offices and branches within one company and subsidiaries within group companies. The concept of branch offices includes business offices and sales offices. It also includes the concept of each department of a company, which is a term that indicates each organization of a company. In addition, in the accounting process between branch offices, accounting processes such as accounting data input are performed by those who belong to the accounting department having accounting knowledge in the head office etc. Transaction data by persons who do not have accounting knowledge in the branch office etc. Input is often performed.
An at-home worker (home-based person) means that an accounting office teacher or staff who performs accounting work at a place outside the accounting office such as a home is outsourced when part of the accounting work is outsourced Company or private business owner etc.
An audit refers to whether the accounting firm checks the accounting data of the adviser on a monthly basis (monthly basis) or quarterly accounting period to quarterly accounting period and whether it complies with accounting standards and tax laws, and revises them as appropriate. Say what to do. In the case of an accounting firm's tax accountant or a teacher or staff member such as an accountant, as accounting audit, the accounting data displayed in the accounting system is verified (checked), and as a result of verification, correction of accounting data as necessary Check the appropriateness of the accounting data (date, items, amount, transaction information, etc.) displayed in the accounting system while conducting etc.
The image management DB assigns IDs to images such as receipts, and manages them for each adviser or company, or for each organization such as head office, branch or sales office within the company, and results of character recognition of images such as receipts This is a database for correlating with feature patterns and data acquired by inputting accounting data. The contents are described in the image management table of FIG. 5-1.

As shown in FIG. 5-3, the threshold value of duplication possibility for each sheet is “high, middle, and high possibility of duplication” by the combination of the comparison result of the character recognition and the comparison result of the feature pattern of the image. In the case where it is possible to distinguish stepwise, for example, from “degree, low, almost nothing”, for example, in the case where the possibility exceeds “middle” as the threshold, for example, FIG. Present the user with images such as receipts that may overlap.

The threshold value of overlap possibility for determining a plurality of sheets collectively is, as shown in FIG. 5-4, a whole of the past as to an image such as a read receipt when the plurality of sheets are read collectively or continuously. A threshold value for determining whether there is a high possibility of overlapping with an image such as a receipt registered, for example, the possibility of overlapping among 10 images newly read such as a receipt If there are two or more cards with medium or higher, such should be rare, so if this "two or more (or 20% or more, etc.)" is the threshold, and this threshold is exceeded, the whole It is determined that there is a high possibility of duplication.

When it is misjudged as overlapping even though they are not actually overlapping, the number of digits such as the amount of money or the date matches when the character recognition result such as date or amount of money is misrecognized There are cases where it is determined that the feature patterns of the images are the same. Specifically, if you use the same taxi company on the go, repeat the move, and if the number of digits in the amount is the same, or if you purchase the same item multiple times in the same store, such as date, amount, store name, etc. If the character recognition result is misrecognized due to the similarity of the feature patterns of the images, it is determined that there is a possibility of overlapping.
In addition to the case where print condition is bad and characters and shop name are faint when character recognition results of images such as receipts can not be acquired or misrecognized, paper jams occur with the scanner and paper such as receipts In some cases, it may break, wrinkle or break.
In order to accurately determine the presence or absence of duplication in such a usage environment, it is also possible to adjust the threshold of the duplication possibility for each sheet and the threshold of the duplication possibility for determining a plurality of sheets collectively (FIG. 3A) And FIG. 3C described later).

[Configuration of One Embodiment of Accounting System]
FIG. 1A is an example showing a set of the terminal device 10 and the image capturing device 50 of the present invention, and the image capturing device 50 may be a camera configured in the terminal device 10 such as a mobile device. Good.
In this aspect, it is mainly assumed that staff of accounting firm or accounting personnel such as companies input accounting data using a terminal device as a stand-alone.
For example, when receiving receipts of a staff adviser company of an accounting firm and entering accounting data, or when bringing a terminal device (note type or mobile) to the adviser company and entering accounting data at an adviser site, This is a case where a person in charge of accounting in a company inputs accounting data, such as receipts and slips generated in their own transactions.

In FIG. 1-2, a set of the terminal device 10 and the image capturing device 50 (see FIG. 1-1) of the present invention are mutually connected via the LAN 60 to share an image such as a read receipt etc. It is a figure which shows a case.
In FIGS. 1-3, a set of the terminal device 10 and the image capturing device 50 of the present invention is connected to the server 100 via the LAN 60, and manages an image such as a read receipt etc. in the storage unit of the server 100. It is a figure which shows a case. The server 100 includes a virtual server and the like in addition to a real server.
In the case of FIG. 1-3, the terminal device 10 reads receipts and the like while being connected to the LAN 60 and the server 100, separates from the LAN 60 and the server 100, and brings it to a consulting company or branch office to read receipts and the like. Suppose that there is something.
The image capturing apparatus 50 may be connected to the server 100 via an interface such as a wireless LAN or WAN to upload an image such as a receipt.

1-4, a set of the terminal device 10 and the image capturing device 50 according to the present invention is connected to the server 100 at the end of a wide area network (WAN) 70 via a LAN or a router 60, and the receipt is read. And the like are managed in the storage unit of the server 100. Alternatively, the terminal device 10 and the image capturing device 50 may be connected to the server 100 to determine whether an image such as a read receipt is duplicated and to register an image such as a receipt.
In this configuration, in the case where the LAN system configuration side is not the accounting office system, but the accounting office staff is visiting the adviser company, the terminal device 10 brought in to the server 100 on the WAN 70 side. By connecting, an image such as a receipt can be managed in the storage unit of the server 100.
Although not illustrated, the terminal device 10 in the LAN system may be further connected to a server device, and may be connected to the server 100 located beyond the network (WAN 70) via the server device.

Alternatively, the side of the LAN system configuration may be a system of an adviser company, which may read a receipt or the like and register an image such as the receipt or the like in the server 100 of the accounting office ahead of the WAN 70. Besides, the LAN system configuration may be a system of a home worker, an outsourcing company, or each organization in a company.
The server 100 may be a server in an accounting office, may be a real server of a cloud or an IDC (Internet data center), or may be a virtual server. In this case, the staff of the accounting office can use the terminal device 10 connected to the server 100 to input accounting data while displaying an image such as a receipt sent from the adviser company.

Alternatively, the side of the LAN system configuration may be a system such as a branch office of a company, and the server 100 ahead of the WAN 70 may be a server installed in a head office or the like. In this case, a person in charge at a branch office or the like reads a receipt or the like by the terminal device 10 and stores it in the storage unit of the server 100. Then, a person in charge of accounting, such as the head office of a company, inputs accounting data using the terminal device 10 connected to the server 100 on the head office side.

FIGS. 1-5 is a figure which shows the system configuration at the time of exchanging images, such as a receipt, via WAN70 between an accounting office and an adviser. In this case, reading an image such as a receipt and storing it in the storage unit of the server 100 separates the terminal device 10 of the accounting office from the server 100 in addition to the case of the terminal device 10 of the advisor. After bringing it in and reading it, after returning to the accounting office, it may be connected to the server 100 again and stored in the storage unit of the server 100.

FIGS. 1-6 is a figure which shows the system configuration | structure at the time of exchanging images, such as a receipt, via WAN70 between an accounting office and a home worker. The home worker may be an external contractor or a sole proprietor in addition to the staff of the accounting office. In the case of an external trader or an individual business owner, the accounting office temporarily stores the receipt image etc. for each adviser in the server without making a duplicate judgment, and stores the duplicate judgment and the accounting data input at the terminal of the home worker Do with

FIG. 1-7 is a diagram showing a system configuration when exchanging an image such as a receipt between the head office (head office) of a company and a branch office via the WAN 70. In this case, reading an image such as a receipt and storing it in the storage unit of the server 100 separates the terminal device 10 of the head office (head office) from the server 100, in addition to the terminal device 10 such as a branch After bringing it in and reading it, after returning to the main store, it may be connected to the server 100 again and stored in the storage unit of the server 100.

FIG. 2A is a functional block diagram of the terminal device 10.
In the terminal device 10, an image such as a receipt is read by the image capturing device 50 connected via the communication unit 16 or the image capturing unit 12 such as a built-in camera, and extracted by the OCR processing unit of the control unit 14. The character recognition process is performed on the text character, and the feature pattern extraction unit of the image extracts the feature pattern of the image for each area such as a receipt.
Then, the character recognition result obtained by the character recognition processing and the feature pattern of the image obtained by the feature pattern extraction unit of the image are associated with the image such as a receipt and registered in the storage unit 15, and the overlap possibility determination unit Compared with character recognition results of images such as receipts captured in the past and feature patterns of images, it is determined whether an image such as a newly read receipt overlaps an image such as a receipt registered in the past Then, the image is presented to the user via the output unit 13 by the display image control unit.

In each processing unit that executes each function of the control unit, a CPU (central processing unit) loads a program stored in a storage unit such as an HDD into a main memory configured by DRAM, SRAM, etc. (Central processing unit) is implemented by executing each step of the program.

FIG. 2B is a functional block diagram of the server 100.
The server 100 may be a real server or a virtual server or other cloud.
The server 100 receives an image such as a receipt read by the terminal device 10 via the communication unit 160, stores the image in the storage unit 150, and the OCR processing unit of the control unit 140 according to the operation of the terminal device 10. The character recognition process is performed on the text characters extracted in the above, and the feature pattern extraction unit of the image extracts the feature pattern of the image for each area such as a receipt.
Then, the character recognition result obtained in the character recognition process and the feature pattern of the image obtained by the feature pattern extraction unit of the image are associated with the image such as a receipt and registered in the storage unit 150, and the overlap possibility determination unit Compared with character recognition results of images such as receipts captured in the past and feature patterns of images, it is determined whether an image such as a newly read receipt overlaps an image such as a receipt registered in the past The determination result of the possibility of duplication is stored in the storage unit 150.
The determination result of the possibility of duplication is displayed on the display unit of the terminal device 10 via the network of the WAN 70 or the LAN 60.


[Overall processing flow]

FIG. 3A is a flowchart showing the entire processing contents of the accounting system of the present invention.
Images such as receipts are managed for each adviser for the accounting office or for each branch for the head office, and a company master is prepared for the management.
The following description will be mainly made on the assumption that the accounting firm performs accounting special services for the adviser, but the same is true for the relationship between the head office and the branch office.

In FIG. 3A, first, an adviser to be processed is selected (step S3A-1), and a predetermined number of receipts and the like are read (step S3A-2).
After reading a predetermined number of receipts and the like (step S3A-3), character recognition processing (OCR processing) for grasping the character string printed on the receipt etc as text is performed, and the printed logo is displayed. Etc. Extract the store name such as, item name, quantity, amount of money, image feature pattern such as date, as a bitmap image or coordinates of start point and end point, position or range information of rectangular area (hereinafter, feature pattern of image) Processing is performed (step S3A-4).

The reading process of an image such as a receipt from step S3A-2 to step S3A-3 and the character recognition process such as text of step S3A-4 and the extraction process of a feature pattern of an image may be performed independently of each other. The reading of an image such as a receipt, registration processing of character recognition results and feature patterns of extracted images, and overlap judgment processing may be performed continuously, and an image such as a receipt or character recognition result and an extracted image The overlap determination process may be performed after the registration process of the feature pattern of.
The state of the setting screen of whether or not to apply this duplication determination process is shown in FIG. 6-5.
According to FIG. 6-5, when performing the duplication determination process whether to set the type (built-in camera, external scanner, etc.) of the image capture device 50 used in the terminal device 10, and whether to perform the duplication determination process. Setting “read now” or “read later (separately)” temporarily reading an image such as a receipt and temporarily storing it in a storage unit. it can.
If it is set to "do later (separately) do the setting of the overlap determination processing", a flag is set in the image management table of FIG. At the time, a duplicate determination process is automatically executed with the flag, or a screen is displayed to urge execution of the duplicate determination process (not shown).

As shown in FIG. 7, as one specific example of the feature pattern of the image, logo areas such as company name, store name and phone number, person in charge, date of transaction, item name, quantity, in order from the top of the receipt etc. The amount of money, the bar code, and the like are printed for each paragraph or region, respectively, as bitmap image, coordinates of start point and end point, and information of position and range of rectangular region. The contents of the transaction include company name, telephone number, person in charge, date of transaction, item name, quantity, amount and store name, as well as other department signatures, supplier names, and names of branch offices and sales offices.
Here, for example, the feature pattern of the image of the date and the amount of money has a feature in which the number of digits of the date and the amount of money is reflected in the width, and in the comparison of the feature pattern of the image Can be used as a judgment index as to whether the contents of the transaction are the same.
As a feature pattern of an image, there is a feature pattern obtained by grasping the whole of a receipt or the like in addition to a feature pattern of a partial image for each region, and an example is shown on the right side of FIG. It shows.

Here, the extraction process of the feature pattern of the image may be performed after grasping the store name such as the logo, and the attribute such as the item name, the quantity, the amount, the price, the date, etc. In addition to the rough grasp of what the upper row is what, and the middle row what, without grasping the attribute of the individual, it is an individual's grasping method of information useful for accounting processing of any store name or item name etc. The feature pattern of the image may be extracted regardless of the attribute of the feature pattern of the image.
Ideally, it is possible to accurately identify and identify shop names such as individual logos, item names, quantities, amounts, amounts of money, dates, and other attributes, but it is ideally possible to make duplicate determinations, but character recognition failure or character recognition due to character blur etc. In the present invention, the character recognition fails or the character recognition accuracy is low, and the shop name, the item name, the quantity, the amount, the date, etc. is low. This is intended to expand the range in which duplicate determination can be performed, assuming a case where the attribute can not be grasped.
In addition, color information can also be used as the feature pattern of the image.

The feature pattern of the image is not limited to the one exemplified in the image management table described above and in FIG. 5-1, and any feature pattern may be used as long as it represents the feature of the image. Of course, the external size of a receipt etc. is also included. The external size increases correspondingly as the number of products increases, and a unique size is adopted for each store and customer, so that the features are well expressed.
In addition, any one or more of the illustrated feature patterns may be used as the feature pattern of the image, and the accuracy of the determination is improved by combining a plurality.

When the character recognition process of step S3A-4 in FIG. 3A and the extraction process of the feature pattern of the image are finished, a subroutine of overlap determination process with an image such as a receipt read in the past is called and executed (step S3A-5).
Details of the subroutine of the overlap determination process will be described later.

When the duplication determination processing is completed for the predetermined number of receipts and the like read, the user is presented with an item whose possibility of duplication exceeds a certain threshold, and the read receipt and other images are displayed in the storage unit. A screen is displayed asking for an operation of officially registering or deleting (step S3A-6).
The features of the confirmation display will be described later.

As a result of confirmation by the user, when the image is not duplicated (step S3A-7: no duplication), an image such as a newly read receipt is associated with the information of the recognition result and the feature pattern of the image and formally stored in the storage unit. Registration (learning) is performed (step S3A-8).
Thereafter, an image such as a receipt registered in the storage unit is managed as a receipt image read in the past, and used at the time of overlap determination.
On the other hand, when it is determined that the image is duplicated (step S3A-7: presence of duplication), a process of deleting the image such as the newly read receipt is performed by the user operation, and the result of the duplication determination is It registers in a memory | storage part (step S3 A-9).

When the process of determination of duplication of images such as receipts in step S3A-9 of FIG. 3A is completed, processing of accounting data input is performed (step S3A-10).
In the accounting data input process, for example, an application for inputting journal data is activated, and a cashier application for inputting cash transactions is activated and performed (step S3A-10).
Among the input information, a store name dictionary, a product name dictionary, and the like are further registered in advance for the summary information, the store name and product name information is extracted from the summary information, and the character recognition result and the image feature Learning, registration and reuse can be associated with pattern information (step S3A-11).

For example, by linking the store name with other information in the image management table (FIG. 5-1) and sorting by store, the image management table can be used as a dictionary to perform image management for each store, In some stores, it is also possible to associate items such as daily necessities and certain stores with food and beverage expenses, and if you try to enter accounts different from these items, you will be given a notice or automatically suggested recommended items. be able to.
Also, in the image management table (Fig. 5-1), since images such as receipts are linked with accounting data such as journals, recommended items are presented, and store names and subjects such as receipts Relevant journal data can also be presented.
In addition, with respect to an image such as a receipt, an image such as a related receipt can be retrieved and presented. In the search process at that time, the user selects a search item or accepts an input using a UI (user interface) for inputting a search condition, or an item, an abstract, a shop name, or the like associated with an image such as a receipt. The registered receipt image is searched based on the information of.
Alternatively, images such as receipts may be sorted in the order of many matching items by character recognition results such as registered receipts or comparison of feature patterns of images, and may be listed in order of many matching items. .

As shown in FIG. 7, the character recognition result and the feature pattern (part) of the image for each area obtained from the image of the receipt etc. through the character recognition process and the feature pattern extraction process of the image are finally account data It is registered in the image management table (FIG. 5-1) in association with data (date, amount of money, abstract, store name, item name, account item) acquired by input and the like.

[Drug Judgment Processing Subroutine with Registered Receipts etc Imported in the Past]

FIG. 3B is a diagram showing a processing flow of an overlap determination processing subroutine with registered receipts and the like fetched in the past, in which the processing contents of step S3A-5 of the entire processing flow of FIG. 3A are described in detail.
In this subroutine, comparison processing using text characters such as date and amount acquired in character recognition processing and comparison processing using feature pattern information of images are combined, and an image such as a newly read receipt etc. is in the past Processing to determine the possibility of overlapping with an image such as a receipt read in
Here, "I" described here is the result of character recognition "Available (result could be obtained)" and the comparison result of character recognition "match", "J" the result of character recognition "Available (result acquired And the character recognition comparison result is “partially non-matching (partially matching)”, “K” is character recognition result “none (cannot obtain results)”, and “L” is a feature pattern of the image The comparison result is “match” and the comparison result of the whole image is “match”, “M” is the comparison result of the feature pattern of the image is “match”, the comparison result of the whole image is “mismatch”, and “N” is the feature of the image It is a code simply attached to show the situation of each comparison result that the pattern is "mismatch".

In FIG. 3B, first, in order to speed up the process as necessary, the text character of the character recognition result of the image such as a receipt read in the past and the information of the feature pattern of the image are loaded into the main memory (step S3B-1) may be compared with those stored in HDD, SSD, flash memory, and other long-term storage media.

Images such as receipts referred to as duplication sources are those managed for each company ID (that is, for each adviser, each department such as a branch of a company) as in the image management table of FIG. 5-1.
In addition, as the time range, it is the user setting or system default such as “from the registered receipt etc. of the transaction on the first day of the fiscal year” or “from the registered receipt etc. of the transaction on the first day of this month”. The set range is considered.
As the default setting of the system, for example, in the case of closing on the first day of the fiscal year, for example, at the end of March, it may be considered that the receipt range after April 1 is set as the target range.
Alternatively, if the management of receipts etc. is strict in companies such as advisors, and there is no possibility of mixing up until the previous month, the scope of images such as receipts from the first day of the current month by user setting It may be possible to change to the setting to be. By narrowing the target range, it is possible to speed up the process of the overlap determination.
These settings are reflected and stored in FIG. 5-5 (company master) and the like.

Next, the process proceeds to duplication determination processing, but when an image such as a receipt from the first day of the current period is included in the target range, if there are 1,000 receipts or other images per month per company, December Since the number of images such as receipts to be referred to is nine months from April to December, that is, 9,000 sheets, it is necessary to devise the processing content and processing order of the duplication determination processing.
In order to speed up this process, for example, the comparison process of the character recognition result and the comparison process of the feature pattern of the image are activated as parallel processes (step S3B-2).
Then, in the comparison process of the character recognition result, it is confirmed whether the character recognition result of the image such as a newly read receipt can be acquired (step S3B-3), and the character recognition result can not be acquired in the first place due to blurring or the like. (“NO” in step S3B-3), the status of the comparison result is recorded as “K” (step S3B-7), and the character recognition result can be obtained (“YES” in step S3B-3) ), Compare the text characters, and if all match, partial match (partial match), etc., the status of each character recognition result is “I”, Record as “J”,... (Steps S3B-4, S3B-7).

Further, as a process parallel to the character recognition process from step S3B-2 in FIG. 3B, a process of comparing feature patterns of images is performed (steps S3B-5, S3B-6).
Specifically, the feature patterns of each area of the image are compared with each other (step S3B-5), and when some feature patterns of the image contain a mismatch ("No match" in step S3B-5), the comparison is performed. The resulting status is recorded as "N" (step S3B-8), and if all the feature patterns of each area of the image match ("match" in step S3B-5), the image of the entire image is The feature patterns are compared with each other (step S3B-6), and if they match ("match" in step S3B-6), the status of the comparison result is recorded as "L" (step S3B-8), If they do not match (“mismatch” in step S3B-6), the status of the comparison result is recorded as “M” (step S3B-8).

In this case, in the comparison processing of the feature patterns of the image, the order of the comparison of each area or the entire image may be reversed.
Further, although an example in which parallel processing is performed is described in FIG. 3B of the embodiment of the present application, comparison processing of character recognition results and comparison processing of feature patterns of images may be processing in which either one precedes or follows the other.
Then, synchronization processing such as adjusting timing is performed so that both the comparison processing of the character recognition result and the comparison processing of the feature pattern of the image are completed (step S3B-9).
It should be noted that comparison processing of character recognition results is relatively light processing because it is comparison of text characters, while comparison processing of feature patterns of images is processing that is relatively heavy. The comparison processing is performed first to narrow down as much as possible the images such as registered receipts that may have duplicates, and thereafter the processing of comparing the feature patterns of the images is performed to speed up the processing time. It is good.

Although the character recognition result does not match due to the process combining the comparison process using text characters such as date and amount acquired in the above character recognition process and the comparison process using information of the feature pattern of the image, the feature pattern of the image The comparison results of various aspects are included, including the case where the comparison of the two matches (for example, the amount of money matches but the date is the case where the characters are complicated and misrecognized). obtain.
Based on such comparison results, a comprehensive judgment process is performed to comprehensively judge the possibility that the image of the newly read receipt etc. is overlapped with the image of the receipt read in the past Step S3B-10) The determination result is stored in the main storage or a storage unit such as an HDD (step S3B-11).

The judgment result for each sheet is read out and used in the judgment of a plurality of sheets collectively in step S3B-13, and used when displaying on the duplication confirmation screen (FIGS. 6-1 to 6-4) such as duplicate receipts. Be
Note that the comparison processing of character recognition results and the comparison processing of feature patterns of images tend to have a larger processing amount in the comparison processing of feature patterns of images, and either processing is performed in parallel It can be determined by making a determination as to whether or not to perform first.

Although the comprehensive judgment processing for each sheet can be performed using the overlapping comprehensive judgment table as shown in FIG. 5-3, the present invention is not limited to this, and can be carried out in the form of a combined processing flow of cases. .
In the overlap total determination table of FIG. 5-3, the character recognition results match (the state of the comparison result of the character recognition is I), and the comparison result of the feature patterns of the image is also the feature pattern of the partial image of each area. If the comparison results also match, and if the comparison results of the feature patterns of the image in the entire image also match (the status of the comparison results of the feature patterns of the image is L), it is highly likely that they overlap. It is judged.

On the other hand, when the character recognition result is in agreement (the state of the comparison result is I), and in the comparison result of the feature patterns of the image, the non-matching region is included in the comparison result of the feature patterns of partial images for each region. (The situation of the comparison result of the feature pattern of the image is N), and it is presumed that the number of digits such as date and money is different, so it is presumed that the character recognition is wrong recognition etc. There is a possibility of overlapping It is determined to be low.

Similarly, even when the character recognition result can not be acquired (the character recognition status is K), the comparison results of the feature patterns of the images indicate that the feature patterns of partial images of each region match each other and the features of the entire image If the patterns match, it is assumed that the date and the number of digits of the amount are also the same, and it is assumed that the item name and the number etc. are also the same, and furthermore, the issued stores are also presumed to be the same. It is presumed that the character recognition did not work well by chance etc., and it is judged that the possibility of duplication is medium, and leak of duplication judgment is prevented, and adjustment is made so that it can be presented to the user just in case To judge.
Then, with respect to all the predetermined number of receipts and the like read this time, the respective duplication possibilities are determined (step S3B-12 in FIG. 3B).

When the process of determining the possibility of duplication for each sheet is completed, the process of determining whether a plurality of sheets are combined is performed (step S3B-13).
Here, processing is performed using the overlapping comprehensive determination table (for determination of a plurality of sheets collectively) shown in FIG. 5-4 to be described later. In the case where medium or higher duplication possibility is included, the series of receipts etc. show that there is a high possibility of duplication with the image etc. of receipts etc. read in the past. In the case of 1 sheet 1 sheet duplicate judgment, it is complemented that there is a case where the duplication possibility may not necessarily be judged with high accuracy, comprehensively, the receipt newly read this time is insufficiently managed, and the past Information can be presented to the user in a timely manner that there is a high possibility of duplication with the receipt etc. read by the user, and the user interrupts reading of the remaining dozens and hundreds of receipts etc. Remaining receipts etc. contained It can be re-confirmation, in that it is not necessary to wasteful read work, is a practice useful.

In the case of perfect coincidence with a past read image of a receipt etc. by duplication judgment of a plurality of sheets collectively, the transaction content is in the case where the user only rereads the same receipt etc. when only partial matches. It may be determined that a new receipt or the like including the same receipt or the like has been read.
When the above process is completed, the subroutine is ended, and the process proceeds to the confirmation display of step S3A-6 of the entire processing flow.

[Modification example of duplication judgment processing subroutine with registered receipts etc. taken in the past]
A modified example of the overlap determination processing subroutine with registered receipts and the like fetched in the past will be described with reference to FIGS. 3C, 3D and 3E.
The difference from FIG. 3B is that the comparison processing of character recognition results and the comparison processing of feature patterns of the image are respectively made into subroutines as in FIG. 3D and FIG. 3E, the comparison processing of character recognition results and the comparison processing of feature patterns of the image For each of the above, processing such as looping a plurality of times is added by changing parameters such as a threshold value of overlap determination for comparison.

In addition, there is a certain tendency that receipts etc. to be issued have small size of characters or something like logos and figures depending on customers and stores, etc. It is compatible with feature pattern comparison processing.
Therefore, by making the comparison process of character recognition result and the comparison process of feature pattern of the image mutually go back and forth, it is necessary to narrow down the image such as receipt matching in the comparison process of the character recognition result or the feature pattern of the image Multiple processes such as narrowing down images such as receipts that match in the comparison process, or operating both comparison processes in parallel and merging both comparison processes and then narrowing down instead of narrowing down with one or the other By running a pattern, it is possible to select a comparison method that is respectively compatible with each other, to weight each one, and to optimize the weight value by learning.

In FIG. 3C, for example, when the printed character is large, the image is clear, and the reliability of the character recognition result is high, first, the comparison processing of the character recognition result is performed in step S3C-3, and the receipt read in the past is received. Etc. are narrowed down to a certain extent, and then the feature pattern comparison processing of the image is performed in step S3C-4 for the suspected duplication in the comparison of the character recognition results, and further parameters such as threshold values Can be changed (step S3C-5), and the comparison process of the character recognition result and the comparison process of the feature pattern of the image can be performed again.
Alternatively, if the characters printed are small or there is a lot of blurring or the like in the image of the receipt, etc., the reliability of the character recognition result is low, so first the process of comparing the feature patterns of the image is executed to Pick up ones that have similar feature patterns, execute comparison processing of character recognition results on that later, narrow down further, and change parameters such as threshold (step S3C-5), compare character recognition results The processing and the comparison processing of the feature patterns of the image can also be performed again.

In FIG. 3B, in the overlap determination process of B and B, and the overlap determination process of combining C and B, (1) default pattern in FIG. 5-3 or customized for each adviser or company type It is also possible to change each threshold (parameter) using the determination table and repeat each duplication determination process.
Synchronous processing is performed so that each overlap determination process is completed (step S3C-7), and four overlap determination results (for each sheet) are obtained as the determination results of the overlap determination process. Be
Note that the results of c and d are applied to the overlap determination table of FIG. 5-5, and output the possibility of overlap obtained by combining the comparison of the character recognition result and the comparison result of the feature pattern of the image.
Each judgment result is output with the possibility of duplication being large, medium, small, etc. In addition, 0.9 (90%) in the case of large possibility of duplication, 0.6 (60%) in the case of medium In the case of, numerical values such as overlapping probabilities such as 0.2 (20%) may be used.

Then, after four duplicate determination results (one sheet) are obtained for B, B, C, and C, of these, each of the results of the duplication determination process is weighted according to the compatibility, A comprehensive determination of the possibility of duplication is performed (S3C-8).
The comprehensive judgment is applied by being applied to FIG. 5-6 overlapping comprehensive judgment table (for judgment for each sheet) (for FIG. 3C) and output.
At this time, weighting (numerical values of a, b, c) on each determination result can be performed for each adviser or for each type of business of the company, and optimization can be made to overlap determination according to each type of business.
It should be noted that it is often the case that the client is in a certain range for each adviser or for each industry, adjusting the weighting for each adviser or for each industry of the company, and as a result, the receipt etc. Since a tendency will occur, in the overlap determination process for images such as receipts for each adviser or for each type of industry, the order of the process for determining the character recognition result and the process for comparing the feature patterns of the image and the threshold (parameter) This is because compatibility occurs with the setting.

In addition, comparison processing of character recognition results is relatively light processing because it is comparison of text characters, while comparison processing of feature patterns of images is relatively heavy processing, so at system initialization, character recognition results The comparison process is performed first to narrow down as much as possible the images such as receipts that may be duplicated, and then the process of comparing the feature patterns of the images (that is, the process to “I” in FIG. 3C) is weighted. The processing time may be increased by making the processing time higher (than the processing up to “b” in FIG. 3C in which comparison of feature patterns of images is performed first).

  Although the overlap determination table of FIGS. 5-6 shows the example prepared for every determination pattern like determination pattern 1, 2, 3 ..., the weighting (a, b, c) of each determination pattern is If the possibility of duplication is a certain level or more (for example, medium (0.6) or more), the user is presented to confirm whether or not duplication actually occurs, and the confirmation result is obtained to obtain duplication. Convert the error of the judgment result of the possibility and the confirmation result of the user by the error function such as the root mean square error, and weight value of each judgment pattern (a, b, c) so that the error function is minimized. Machine learning can be performed to update the accuracy of the determination pattern.

Mobile processing flow
In the case where the server 100 of the accounting office and the terminal device 10 of the staff constitute a client server system as in the case of FIG. 1-5 etc., the terminal device 10 is separated from the server 100 and brought to the adviser. Then, when reading an image such as a receipt at the advisor, since the terminal device 10 is carried at the advisor, it will be called portable processing.
The processing flow of such portable processing is shown in FIG.
In FIG. 4, after determining the target adviser (step S4T-1), the data of the adviser bringing in the terminal device 10 (image data such as receipt, character recognition result associated with it, feature pattern of the image Of the accounting office etc. (step S4SV-1) from the accounting office server 10 (step S4SV-1), temporarily store it in the storage unit of the terminal device 10 (step S4T-2), and prepare for duplicate judgment at the adviser on the go Can.

Then, in this case, it is possible to lock all the data of the adviser to be processed and perform a process to make it impossible to add or update any data of the adviser (step S4SV-2). “Lock”), do not lock, another staff member of the accounting office uses another terminal 10 to continue the process of capturing an image such as a receipt, or the adviser can directly directly The server 100 may be accessed to upload an image such as a receipt (“do not lock” in step S4SV-2).

<Common processing when locking and not locking the adviser's data on the server side>
In FIG. 4, an image such as a receipt is read by the terminal device 10 brought to the advisor (step S4T-3), and the image or the character recognition result associated with the image stored in the storage unit of the terminal device 10 The comparison with the feature pattern of the image is performed, and the image such as a receipt is taken in while performing the overlap determination process at the advisor (step S4T-4).

<When locking data of adviser on server side>
Then, when the data of the adviser is locked in the server 100 of the accounting office, after returning to the accounting office, the server 100 is connected again, and the receipt read etc. An image is additionally registered (step S4T-8, step S4SV-5).

<When not locking the adviser's data on the server side>
On the other hand, when the data of the adviser is not locked in the server 100 of the accounting firm, the overlap judgment with the image such as a receipt added from another staff member or the adviser during the mobile processing is further performed ( In step S4T-6), additional registration is performed on the non-overlapping ones (step S4T-7, step S4SV-4).

[table]
Fig. 5-1 is an image management table, in which an image ID is assigned to an image such as a receipt that has been read for each adviser or company, or for each organization such as a head office, branch or sales office within the company. Information about the result of character recognition of images such as receipts and feature patterns of images is related to the feature patterns of partial images for each area or information of feature patterns of the entire image and the image of the receipt such as the user Store names, business partners' names, item names, and information on notes and accounts associated with the contents of transactions as accounting data such as journal data are registered and managed in association with one another.
Here, for example, company ID K0001 indicates company B (adviser B).
The feature pattern of the entire image includes the feature pattern of the size of the outer frame of the image such as a receipt as well as the feature pattern of the image of the entire printing area.
Also, in the reading setting screen of FIG. 6-5, it is determined whether or not to perform the duplicate determination process, a flag for storing the setting of whether to perform now or later, and whether the duplicate determination process has been executed. It has the flag of.

Here, the feature pattern (abbreviated as feature PTN) in FIG. 5-1 is projected from the X and Y directions as well as the width and height of the rectangle of the region with dots, bitmap, and the histogram format. Etc.), vectorized information by mathematical processing, and the like.
Since the feature PTN is an image relating to some accounting, it has accounting attributes (number, item name, date, amount of money, etc.) of each area, and therefore position, range, width, height, etc. have important meanings.
The feature pattern (part) is the feature pattern of the image for each region, and (of feature 1 (feature PTN such as store name), feature 2 (position, feature PTN), feature 3 (position, feature PTN, ...) It can be written as

FIG. 5-2 is data generated using an accounting input application or the like, and is a diagram showing the data configuration of accounting data such as journal data, and a journal data ID etc. is assigned to each adviser, The information such as the description, the amount of money, the date, and the like are associated with each other, and further, the image management table associates which accounting data regarding the image such as a receipt is associated with.

Figure 5-3. Duplicate comprehensive judgment table (for each judgment) and FIG. 5-4. The duplicate comprehensive determination table (for determination of a plurality of sheets collectively) has been described above, and thus details thereof will be omitted. In the Overlap Comprehensive Judgment Table (for multiple judgments collectively), it is described as two or more of the readings as the judgment condition, but in the case where two or more sheets overlap, or at a ratio of 20% or more It can replace with a certain threshold, such as overlapping.

In short, if 10 sheets are read and there is a possibility that 1 sheet is duplicated, the 10 sheets are insufficiently managed in the range of 1 week ago, the previous month, the same month last year, the same store, etc. It can be said that the possibility of overlapping with an image such as a receipt read in the past is low, but when it is judged that the possibility of overlapping two or three sheets is high, it is not accidental but whole. There is a high possibility that images of 10 sheets and a large number of receipts etc. which are going to be read subsequently are overlapped with images of receipts etc. which are typically read in the past, and it is quick for the user, images of all receipts etc. Before loading, you can perform operations such as checking carefully, and you can do lean accounting.

[Features of screen display when it is determined that there is a possibility of overlapping]
Next, a description will be given of a confirmation screen such as a receipt when it is determined in step S3A-6 of the overall processing flow chart (FIG. 3A) that the possibility of overlapping is greater than or equal to a certain level (possibility is middle or higher). Do.
FIG. 6A is a confirmation screen (left-right comparison) of receipts and the like when it is determined that the possibility of overlapping is higher than or equal to a certain level (possibly middle or higher). In this screen example, among the images such as receipts newly read this time on the right are listed three sheets that may overlap with the images such as receipts read in the past, and among them, the receipt number is ID0065. When the item is selected, the image of the receipt etc. is displayed, and at the same time, the image of the duplicate original receipt etc., which may be duplicated, is displayed on the screen on the left side. It shows how it is.
In this screen example, it is possible to check while comparing images such as receipts that may overlap left and right, so work efficiency is improved.
Note that the screen to check the left / right contrast is an example, and if you compare the top and bottom, or display only the image of the newly read receipt etc. and focus the mouse cursor etc. on the image of the receipt etc, registered in the past read It is also possible to pop-up an image such as a receipt having a possibility of overlapping (the same applies to FIGS. 6-2 and 6-3).

6-2 shows any receipt image for the confirmation screen (left-right contrast) such as a receipt when it is determined that the overlapping possibility in FIG. 6-1 is determined to have a certain level or higher (possibly middle or higher). When is displayed in an enlarged manner, the other image is displayed at the same enlargement ratio, and when one image is scrolled, the other image is similarly automatically scrolled. By applying the value of the scroll amount of the image to the other, the same portion can be enlarged and confirmed, and it is supported to reliably confirm whether or not the image is duplicated.

FIG. 6C is an example of the case where there are a plurality of captured receipts and the like that may be duplicated on the confirmation screen of the duplicate receipts and the like.
FIGS. 6-1 and 6-2 illustrate the case where there is one image such as an already-captured receipt which may be overlapped with one image of a newly-read receipt or the like. On the other hand, in FIG. 6C, there is a difference in that there is a case where there are a plurality of images such as already-captured receipts which may be overlapped with one image of the newly read-in receipt and the like.
In FIG. 6-3, there is a possibility of overlapping when there are two images such as already acquired imported original receipts which may be overlapped with one image of the newly read receipt etc. Receipt numbers are listed in descending order, and it is shown that the selected image is displayed on the left side and can be checked in contrast to the image displayed on the right side, such as a newly read receipt this time.
The level of the possibility of duplication may be a distinction such as high or medium as shown in the duplication total determination table of FIG. And the level expressed by the probability may be used.

FIG. 6-4 shows an example of a distinguishable display in the case where a plurality of sheets of a plurality of images such as receipts newly read this time overlap with an image such as a receipt read in the past.
On the right side, 5 sheets of images such as receipts newly read this time are displayed, and among them, the background is distinguished and displayed for those having a certain level of possibility of overlapping with the images such as receipts read in the past Examples are shown.
In this case, when the mouse cursor is placed on an image such as a receipt that may be overlapping on the right side, or in the case of a mobile terminal, the image such as the receipt is touched on the touch panel on the screen. On the left side of the screen, images such as already-captured duplicate original receipts which may be duplicates are successively switched and displayed, and it is possible to check whether duplicates are present or not.

And in this example, it is shown that there is a possibility that images such as two out of five receipts may overlap, and in such a case, management of receipts etc. is insufficient and it is going to be read from now on At the top of the screen, there is a possibility that the images, such as receipts, are generally overlapped in a typical manner, “This group of receipts, etc. is highly likely to be overlapped with the receipts, etc. taken earlier. Therefore, by displaying a warning such as “!”, The user immediately interrupts the reading of images such as receipts, confirms the management status of images such as receipts stored in a bag, etc., and has already taken in receipts. It can be checked whether it is the same or not, and it is possible to prevent unnecessary reading work.
Although not shown, it is also assumed that there are a plurality of imported duplicate original receipts and the like, and in this case, duplicate overlapped original receipts and the like may be displayed as shown on the left side of FIG.

Also, regardless of whether or not the above-described duplication determination processing is performed, character recognition results such as receipts and feature patterns of images, and information such as account items, notes or store names are registered, so an image such as a certain receipt On the other hand, using these pieces of information as search keys, it is possible to search for items that hit many items.
For an image such as a receipt, the information such as an account name, a note or store name is used as a search key to search for and list images such as related receipts, or an account attached to an image such as a listed receipt You can also refer to accounting data such as notes or journals.

Alternatively, for an image such as a receipt, the image of a similar receipt or the like is searched using information on character recognition results and feature patterns of the image as a search key, and the images are listed in descending order of hit items. It is also possible to refer to accounts attached to images such as receipts and accounting data such as notes or journals. In this case, either one of the comparison of the character recognition result and the feature pattern of the image can be used, or both comparison processes can be combined similarly to the overlap determination. It is possible to present data such as sorting in the order of many matching items by character recognition results or comparison of feature patterns of images, etc., listing them in descending order of matching items, or listing only 10 high-order items. .

The above search processing and enumeration display can be applied to either the case where an image such as a receipt is newly read or the case where an image such as an arbitrary receipt is retrieved from among already read and registered receipts. It is.

As described above, by combining the comparison of the character recognition result and the comparison of the feature pattern of the image to determine the possibility of duplication, even if the character recognition can not be accurately performed due to small blurring or characters, etc., The possibility of duplication can be determined with high accuracy.

In addition, when the possibility of duplication is a certain level or more, by displaying a screen that is easy to confirm to the user whether or not duplication is performed, it is possible to prevent registration of images such as duplicate receipts. Because it is possible to input the accounting data after confirming that there are two different transactions, which are images of similar receipts etc., while preventing the creation of duplicate accounting data. The accounting can be made more efficient and reliable by supporting the creation of appropriate accounting data while distinguishing the occurrence of double journaling and the occurrence of two transactions.
And, according to this system, audits by the accounting firm and the head office in the company, and audits at the service receiving side such as the adviser and the branch of the company trace back to the proof image of the receipt etc. It makes it easier to give accurate guidance above, which can lead to more efficient operations.

10 terminal 11 input unit 12 image capture unit 13 output unit (display, printing)
14 control unit 15 storage unit (main storage such as RAM, long-term storage such as SRAM and HDD)
16 communication unit 50 image capturing device (camera, scanner)
60 LAN (Local Area Network)
70 WAN (Wide Area Network, Internet etc)
100 servers (including virtual servers)
130 output unit 140 control unit 150 storage unit 160 communication unit

Claims (7)

A system for performing accounting processing with an image DB for reading a voucher image, registering a read voucher image, and retrieving a similar voucher image ,
From the read testimony possession image,
Character recognition means for character recognition of characters including date and money;
Image feature pattern extraction means for extracting, for each predetermined area, a feature pattern of an image of information useful for accounting processing including a date, an amount of money, and a store name;
Character recognition result registration means for registering the character recognition result acquired by the character recognition means in a storage unit;
Image feature pattern registration means for registering the image feature pattern extracted by the image feature pattern extraction means in the storage unit ;
A duplication degree detection unit that detects a possibility that the new voucher image and the registered voucher image overlap ;
The duplication degree detection means
Wherein at least all matching search results for the registered card possessed image character recognition result of a part matching the character classified mismatched items that the character recognition result of the new read voucher image search condition by said character recognition means Acquired as similarity of recognition result,
At least all matching search results for the characteristic pattern of the image of the registered card possessed image to search the characteristic pattern of the images extracted for the new read voucher image by the feature pattern extraction means of the image, some matches , Acquired as the similarity of the feature pattern of the image classified into unmatched items,
Prepare an overlap detection matrix with the overlap state set as a matrix element,
Setting items including at least all matching, partial matching and non-matching items at least in the item of similarity of character recognition result as a column of the matrix for detecting the degree of duplication,
As the rows of the multiplicity detection matrix, the items including at least all matching, partial matching, and non-matching at least in all the items of the similarity of the feature pattern of the image are set,
The duplication degree detection matrix is searched using a combination of the obtained similarity degree of the character recognition result and the obtained similarity degree of the feature pattern of the image as a key to detect the state of the duplication degree which is an element of the matching matrix,
If the degree of duplication detected is high,
Prevent the capture of new voucher images ,
The accounting processing system characterized in that a new voucher image is registered in the image DB unless the detected degree of duplication is high .   The storage unit includes a company master for managing for each adviser and a matrix for detecting the degree of duplication for each adviser.
  2. The accounting system according to claim 1, wherein, when an adviser is selected from the company master, the overlap is detected using a matrix for detecting the overlap of the selected adviser. Read the testimony possession image, a system for accounting,
From the read testimony possession image,
Character recognition means for recognizing characters including texts and / or amounts as texts;
Image feature patterns of information useful for accounting including date, amount and / or store name ,
Partial image feature patterns for each predetermined area,
Feature patterns of the entire print area ,
And image feature pattern extraction means for extracting a feature pattern of the image including
Character recognition result registration means for registering the character recognition result acquired by the character recognition means in a storage unit;
Image feature pattern registration means for registering the feature pattern of the image extracted by the image feature pattern extraction means in the storage unit;
For the certificate possessed image newly read,
The character recognition means recognizes characters including date and money as texts,
While extracting the feature pattern of the image by the feature pattern extraction means of the image,
From the storage unit, it reads the feature pattern of a character recognition result and image registered testimony possession image,
A character recognition result comparison means for comparing the character recognition result of the new read testimony possession character recognition result and the registered card possessed image of the image,
When comparing the characteristic pattern of the image of the new read testimony possession characteristic pattern and the registered card possessed images of the extracted image on images comparison of characteristic patterns each other partial image for each of the predetermined areas A feature pattern comparison unit of an image including content and comparison content of feature patterns of images of the entire print area ;
A comparison result by the character recognition result comparison means ;
Combining the comparison content of the feature patterns of the partial images of each predetermined area, which is the comparison result by the feature pattern comparison means of the image, with the comparison content of the feature patterns of the entire print area ;
Presence or absence of a character recognition result by the character recognition unit and / or a matching degree of a comparison result by the character recognition result comparison unit with respect to a partial image of each predetermined area;
The matching degree of the comparison result of the feature patterns of the entire print area by the feature pattern comparison means of the image and the matching degree of the comparison result of the partial image of each area ,
The negative comparison result by the character recognition result comparison means is evaluated low, or the negative comparison result by the feature pattern comparison means of the image is prioritized, or the positive comparison result by the feature pattern comparison means of the image is prioritized. Including complementary duplication determination means,
The a new testimony possession image, wherein the registered testimony possessed images, overlap possibility determining means for determining the possibility of the overlapping,
By having
Accounting system characterized by preventing new testimony possession of the image capture it is determined that there may overlap by the overlap possibility determining means. The duplication possibility determination means according to claim 3,
In the comparison result by the character recognition result comparing means, even if a part of the character recognition result does not match,
In the comparison result by the feature pattern comparison means of the image, when the comparison result of the feature pattern of the entire print area and the feature pattern of the partial image for each area is identical,
Evaluating the comparison result by the character recognition result comparison means low in consideration of the possibility of partial failure of the character recognition process,
First duplication determining means for determining that the possibility of duplication is medium;
In the comparison result by the character recognition result comparison means, even when the character recognition result matches,
In the comparison result by the feature pattern comparison means of the image, when the comparison result of the feature pattern of the partial image for each region includes an unmatched region,
A second overlap determination unit that prioritizes the comparison result by the feature pattern comparison unit of the image, and determines that the possibility of overlapping is low, by inferring that the number of digits such as date or money is different;
Even if the character recognition result can not be obtained by the character recognition means,
In the comparison result by the feature pattern comparison means of the image, when the feature pattern of the partial image for each area and the feature pattern of the image of the whole print area coincide with each other,
Prioritizing the comparison result by the feature pattern comparison means of the image, it is presumed that the date or the number of digits, the item name, the number, or the issued store is also the same, and it is presumed that accidental character recognition failed A third duplication determining means for determining the possibility of duplication as medium;
An accounting system characterized by comprising any one or more of the above. When reading the testimony possessed images plurality collectively, together determine one single Duplicatability testament possession image for collection of the plurality of testimony possession image, the registered testimony possession image, day Determining and outputting the possibility of overlapping with a predetermined unit group including a unit, a week unit,
The accounting system according to any one of claims 1 to 4 , characterized in that Read the testimony possession image, the contents of transactions involving the witness possession image, receives an input as accounting data, among the acquired accounting data, associated with at least summary information or subject information, the feature pattern of the image of the witness possessed images register Te, the time of input of the next accounting data, in addition to said comparison processing by the comparison means, wherein the pattern of the character recognition result of the comparison means and the image, or summary information associated with the certificate possessed images To compare the subject information with the input summary information or subject to improve the accuracy of the accounting data entry,
The accounting system according to any one of claims 1 to 5 , characterized in that When read by newly grouped plurality testimonies possession image, or when read in succession, and determine one single Duplicatability testament possession image overlaps with the registered testimony possessed images If newly read testimony possession images that may have been present, in a plurality of sheets, or the testimony possessed images read in succession, at least two, with simultaneously displayed, the a registered testimony possessed images the testimony possession image newly read that may be overlapping, that displayed differently from others,
The accounting system according to any one of claims 1 to 6 , characterized in that