JP2021002183A - Accounting support apparatus, accounting support program, and accounting support method - Google Patents

Abstract

To provide an accounting support apparatus, an accounting support program, and an accounting support method capable of improving the conversion accuracy by OCR software for vouchers in which a plurality of fields are irregularly arranged.SOLUTION: In an accounting support system 100, an accounting support apparatus 1 comprises: acquisition means (a communication unit 13) for acquiring medical receipt image data D2; frame identification means (a control unit 11) for identifying a frame that is an area enclosed by a ruled line L from the medical receipt image data D2; frame image data creation means (the control unit 11) for creating frame image data D7 which is image data acquired by extracting the frame portion from the medical receipt image data D2; and conversion means (the control unit 11) for converting the frame image data D7 into a character code and acquiring frame image conversion result data D10 using OCR software 129.SELECTED DRAWING: Figure 1

Description

The present invention relates to an accounting support device, an accounting support program, and an accounting support method.

In order to use a paper voucher indicating a user's transaction, such as an invoice or receipt, for various accounting processes, it is desirable to convert the description content into a character code and convert it into electronic character data.
Therefore, conventionally, tax accountants, certified accountants, etc. have read the contents of medical receipts one by one and manually input them into a computer.

However, it is very troublesome to manually input the contents of the voucher, and it is unavoidable that an input error occurs, and it cannot be said that the accuracy is sufficient.
Therefore, there is known a system that can automatically convert the contents of a predetermined voucher into a character code by using OCR (Optical Character Recognition / Reader) software (for example, Patent Documents). 1).

Japanese Patent No. 6404524

Conversion of the contents of the voucher to the character code using OCR software is usually done in order from left to right and top to bottom of the voucher, so it is a voucher in which characters are regularly arranged. If so, it was easy to improve the conversion accuracy.
However, for example, for vouchers such as medical receipts, salary details, and social insurance payment slips, when multiple entry fields (hereinafter referred to as "frames") surrounded by ruled lines are irregularly arranged. For such vouchers, it was difficult to convert them accurately.

An object of the present invention is to provide an accounting support device, an accounting support program, and an accounting support method capable of improving the conversion accuracy by OCR software for a voucher in which a plurality of entry fields are irregularly arranged.

In order to solve the above problems, the invention according to claim 1 is used in an accounting support device.
An acquisition means for acquiring voucher image data, which is the image data of the voucher to be converted, and
A frame identification means for identifying a frame that is an area surrounded by a ruled line from the voucher image data,
A frame image data creation means for creating frame image data, which is image data obtained by extracting the frame portion from the voucher image data, and
A conversion means for converting the frame image data into a character code and acquiring the frame image conversion result data using OCR software, and
It is characterized by having.

The invention according to claim 2 is the accounting support device according to claim 1.
The frame identification means
After detecting a straight line from the voucher image data,
The area surrounded by the straight line is identified as the frame.

The invention according to claim 3 is the accounting support device according to claim 1 or 2.
An out-of-frame image data creation means for creating out-of-frame image data, which is image data obtained by deleting the description of the frame portion from the voucher image data, is provided.
The conversion means is characterized in that the out-of-frame image data is converted into a character code by using OCR software and the out-of-frame image conversion result data is acquired.

The invention according to claim 4 is the accounting support device according to any one of claims 1 to 3.
Using the frame image conversion result data, it is determined whether or not the frame image data includes a predetermined item name, and the item name frame image data which is the frame image data including the predetermined item name is extracted. It is characterized by being provided with an extraction means.

The invention according to claim 5 is the accounting support device according to claim 4.
It is characterized by comprising a two-frame image data creating means for creating two-frame image data by synthesizing frame image data of adjacent frames with the item name frame image data.

The invention according to claim 6 is the accounting support device according to any one of claims 1 to 5.
It is characterized by providing a line number specifying means for specifying the number of lines of a character string included in the frame image data based on the frame image conversion result data.

The invention according to claim 7 is the accounting support device according to claim 6.
When the frame image data includes a character string of a plurality of lines, the frame image data is provided with a line-by-line image data creating means for creating the line-by-line frame image data by dividing the frame image data into lines.

The invention according to claim 8 is the accounting support device according to claim 7.
Using the frame image conversion result data, it is determined whether or not the frame image data includes a predetermined item name, and the item name frame image data which is the frame image data including the predetermined item name is extracted. Equipped with extraction means
The item name frame extraction means
With respect to the frame in which the line-by-line frame image data is created, it is determined whether the line-by-line frame image data includes a predetermined item name, and the line-by-frame image data including the predetermined item name is used as the item. Extracted as name frame image data,
For frames for which frame-by-line image data has not been created, it is determined whether the frame image data includes a predetermined item name, and the frame image data including the predetermined item name is used as the item name frame image. It is characterized by being extracted as data.

The invention according to claim 9 is the accounting support device according to claim 8.
A two-frame image data creation means for creating two-frame image data by synthesizing frame image data of adjacent frames with the item name frame image data is provided.
The two-frame image data creation means
When the line-by-frame image data is extracted as the item name frame image data,
After determining whether the number of rows of the frame containing the frame-by-line image data matches the number of adjacent frames, if they match, the frame-by-line image data and the frame adjacent to the right are the same. Combine the line-by-line frame image data
When the frame image data is extracted as the item name frame image data,
After determining whether the widths of the frame related to the frame image data and the adjacent frame below match in the left-right direction, if they match, the frame image data and the frame image data of the adjacent frame below It is characterized by synthesizing and.

The invention according to claim 10 is the accounting support device according to any one of claims 1 to 9.
A transmission means for transmitting the frame image data and the frame image conversion result data corresponding to the frame image data to a separate worker terminal for each frame image data.
A receiving means for receiving the frame image conversion result data after confirmation and correction by the worker from the worker terminal, and
It is characterized by having.

The invention according to claim 11 is the accounting support device according to any one of claims 1 to 10.
It is characterized by comprising a data extraction means for extracting data about an item that matches the extraction item data, which is data related to a predetermined extraction item, from the frame image conversion result data.

The invention according to claim 12 is the accounting support device according to claim 5 or 9.
A data extraction means for extracting data for items that match the extraction item data, which is data related to a predetermined extraction item, from the frame image conversion result data is provided.
When the frame image conversion result data corresponding to the item name frame image data of the two-frame image data includes an item name matching the extracted item data, the data extraction means synthesizes the item name frame image data. It is characterized in that the frame image conversion result data related to the frame image data is extracted.

The invention according to claim 13 is the accounting support device according to claim 12.
When the frame image conversion result data corresponding to the item name frame image data of the two-frame image data does not include an item name matching the extracted item data, the data extraction means all the frame image conversion results. It is characterized in that data for an item matching the extracted item data is extracted from the data.

The invention according to claim 14 is the accounting support device according to any one of claims 11 to 13.
The data extraction means selects data whose description format matches the extraction item format data, which is the data related to the description format of the data extracted by the data extraction means, among the data about the items matching the extraction item data. It is characterized by extracting.

The invention according to claim 15 is the accounting support device according to claim 3.
From the frame image conversion result data, data for items that match the extraction item data, which is data related to a predetermined extraction item, is extracted.
Data extraction means for extracting data for items that match the extracted item data from the out-of-frame image conversion result data when data for items that match the extracted item data is not extracted from the frame image conversion result data. It is characterized by having.

The invention according to claim 16 is used in an accounting support program.
Computer,
Acquisition means for acquiring voucher image data, which is the image data of the voucher to be converted,
A frame identification means for identifying a frame that is an area surrounded by a ruled line from the voucher image data.
A frame image data creation means for creating frame image data which is image data obtained by extracting the frame portion from the voucher image data.
A conversion means for converting the frame image data into a character code and acquiring the frame image conversion result data using OCR software.
It is characterized by functioning as.

The invention according to claim 17 is an accounting support method.
The acquisition step to acquire the voucher image data, which is the image data of the voucher to be converted,
A frame identification step for identifying a frame that is an area surrounded by a ruled line from the voucher image data,
A frame image data creation step for creating frame image data, which is image data obtained by extracting the frame portion from the voucher image data, and
A conversion step of converting the frame image data into a character code and acquiring the frame image conversion result data using OCR software, and
It is characterized by including.

According to the present invention, it is possible to provide an accounting support device, an accounting support program, and an accounting support method capable of improving the conversion accuracy by the OCR software for a voucher in which a plurality of entry fields are irregularly arranged.

It is a block diagram which shows the structure of the accounting support system which concerns on embodiment. It is a block diagram which shows the flow of data in the accounting support system which concerns on embodiment. It is a flowchart which shows the outline of the operation flow in the accounting support system which concerns on embodiment. It is a flowchart which shows the flow of the pre-registration in the accounting support system which concerns on embodiment. It is a flowchart which shows the flow of conversion of the medical receipt in the accounting support system which concerns on embodiment. It is a flowchart which shows the flow of the check by a cloud worker in the accounting support system which concerns on embodiment. It is a flowchart which shows the flow of the extraction of the data for medical expense deduction application in the accounting support system which concerns on embodiment. It is a figure which shows an example of the medical receipt image data. It is a figure which shows an example of the medical receipt image data. It is a figure which shows an example of frame image data. It is a figure which shows an example of frame image data. It is a figure which shows an example of frame image data. It is a figure which shows an example of frame image data. It is a figure which shows an example of the out-of-frame image data. It is a figure which shows an example of the frame image data for every line. It is a figure which shows an example of the frame image data for every line. It is a figure which shows an example of two-frame image data. It is a figure which shows an example of two-frame image data.

Hereinafter, the accounting support system 100 according to the embodiment of the present invention will be described with reference to FIGS. 1 to 18. In the following, a case where the voucher to be converted by the OCR software is a medical receipt and data on items required for medical expense deduction application is extracted from the conversion result will be described as an example. The technical scope is not limited to this.

[Explanation of the first configuration]
First, the configuration of the accounting support system 100 will be described with reference to FIG.
As shown in FIG. 1, the accounting support system 100 is composed of an accounting support device 1 that converts the description content of the medical receipt image data (medical receipt image data D2) into a character code, and the accounting support device 1. The accounting support device 1 and each worker terminal 2 are provided with a worker terminal 2 used by each of a plurality of workers W for confirming the conversion result and a user terminal 3 used by each user U of the accounting support system 100. And between the accounting support device 1 and each user terminal 3 are connected via the communication network N.

[1 Accounting support device]
The accounting support device 1 is, for example, an information device such as a PC (Personal Computer) or WS (Work Station) installed in a company operating the accounting support system 100, a tax accountant corporation, or the like, and is controlled as shown in FIG. It is configured to include a unit 11, a storage unit 12, a communication unit 13, a display unit 14, and an operation unit 15.

[(1) Control unit]
The control unit 11 is a part that controls the operation of the accounting support device 1, and specifically, is configured to include a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. Each part of the accounting support device 1 is collectively controlled by the cooperation between the CPU and the ROM stored in the ROM or the storage unit 12 developed in the work area of the RAM.

[(2) Storage unit]
The storage unit 12 is a part that stores various information necessary for the operation of the accounting support device 1, and is composed of, for example, an HDD (Hard Disk Drive), a semiconductor memory, or the like, and is data such as program data and various setting data. Is readable and writable from the control unit 11.

The storage unit 12 includes a user data storage unit 121, a conversion target data storage unit 122, a conversion data storage unit 123, an extraction data storage unit 124 for medical expense deduction application, an extraction item data storage unit 125, and an extraction item. A format data storage unit 126, a medical receipt conversion engine 127, a data extraction engine 128 for medical expense deduction application, and OCR software 129 are provided.

It should be noted that these databases, software, etc. do not necessarily have to be provided in the storage unit 12, but are provided in a server or the like separate from the accounting support device 1, and are provided in the storage unit 12. Only the URL, password, etc. for access may be stored, and the accounting support device 1 may acquire and use the information via the communication network N, if necessary.

The user data storage unit 121 is a database that stores user data D1 which is data about a user U who uses the accounting support system 100.
The user data D1 is data in which various information such as the name and company name of the user U are stored in association with an ID and a login password set for each user U, for example.

The conversion target data storage unit 122 converts the medical receipt image data D2, which is the target to be converted into the character code by the accounting support device 1, into the user data D1 and the conversion presence / absence data D3 of the user U related to the medical receipt image data D2. It is a database that stores in association with.

The medical receipt image data D2 is, for example, data obtained by converting a medical receipt issued by a hospital, a pharmacy, etc. into image data as shown in FIGS. 8 and 9, and is a general predetermined digital such as PDF, JPEG, etc. An image in a file format related to still image data can be used. The medical receipt image data D2 can be obtained, for example, by converting a medical expense receipt, which is a paper medium, into digital still image information using a general image scanner.
The conversion presence / absence data D3 is data indicating whether the medical receipt image data D2 is data that has already been converted into a character code in the accounting support device 1, and the processing such as conversion is completed as described later. It is stored that the conversion has not been performed before the conversion, and when the processing such as conversion is completed, the stored content is switched to the storage indicating that the conversion has already been performed.

The conversion data storage unit 123 is a database that stores the data after conversion by the OCR software 129, and as will be described later, the frame image data D7, the frame position information data D8, the out-of-frame image data D9, and the frame image conversion result data D10. , Out-of-frame image conversion result data D11, row-by-frame image data D12, row-by-frame image conversion result data D13, item name frame image data D14, item name frame image conversion result data D15, and two-frame image data D16 are stored.

As will be described later, the extraction data storage unit 124 for the medical expense deduction application extracts the data related to the items required for the medical expense deduction application extracted from the conversion result of the medical receipt image data D2 for the medical expense deduction application. This is a database that stores data D21 in association with user data D1 of user U related to the data.
The extracted data D21 for medical expense deduction application includes date data D21-1 which is data related to the date, name data D21-2 which is data related to the name, and payee data D21-3 which is data related to the payee. , Used drug data D21-4 which is data related to used drug, payment amount data D21-5 which is data related to payment amount, and medical / treatment score data D21-6 which is data related to medical / treatment score. , Medical product purchase score data D21-7, which is data related to the score of medical product purchase, and nursing care insurance service score data D21-8, which is data related to the score of the care insurance service.

The extraction item data storage unit 125 is a database in which the extraction item data D4, which is data related to the extraction items required for the medical expense deduction application, is stored in advance.
The extracted item data D4 includes date item data D4-1, name item data D4-2, medical institution data D4-3, drug item data D4-4, first payment item data D4-5, and second. It includes payment item data D4-6, medical / treatment item data D4-7, medical product purchase item data D4-8, and nursing care insurance service item data D4-9. The content of the extracted item data D4 is not limited to these, and if there is an increase or decrease in the data required for the medical expense deduction application, it can be added or omitted as appropriate.

The date item data D4-1 is a list of item names attached to the description of the date in the medical receipt, and for example, the item names such as "medical treatment date" and "issue date" are stored.
The name item data D4-2 is data in which the description contents for extracting the description of the name are stored in the medical receipt, and for example, the character "sama" is stored.
The medical institution data D4-3 is data in which the names of medical institutions such as hospitals and pharmacies are associated with the telephone numbers of the medical institutions.
The drug item data D4-4 is data in which a predetermined drug name that may appear in the medical receipt is stored.
The first payment item data D4-5 is a list of item names attached to the description of the payment amount in the medical receipt, for example, item names such as "receipt amount", "billing amount", and "payment amount". Is remembered. As will be described later, the first payment item data D4-5 is data that is preferentially used over the second payment item data D4-6.
The second payment item data D4-6 is data that stores the description contents for extracting the description of the payment amount in the medical receipt, and for example, characters such as "¥", "yen", and "total" are displayed. It is remembered.
The medical / treatment item data D4-7 is a list of item names attached to the description of medical / treatment in the medical receipt, for example, "medical treatment", "treatment", "needle", and "examination cost". , "Attendant fee" and other item names are stored.
The medical product purchase item data D4-8 is a list of item names attached to the description of the medical product purchase in the medical receipt, and is, for example, "dispensing technology", "pharmacy management", "drug", "specific health". Item names such as "medical material" and "medical material" are stored.
The long-term care insurance service item data D4-9 is a list of item names attached to the description of the long-term care insurance service in the medical receipt, and is, for example, "home-visit care", "home help service", "home-visit bathing care", etc. Item names such as "visit rehabilitation" and "day service" are stored.

The extraction item format data storage unit 126 is a database in which extraction item format data D5, which is data in which the description format of the data to be extracted is stored for each item required for the medical expense deduction application, is stored in advance.
The extracted item format data D5 includes date format data D5-1, name format data D5-2, telephone number format data D5-3, payment amount format data D5-4, and medical / treatment score format data D5-5. And, the medical product purchase point format data D5-6 and the care insurance service point format data D5-7 are included. The content of the extracted item format data D5 is not limited to these, and if there is an increase or decrease in the data required for the medical expense deduction application, it can be added or omitted as appropriate.

The date format data D5-1 is data related to the description format to be extracted as a date. For example, a format in which a plurality of numbers are separated by "/" or a plurality of numbers are "year" and "month". And / or the format separated by "days" is stored.
The name format data D5-2 is data related to the description format to be extracted as a name, and for example, a character string format that does not include arithmetic numbers or symbols is stored.
The telephone number format data D5-3 is data related to the description format to be extracted as a telephone number. For example, a format in which a plurality of numbers are separated by "-" or a "()" between the plurality of numbers. The format in which the numbers enclosed in "" exist and the format in which both "-" and "()" exist between a plurality of numbers are stored.
The payment amount format data D5-4 is data related to the description format to be extracted as the payment amount, and is, for example, described without being separated by "-", "()" or "/" between the numbers. The format is remembered.
The medical / treatment score format data D5-5 is data related to the description format to be extracted as the medical / treatment score, and for example, only a number or a format in which a “point” follows a number is stored.
The medical product purchase point format data D5-6 is data related to the description format to be extracted as the medical product purchase point, and for example, only a number or a format in which a "point" follows a number is stored.
The long-term care insurance service score format data D5-7 is data related to the description format to be extracted as the long-term care insurance service score, and for example, only numbers or a format in which "dots" follow the numbers are stored.

The medical receipt conversion engine 127 is a program in which the contents of instructions to the accounting support device 1 in a predetermined process from conversion of the medical receipt image data D2 described later to confirmation / correction by the worker W are combined.

The data extraction engine 128 for medical expense deduction application is a program in which the contents of instructions to the accounting support device 1 in the predetermined process related to the extraction of the data required for the medical expense deduction application described later are combined.

OCR software 129 is software related to general OCR (Optical Character Recognition / Reader), and is software that converts an image of a living document into a string of character codes.
However, as described above, it is not essential that the OCR software 129 itself is stored in the storage unit 12, for example, using a cloud-type OCR service, the OCR software 129 itself is stored on a predetermined cloud server. Only information such as a URL and a password for calling such OCR software 129 is stored in the storage unit 12, and the accounting support device 1 uses the communication network N to store the OCR on the cloud server. You may want to access software 129 and use it.

[(3) Communication section]
The communication unit 13 is a part used for communication between the accounting support device 1 and the worker terminal 2 and communication between the accounting support device 1 and the user terminal 3. For example, a communication IC (Integrated Circuit) and It is a communication interface having a communication connector and the like, and under the control of the control unit 11, data communication is performed via the communication network N using a predetermined communication protocol.

[(4) Display]
The display unit 14 includes, for example, a display such as an LCD (Liquid Crystal Display), and displays an image based on a display control signal output from the control unit 11 on the display screen.

[(5) Operation unit]
The operation unit 15 includes, for example, a keyboard having a character input key, a number input key, and other keys associated with various functions, a pointing device such as a mouse, and the like. For example, a company or a tax accountant who operates the accounting support system 100. It receives an operation input from an employee of a corporation or the like and outputs an operation signal corresponding to the operation input to the control unit 11. The operation unit 15 may be, for example, a touch panel integrally formed with the display unit 14.

[2 Worker terminal]
The worker terminal 2 is an information device such as a PC or WS (Work Station) used by each of a plurality of workers W who confirm the conversion result by the accounting support device 1. For example, as shown in FIG. 1, accounting support is provided. Similar to the device 1, the control unit 21, the storage unit 22, the communication unit 23, the display unit 24, and the operation unit 25 are provided.
However, as shown in FIG. 1, unlike the accounting support device 1, the storage unit 22 is a screen to be displayed to the worker W after applying the data received from the accounting support device 1 as described later. The display screen data storage unit 221 for storing the display screen data D6, which is the data of the above, is provided.
The display screen data storage unit 221 is provided in the storage unit 12 of the accounting support device 1, and the accounting support device 1 applies predetermined data to the storage unit 12 to complete the screen displayed on the worker terminal 2. Then, it may be transmitted to the worker terminal 2.

[3 User terminal]
The user terminal 3 is an information device such as a PC, a smartphone, or a tablet terminal used by each of a plurality of users U of the accounting support system 100. For example, as shown in FIG. 1, a control unit like the accounting support device 1. It includes 31, a storage unit 32, a communication unit 33, a display unit 34, and an operation unit 35.

[4 Communication network]
The communication network N is, for example, the Internet, a telephone line network, a mobile phone communication network, etc., and transmits / receives data between the accounting support device 1 and the worker terminal 2, and the accounting support device 1 and the user terminal 3. There is no particular limitation as long as it is possible to send and receive data between them.

[Explanation of the second operation]
Hereinafter, the flow at the time of using the accounting support system 100 according to the embodiment will be described with reference to FIGS. 2 to 18. In the following, as shown in FIGS. 8 and 9, the top, bottom, left and right of the medical receipt are defined, and the horizontal (left and right) direction of the medical receipt is the X-axis direction, and the vertical (vertical) direction is the Y-axis direction. I will explain it.

The flow when using the accounting support system 100 can be roughly divided into pre-registration (step S1), conversion of medical receipts (step S2), check by cloud worker (step S3), and medical expense deduction, as shown in FIG. It consists of four processes of extracting application data (step S4). Each process will be described in detail below.

[1 step S1: Pre-registration]
First, the user U who uses the accounting support system 100 performs the following pre-registration. As shown in FIG. 4, the pre-registration process consists of two steps, steps S1-1 to S1-2.

[(1) Step S1-1: Registration of user data]
The user U who uses the accounting support system 100 first registers the user data D1 before starting to use the system. Specifically, the user U uses the operation unit 35 of the user terminal 3 to input user data D1 including predetermined information including, for example, a name, an address, a contact information, an ID when using the system, a password, and the like. ..
When these data are input, the user terminal 3 transmits the input data from the communication unit 33 to the accounting support device 1 via the communication network N as shown in FIG. 2, and the communication unit 13 transmits the input data to the accounting support device 1. In the accounting support device 1 that has received the above, the control unit 11 stores the received user data D1 in the user data storage unit 121 of the storage unit 12.

[(2) Step S1-2: Registration of medical receipt image data]
The user U who uses the accounting support system 100 subsequently registers the medical receipt image data D2.
Specifically, the user U creates the medical receipt image data D2 by scanning by an arbitrary method, and then transmits the medical receipt image data D2 stored in the user terminal 3 from the communication unit 33 to the communication network N. Is transmitted to the accounting support device 1 as shown in FIG.
In the accounting support device 1 that has received this from the communication unit 13, the control unit 11 associates the received medical receipt image data D2 with the user data D1 and the conversion presence / absence data D3 related to the user, and the storage unit 12 Is stored in the conversion target data storage unit 122 of. At the stage of registration, the conversion presence / absence data D3 is all data indicating that the conversion has not been performed.

It should be noted that, without going through the above process, for example, an employee of a company operating the accounting support system 100 or a tax accountant corporation uses a scanner or the like directly connected to the accounting support device 1 to receive the receipt from the user U. The accounting support device 1 may directly acquire the medical receipt image data D2 by scanning the original medical receipt or a copy thereof.

[2 Step S2: Conversion of medical receipt]
Subsequently, in the accounting support device 1, processing such as conversion of the medical receipt image data D2 into a character code by the OCR software 129 is performed. As shown in FIG. 5, the process of converting the medical receipt consists of eight steps from steps S2-1 to S2-8.
Note that these processes are automatically performed by the control unit 11 according to the medical receipt conversion engine 127.

[(1) Step S2-1: Extraction of data to be converted]
The accounting support device 1 first extracts the unconverted medical receipt image data D2 to be converted from the conversion target data storage unit 122.
Specifically, the control unit 11 has already performed the later processing such as conversion by the OCR software 129 from the conversion target data storage unit 122 of the storage unit 12 by the conversion presence / absence data D3 for each of the medical receipt image data D2. After determining whether the data is made, select and extract the unconverted data.

[(2) Step S2-2: Generation of frame image data]
When the unconverted medical receipt image data D2 to be converted is extracted in the accounting support device 1, the control unit 11 subsequently refers to the extracted medical receipt image data D2 as frame C, that is, a ruled line. The frame image data D7 is generated by dividing each area surrounded by L. Specifically, it is as follows.

[Step S2-2-1: Straight line detection]
First, the control unit 11 detects a straight line from the medical receipt image data D2. The method for detecting the straight line is not particularly limited, but for example, the straight line may be detected by extracting an edge (a portion where the change in brightness is large) from the medical receipt image data D2 converted to monochrome.

[Step S2-2-2: Identification of frames]
Subsequently, the control unit 11 identifies the coma C based on the straight line detected in step S2-2-1. Specifically, from the position of the detected straight line, a region surrounded by straight lines on all sides and a rectangle is formed is calculated, and the region is identified as the frame C surrounded by the ruled line L.
As shown in FIG. 9, in the area surrounded by the ruled line L, there is an area in which a curved portion exists at a part of the corners of the rectangle. In such an area, for example, each of the four sides When at least a part of it is a straight line, it can be identified as a frame C by identifying it as a frame C. In the present invention, such a region is also surrounded by straight lines as long as at least a part of each of the four sides of the top, bottom, left, and right is a straight line.

[Step S2-2-3: Generation of frame image data]
Subsequently, the control unit 11 generates frame image data D7 as shown in FIGS. 10 to 13 by cutting out each frame C identified in step S2-2-2 from the medical receipt image data D2. To do. The frame image data D7 shown in FIGS. 10 to 13 is only an example, and in addition to these, frame image data D7 is created for all the portions identified as frame C in step S2-2-2. ..

When creating the frame image data D7, it is preferable to generate an image one size larger than each frame C by slightly cutting out the periphery of the frame C identified in step S2-2-2.
In vouchers such as medical receipts, as a result of the printing position shifting from the table in the voucher, the print that should originally fit in the frame of the table may slightly protrude from the table, but the frame image data By making D7 larger than each frame C, it becomes easy to correspond to the medical receipt image data D2 related to the medical receipt whose print is out of the table.

Further, when the frame image data D7 is generated, at the same time, the coordinates in the X-axis direction and the Y-axis direction on the image related to the medical receipt image data D2 are recorded for each frame C, for example, for the four corners thereof. By the method, for each frame C related to each frame image data D7, frame position information data D8, which is data capable of determining the position on the medical receipt image data D2, is created.
The frame image data D7 and the frame position information data D8 are assigned numbers to each frame C related to the frame image data D7, and the numbers are assigned to both the frame image data D7 and the corresponding frame position information data D8. It is associated by a method such as storing.

[(3) Step S2-3: Generation of out-of-frame image data]
When the frame image data D7 is generated, the control unit 11 of the accounting support device 1 subsequently refers to the frame C, that is, step S2, with respect to the medical receipt image data D2 for which the frame image data D7 was generated in step S2-2. The out-of-frame image data D9, which is the image data excluding the portion identified as the region surrounded by the straight line in -2-2, is generated.

Specifically, all the description of the frame C portion identified in step S2-2-2 is deleted, and an image in which only the portion other than the frame C is described, as shown in FIG. 14, is generated. The image data is the out-of-frame image data D9.

[(4) Step S2-4: OCR processing]
When the frame image data D7 is generated in step S2-2 and the out-of-frame image data D9 is generated in step S2-3, OCR processing is subsequently performed.
Specifically, the control unit 11 of the accounting support device 1 individually converts each of the frame image data D7 and the out-of-frame image data D9 into a character code using the OCR software 129, and converts each of the frame image data D7 into a character code. The corresponding plurality of frame image conversion result data D10 which is the conversion result data by the OCR software 129 and the out-of-frame image conversion result data D11 which is the conversion result data by the OCR software 129 corresponding to the out-of-frame image data D9. create. That is, the control unit 11 functions as a conversion means for the frame image data D7 and the out-of-frame image data D9.

For example, the OCR software 129 itself is stored on a predetermined cloud server without being stored in the storage unit 12 of the accounting support device 1, and the URL for calling such OCR software 129 is stored in the storage unit 12. , Passwords and other information are stored, and even when the OCR software 129 on the cloud server is called one by one for conversion, the control unit 11 makes such a call. Therefore, in this case as well, the control unit 11 However, it functions as a conversion means for the frame image data D7 and the out-of-frame image data D9.
Further, in order to acquire the frame image conversion result data D10 and the out-of-frame image conversion result data D11 in the present invention, the control unit 11 itself converts the frame image data D7 and the out-of-frame image data D9 by using the OCR software 129. , Both are included in the case where the frame image data D7 and the out-of-frame image data D9 are converted by the OCR software 129 on the cloud server and received via the communication unit 13.

The frame image data D7 and the frame image conversion result data D10 which is the result of converting the frame image data D7 are assigned a number to each frame C related to the frame image data D7, and correspond to the frame image data D7. Both of the frame image conversion result data D10 are associated with each other by a method such as storing the number.

[(5) Step S2-5: Division of frame image]
When the frame image conversion result data D10 and the out-of-frame image conversion result data D11 are created in step S2-4, the control unit 11 subsequently includes a plurality of lines of character strings in the frame image data D7. Divide things.
Specifically, the division is performed through the following four steps from step S2-5-1 to step S2-5-4.

[Step S2-5-1: Judgment of existence / absence of character string]
First, for each frame image data D7, the control unit 11 determines whether or not the frame image conversion result data D10 created in step S2-4 includes the character code generated by the OCR software 129.

Here, the frame image data D7 determined to include the character code generated by the OCR software 129 in the frame image conversion result data D10 means the data related to the frame C in which the character string exists. Then, the process proceeds to step S2-5-2.
On the other hand, here, the frame image data D7 determined that the frame image conversion result data D10 does not include the character code generated by the OCR software 129 is a blank frame in which no character string exists. It means the data related to C, is excluded from the subsequent processing in step S2-5, and proceeds to step S2-6.

[Step S2-5-2: Determining the number of lines]
Subsequently, the control unit 11 uses each frame image data D7 and the corresponding frame image conversion result data D10 to determine whether or not each frame image data D7 includes a character string of a plurality of lines. ..

Specifically, since the position information for the original image (here, the frame image data D7) is given to each character in the character code string acquired by the OCR processing, X is based on this position information. A character string located close to each other in the axial direction (horizontal direction) is estimated as a line, and the number of lines is determined from the number of such lines in the Y-axis direction (vertical direction).
Then, the frame image data D7 corresponding to the frame image conversion result data D10 in which the number of lines is determined to be a plurality of lines in this way is the frame image data D7 including a character string of a plurality of lines as shown in FIGS. 10 and 11. The frame image data D7 corresponding to the frame image conversion result data D10, which is determined to be only one line, is the frame image data that does not include a multi-line character string as shown in FIGS. 12 and 13. It is determined that it is D7.
Here, the frame image data D7 determined not to include the character strings of a plurality of lines is excluded from the subsequent processing in step S2-5, and the process proceeds to step S2-6.

[Step S2-5-3: Determination of matching of the number of lines with the adjacent frame]
Subsequently, the control unit 11 matches the number of lines of the frame C related to the frame image data D7 and the frame C adjacent to the right side of the frame image data D7 determined to include the character strings of a plurality of lines. Determine whether or not to do so.

Specifically, it is located to the right of the frame C related to the frame image data D7 determined to include a character string of a plurality of lines based on the frame position information data D8, that is, the frame C adjacent to the frame C, that is, to the left. After determining whether there is a frame C in which the right border of the frame C and the left border of the frame C located on the right are common, if it is determined that the frame C exists, then for each of these two frames C , Determine if the positions of the upper and lower ends in the Y-axis direction match.
Then, when they match, the number of rows determined in step S2-5-2 is further compared for these two frames C, and it is determined whether or not the numbers of rows match.

[Step S2-5-4: Generation of line-split image]
Subsequently, the control unit 11 divides the frame image data D7 determined to include the character strings of a plurality of lines in step S2-5-2 into each line, as shown in FIGS. 15 and 16. , Line-by-line frame image data D12 is generated. Note that FIG. 15 is a cut-out of the first line of the frame image data D7 according to FIG. 10, and FIG. 16 is a cut-out of the first line of the frame image data D7 according to FIG.
Further, the frame image conversion result data D10 related to the frame C in which the line-by-line frame image data D12 is generated is divided according to the line-by-line frame image data D12, and the line-by-frame image conversion result data D13 is generated.

The line-by-line frame image data D12 and the line-by-line frame image conversion result data D13 are, for example,, as described above, each frame C related to the frame image data D7 is numbered, and the line-by-line frame image data D12 is used. The line-by-line image conversion result data D13 corresponding to this is associated with the data by a method such as storing the data of the line from the top in the frame C in addition to the number.

[(6) Step S2-6: Judgment of item name frame]
Subsequently, the control unit 11 determines whether each frame image data D7 or line-by-line frame image data D12 includes a predetermined item name.

First, the control unit 11 determines whether the line-by-line frame image data D12 includes a predetermined item name for the frame C divided into the line-by-line frame image data D12 in step S2-5-4.
Specifically, the data about the predetermined character string corresponding to the item name is stored in the storage unit 12 in advance, and the character string matching the character string included in the data is stored in the line-by-frame image conversion result data D13. It is determined whether or not the data is included, and if it is included, it is determined that the row-by-line frame image data D12 corresponding to the row-by-frame image conversion result data D13 includes a predetermined item name.

Subsequently, the control unit 11 determines whether the frame image data D7 includes a predetermined item name for the frame C that is not divided into the line-by-line frame image data D12 in step S2-5-4.
In this case as well, the data for the predetermined character string corresponding to the item name is stored in the storage unit 12, and the character string matching the character string included in the data is included in the frame image conversion result data D10. If it is included, it is determined that the frame image data D7 corresponding to the frame image conversion result data D10 includes a predetermined item name.

In the present embodiment, among the extracted item data D4, date item data D4-1, first payment item data D4-5, medical / treatment item data D4-7, medical product purchase item data D4-8, nursing care insurance service Item data D4-9 corresponds to the data for a predetermined character string corresponding to the item name.
The line-by-line frame image data D12 and the frame image data D7 determined to include the item name are collectively referred to as the item name frame image data D14. Further, the line-by-line frame image conversion result data D13 and the frame image conversion result data D10 corresponding to the item name frame image data D14 are referred to as item name frame image conversion result data D15.

[(7) Step S2-7: Combining frame images]
When the item name frame image data D14 is extracted in step S2-6, the control unit 11 subsequently synthesizes the frame images.
It should be noted that the combined image data is referred to as two-frame image data D16 regardless of whether the frame image data D7 are combined with each other or the frame-by-line image data D12 are combined with each other.

[Step S2-7-1: Combining line-by-line frame images]
First, when a plurality of lines of character strings are included and frames C having the same number of lines are adjacent to each other on the left and right, frame-by-line image data D12 of the same line of the left and right frames C are combined.
Specifically, first, in step S2-5-2, it is determined that a plurality of lines of character strings are included, and in step S2-5-3, it is determined that the number of lines matches the adjacent frame C. The data D7 extracts the row-by-line frame image data D12 generated by being divided in steps S2-5-4, and further, in step S2-6, it is determined that the data D7 corresponds to the item name frame image data D14. Extract the data.
Then, the item name frame image data D14 and the frame image data D7 determined to be related to the adjacent frame C having the same number of lines in step S2-5-3 are divided in step S2-5-4. Of the row-by-frame image data D12 generated in this process, the row-by-frame image data D12 of the same row counting from the top is combined.

In FIG. 17, as an example, it is determined that the line-by-line frame image data D12 corresponding to the first line counting from the top of the frame image data D7 shown in FIG. 10 corresponds to the item name frame image data D14, and is adjacent to the right side thereof. The two-frame image data D16 combined with the line-by-line frame image data D12 corresponding to the same line, that is, the first line, counting from the top of the frame image data D7 shown in FIG. 11 is shown.

[Step S2-7-2: Combining frame images]
Subsequently, when the frames C not including the character strings of a plurality of lines are vertically adjacent to each other and their widths in the X-axis direction are equal, the frame image data D7s of the vertically adjacent frames C are combined.
Specifically, first, the frame image data D7 that was not determined to contain the character strings of a plurality of lines in step S2-5-2 is extracted, and then the item name frame is further extracted in step S2-6. The data determined to correspond to the image data D14 is extracted.
On top of that, under the item name frame image data D14, the border line adjacent to the frame C, that is, the border line below the frame C located above and the border line above the frame C located below are common. After determining whether the frame C exists, if it exists, it is further determined whether the width in the X-axis direction and the positions of the left end and the right end in the X-axis match, and if they match, the item The frame image data D7 corresponding to the name frame image data D14 and the frame image data D7 adjacent to the frame image data D7 are combined.

In FIG. 18, as an example, it is determined that the frame image data D7 shown in FIG. 12 corresponds to the item name frame image data D14, and the width of the frame image data D7 shown in FIG. 13 adjacent to the frame image data D7 in the X-axis direction. Further, it is determined that the positions of the left end portion and the right end portion in the X-axis direction match, and the two-frame image data D16 in which these are combined is shown in the figure.

[(8) Step S2-8: Saving data]
When steps S2-7 are completed, the control unit 11 saves the generated data.

Specifically, the control unit 11 refers to the frame image data D7 for the frame C which is not divided in step S2-5 and is not synthesized in step S2-7, and the corresponding frame image conversion result data. It is associated with D10 and stored in the conversion data storage unit 123 of the storage unit 12.
Further, for the frame C that is divided in step S2-5 and is not synthesized in step S2-7, the row-by-frame image data D12 is associated with the corresponding row-by-frame image conversion result data D13 and stored. It is stored in the conversion data storage unit 123 of the unit 12.
Further, for the frame C divided in step S2-5 and synthesized in step S2-7, the two-frame image data D16 is associated with the corresponding line-by-line frame image conversion result data D13, and is stored in the storage unit 12. Is stored in the conversion data storage unit 123 of.
Further, for the frame C that was not divided in step S2-5 and was synthesized in step S2-7, the two-frame image data D16 is associated with the corresponding frame image conversion result data D10, and the storage unit 12 It is stored in the converted data storage unit 123.
The frame image data D7, the line-by-line frame image data D12, and the two-frame image data D16 stored in this way are collectively referred to as the divided / combined frame image data D17, and are stored in association with these. The frame image conversion result data D10 and the line-by-line frame image conversion result data D13 are referred to as frame image conversion result data D18 after division / synthesis.
The divided / combined frame image data D17 is stored in association with the corresponding frame position information data D8 so that the position on the medical receipt image data D2 can be determined. Further, among these, for the data corresponding to the frame-by-line frame image data D12, the data regarding the row number from the top in the frame image data D7 before division is also stored.

Further, the control unit 11 also stores the out-of-frame image data D9 in the conversion data storage unit 123 of the storage unit 12 in association with the corresponding out-of-frame image conversion result data D11.

[3 Step S3: Check by cloud worker]
Subsequently, the conversion result in step S2 is checked by the cloud worker (worker W). As shown in FIG. 6, the check process by the cloud worker (worker W) consists of five steps from steps S3-1 to S3-5.
It should be noted that these processes, which are performed in the accounting support device 1, are automatically performed by the control unit 11 according to the medical receipt conversion engine 127.

[(1) Step S3-1: Transmission to worker terminal]
First, the control unit 11 of the accounting support device 1 transmits the data to be checked to the worker terminal 2.

Specifically, as shown in FIG. 2, the control unit 11 divides the divided / combined frame image data D17 stored in the converted data storage unit 123 of the storage unit 12 in step S2-8 corresponding to the divided / combined frame image data D17. Along with the post-composite frame image conversion result data D18, the data is transmitted from the communication unit 13 to a separate worker terminal 2 via the communication network N. Further, the out-of-frame image data D9 is also transmitted together with the out-of-frame image conversion result data D11 to a worker terminal 2 different from the one in which the divided / combined frame image data D17 is transmitted.

[(2) Step S3-2: Display on the worker terminal]
In the worker terminal 2 in which the communication unit 23 receives the divided / combined frame image data D17 and the corresponding divided / combined frame image conversion result data D18 transmitted from the accounting support device 1, the control unit 21 The received divided / combined frame image data D17 and the corresponding divided / combined frame image conversion result data D18 are applied to the display screen data D6 stored in the display screen data storage unit 221 of the storage unit 22. Then, it is displayed on the display unit 24.
Specifically, the divided / combined frame image data D17 and the corresponding divided / combined frame image conversion result data D18 are displayed side by side on the same screen. At this time, the divided / combined frame image conversion result data D18 is displayed so that the worker W can modify it by using the operation unit 25.
Regarding the two-frame image data D16, a total of 4 frames image data D7 or line-by-line frame image data D12 related to the two frames and two frame image conversion result data D10 or line-by-line image conversion result data D13 related to these. Will be displayed at the same time.

Further, in the worker terminal 2 in which the out-of-frame image data D9 and the out-of-frame image conversion result data D11 corresponding thereto are received by the communication unit 23, the control unit 21 receives the out-of-frame image data D9 and the corresponding out-of-frame image conversion. The result data D11 is applied to the display screen data D6 stored in the display screen data storage unit 221 of the storage unit 22, and then displayed on the display unit 24.
Specifically, the out-of-frame image data D9 and the corresponding out-of-frame image conversion result data D11 are displayed side by side on the same screen. Further, at this time, the out-of-frame image conversion result data D11 is displayed so that the worker W can correct it by using the operation unit 25.

For both the frame image and the out-of-frame image, if both the image data and the conversion result data are displayed on the same screen and the conversion result data can be corrected by the worker W, the specific method is specified. The display format is not particularly limited.
Further, as described above, the display screen data D6 is stored in the storage unit 12 of the accounting support device 1, and the screen data to which each data is applied is generated in the accounting support device 1 and then sent to the worker terminal 2. You may send it.

[(3) Step S3-3: Check by operator]
Subsequently, the worker W who uses each worker terminal 2 displaying the divided / combined frame image data D17 and the corresponding divided / combined frame image conversion result data D18 in step S3-2 is divided and combined. After comparing the post-combination frame image data D17 with the corresponding split / post-combination frame image conversion result data D18, the division / post-combination frame image conversion result data D18 where there is an erroneous conversion in the conversion content is corrected. Do.

Further, the worker W who uses the worker terminal 2 on which the out-of-frame image data D9 and the corresponding out-of-frame image conversion result data D11 are displayed in step S3-2 is the out-of-frame image data D9 and the corresponding out-of-frame image. After comparing with the conversion result data D11, the out-of-frame image conversion result data D11 at the place where the conversion content has an erroneous conversion is corrected.

It should be noted that the frame image conversion result data D18 after division / composition is checked by the worker W and corrected when there is an erroneous conversion, which is called the frame image conversion result data D19 after the check, and the out-of-frame image conversion result. When the data D11 is checked by the worker W and erroneously converted, the corrected data is referred to as the out-of-frame image conversion result data D20 after the check.

[(4) Step S3-4: Transmission to accounting support device]
When the check is performed by the worker W in step S3-3 and the frame image conversion result data D19 after the check or the out-of-frame image conversion result data D20 after the check is generated, the control unit 21 of each worker terminal 2 displays FIG. As shown in the above, the post-check frame image conversion result data D19 or the post-check out-of-frame image conversion result data D20 is transmitted from the communication unit 23 to the accounting support device 1 via the communication network N.

[(5) Step S3-5: Update conversion data storage unit]
When the accounting support device 1 receives the frame image conversion result data D19 after checking by the communication unit 13, the control unit 11 stores the frame image conversion result data D18 after division / synthesis stored in the conversion data storage unit 123 of the storage unit 12. Is updated to the frame image conversion result data D19 after the check.
Further, when the accounting support device 1 receives the out-of-frame image conversion result data D20 after being checked by the communication unit 13, the control unit 11 displays the out-of-frame image conversion result data D11 stored in the conversion data storage unit 123 of the storage unit 12. After the check, the image conversion result data outside the frame is updated to D20.

[4 Step S4: Extraction of data for medical expense deduction application]
Subsequently, the accounting support device 1 extracts the data necessary for applying for medical expense deduction from the conversion result. As shown in FIG. 7, the process of extracting the data for medical expense deduction application consists of 10 steps from steps S4-1 to S4-10, and the data for medical expense deduction application is extracted through these steps. ..
In addition, these processes are automatically performed by the control unit 11 according to the data extraction engine 128 for medical expense deduction application.

[(1) Step S4-1: Extraction of date]
The control unit 11 extracts data related to the date required for the medical expense deduction application from the post-check frame image conversion result data D19 and the post-check out-of-frame image conversion result data D20. Specifically, it is as follows.

First, the control unit 11 extracts the data corresponding to the two-frame image data D16 from the frame image conversion result data D19 after the check.
Subsequently, the control unit 11 searches for the data corresponding to the item name frame image conversion result data D15 among the checked frame image conversion result data D19 corresponding to the extracted two-frame image data D16, and contains the data corresponding to the item name frame image conversion result data D15. Extracted item data of the storage unit 12 When there is data matching the date item data D4-1 stored in the extracted item data D4 stored in the storage unit 125, that is, a character string of "medical treatment date" or "issue date". In the two-frame image data D16, the check-after frame image conversion result data D19 related to the frame C synthesized with the item name frame image data D14 corresponding to the item name frame image conversion result data D15 is extracted. ..

Further, when the extracted checked frame image conversion result data D19 is compared with the date format data D5-1 of the extracted item format data D5 stored in the extracted item format data storage unit 126 and the formats match. After the check, the frame image conversion result data D19 is extracted as date data D21-1.

In this case, for example, the description content of the frame C combined with the frame C described as "medical treatment date" or "issue date" may be, for example, a format in which a plurality of numbers are separated by "/" or a plurality of numbers. The description will be extracted if is in the form separated by "year", "month" and / or "day".
In the example of FIGS. 9 and 18, the description content of the frame C below the frame C described as "issue date" is separated by a plurality of numbers by "year", "month" and "day". Since it is in the above format, "August 24, 2018", which is the description content of the frame C, is extracted.

Even if the data corresponding to the item name frame image conversion result data D15 is searched among the checked frame image conversion result data D19 corresponding to the extracted two-frame image data D16, the data that matches the date item data D4-1. If is not found, the control unit 11 searches all of the post-check frame image conversion result data D19 and the post-check out-of-frame image conversion result data D20 for a description that matches the format of the date format data D5-1 and finds the data. If so, the description is extracted as date data D21-1. In this case, first, after checking, the frame image conversion result data D19 is searched for a description that matches the format of the date format data D5-1, and if not found, the post-check frame image conversion result data D20 to the date format data D5 It is preferable to search for a description whose format matches -1.
In the case of the example of FIG. 8, since there is a description in the format in which a plurality of numbers are separated by "year", "month" and "day" in the out-of-frame image conversion result data D20 after the check, the content of the description. "January 17, 2018" is extracted.

[(2) Step S4-2: Extraction of name]
The control unit 11 extracts data related to the name required for the medical expense deduction application from the post-check frame image conversion result data D19 and the post-check out-of-frame image conversion result data D20. Specifically, it is as follows.

First, the control unit 11 searches for the frame image conversion result data D19 after all the checks, and in the search, the name item data D4 included in the extraction item data D4 stored in the extraction item data storage unit 125 of the storage unit 12. When there is data that matches -2, that is, a character string including the character "sama" at the end, the data of the part before "sama" in the character string is extracted.

Further, when the extracted data is compared with the name format data D5-2 of the extracted item format data D5 stored in the extraction item format data storage unit 126 and the formats match, the data is referred to as the name data. Extract as D21-2.

In this case, for example, when the description content including the character "sama" at the end is in the form of a character string that does not include arithmetic numbers or symbols, the description is extracted.
In the example of FIG. 8, in the frame C including the character "sama", the description content before "sama" is in the form of a character string that does not include arithmetic numbers or symbols. "A mountain A one" which is the description content before "sama" of the frame C is extracted.
Further, in the example of FIG. 9, in the frame C including the character "sama", the description content described before "sama" is in the form of a character string that is not an arithmetic number or a symbol. "B village B child" which is the description content before "sama" of the frame C is extracted.
Note that, for example, the name format data D5-2 may be set to exclude alphabets as well, but here, it will be described assuming that alphabets are not excluded.

If the frame image conversion result data D19 after the check is searched, but no data in the format of a character string that includes the character "sama" at the end and matches the format of the name format data D5-2 is found, control is performed. After checking, the part 11 searches the data in the form of a character string including the character "sama" at the end from the out-of-frame image conversion result data D20, and if found, further contrasts with the name format data D5-2. , When the formats match, the data in the part before the "sama" in the description is extracted as the name data D21-2.
[(3) Step S4-3: Extraction of payees]
The control unit 11 extracts data related to the payee required for the medical expense deduction application from the post-check frame image conversion result data D19 and the post-check out-of-frame image conversion result data D20. Specifically, it is as follows.

First, the control unit 11 searches all the post-check frame image conversion result data D19 and the post-check out-of-frame image conversion result data D20, and when there is a description in which the format matches the telephone number format data D5-3. The description is extracted as the description of the telephone number.
For example, a format in which multiple numbers are separated by "-", a format in which a number enclosed in "()" exists between multiple numbers, or a "-" and "()" between multiple numbers. If both of the above are in the existing format, the description will be extracted.

Further, the control unit 11 searches for the medical institution data D4-3 included in the extracted item data D4 stored in the extracted item data storage unit 125 of the storage unit 12 using the description of the extracted telephone number, and then searches for the medical institution data D4-3. When a telephone number that matches the description of the extracted telephone number is found, the medical institution name stored in association with the telephone number is extracted as payee data D21-3.

In the example of FIG. 8, the description "(01) 2345-6789" is extracted as the description of the telephone number, and then the telephone number and the medical institution "A hospital" are displayed in the medical institution data D4-3. When the name is associated and memorized, "A hospital" is extracted.
In the example of FIG. 9, the description "987-654-3210" is extracted as the description of the telephone number, and then the telephone number and the medical institution name "B ophthalmology" are displayed in the medical institution data D4-3. "B ophthalmology" is extracted when is associated and memorized.

[(4) Step S4-4: Extraction of drug name]
The control unit 11 extracts data related to the drug name required for the medical expense deduction application from the post-check frame image conversion result data D19 and the post-check out-of-frame image conversion result data D20. Specifically, it is as follows.

First, the control unit 11 searches the post-check frame image conversion result data D19 and the post-check out-of-frame image conversion result data D20, and from among them, the extracted item data D4 stored in the extracted item data storage unit 125 of the storage unit 12. Search for a description that matches the drug name contained in the drug item data D4-4 included in the above, and if this is found, extract it as the drug data used D21-4.

[(5) Step S4-5: Extraction of payment amount]
The control unit 11 extracts data related to the payment amount required for the medical expense deduction application from the post-check frame image conversion result data D19 and the post-check out-of-frame image conversion result data D20. Specifically, it is as follows.

[Step S4-5-1: Comparison with the first payment item data]
First, the control unit 11 extracts the data corresponding to the two-frame image data D16 from the frame image conversion result data D19 after the check.
Subsequently, the control unit 11 searches for the data corresponding to the item name frame image conversion result data D15 among the checked frame image conversion result data D19 corresponding to the extracted two-frame image data D16, and contains the data corresponding to the item name frame image conversion result data D15. Extracted item data of storage unit 12 Data matching the first payment item data D4-5 included in the extracted item data D4 stored in the storage unit 125, that is, "receipt amount", "billing amount" or "payment amount" When there is a character of, in the two-frame image data D16, the frame image conversion after the check related to the frame C combined with the frame C related to the item name frame image data D14 corresponding to the item name frame image conversion result data D15. Result data D19 is extracted.

Further, when the extracted checked frame image conversion result data D19 is compared with the payment amount format data D5-4 of the extracted item format data D5 stored in the extracted item format data storage unit 126, and the formats match. The numerical portion of the frame image conversion result data D19 after the check is extracted as the payment amount data D21-5.

In this case, for example, the description content of the frame C combined with the frame C described as "receipt amount", "billing amount" or "payment amount" includes, for example, a multi-digit number and is between the numbers. If is in the format described without being separated by "-", "()" or "/", the description of the number will be extracted.
In the example of FIG. 9, the description content of the frame C below the frame C described as "receipt amount" and the description content of the frame C below the frame C described as "billing amount" are multiple digits. Since the numbers include the description in the format described without being separated by "-", "()" or "/", "200" which is the number part of the description of the frame C is extracted.

[Step S4-5-2: Comparison with the second payment item data]
When the payment amount data D21-5 is not extracted in step S4-5-1, the control unit 11 subsequently searches for the frame image conversion result data D19 after all the checks, and in the storage unit 12, the storage unit 12 Data that matches the second payment item data D4-6 included in the extracted item data D4 stored in the extracted item data storage unit 125, that is, the description including the character "¥" at the beginning and the character "yen" at the end. If there is a description that includes, or a description that includes the word "total" at the beginning, the data of the part other than "¥", "yen", or "total" of the description is extracted. In addition, for the description including a plurality of "¥", "yen", and "total", the part other than "¥", "yen", or "total" is similarly extracted.
If the data matching the second payment item data D4-6 is not extracted from the frame image conversion result data D19 after all the checks, then the second payment item from the out-of-frame image conversion result data D20 after the check. Data matching data D4-6 is extracted.

Further, the extracted data is compared with the payment amount format data D5-4 of the extracted item format data D5 stored in the extraction item format data storage unit 126, and when the formats match, the data is paid. Extracted as monetary data D21-5.

In this case, for example, the description including the character "\" at the beginning, the description including the character "yen" at the end, or the description including the character "total" at the beginning "\", "yen" or "total" When the description content of the part other than is, for example, a multi-digit number and the number is described without being separated by "-", "()" or "/", the number is concerned. Will be extracted.

In addition, even if the description includes the characters "\" at the beginning, the characters "yen" at the end, or the characters "total" at the beginning, it does not necessarily indicate the payment amount. The accuracy of the extraction will be inferior to that of step S4-5-1, which is compared with the first payment item data D4-5.
Therefore, in the present embodiment, as described above, the step of contrasting with the first payment item data D4-5 having high extraction accuracy and the second step in which some description can be extracted although the extraction accuracy is inferior. By providing two steps, that is, a step of contrasting with the payment item data D4-6, both the extraction accuracy and the reduction of the possibility that the data is not extracted are achieved.

[(6) Step S4-6: Extraction of medical / treatment points]
The control unit 11 extracts data related to the points of medical care / treatment required for the medical expense deduction application from the frame image conversion result data D19 after the check. Specifically, it is as follows.

First, the control unit 11 extracts the data corresponding to the two-frame image data D16 from the frame image conversion result data D19 after the check.
Subsequently, the control unit 11 searches for the data corresponding to the item name frame image conversion result data D15 among the checked frame image conversion result data D19 corresponding to the extracted two-frame image data D16, and contains the data corresponding to the item name frame image conversion result data D15. Extracted item data of the storage unit 12 Data that matches the medical / treatment item data D4-7 included in the extracted item data D4 stored in the storage unit 125, that is, "medical treatment", "treatment", "needle", "needle", " When there is a character of "inspection fee" or "attendant fee", the two-frame image data D16 is combined with the frame C related to the item name frame image data D14 corresponding to the item name frame image conversion result data D15. After the check related to the frame C, the frame image conversion result data D19 is extracted.

Further, the extracted check-frame image conversion result data D19 is compared with the medical / treatment score format data D5-5 of the extracted item format data D5 stored in the extraction item format data storage unit 126, and the formats match. In this case, the numerical portion of the frame image conversion result data D19 after the check is extracted as the medical / treatment score data D21-6.

In this case, for example, the description content of the frame C combined with the frame C described as "medical treatment", "treatment", "needle", "examination cost" or "attendant fee" is, for example, only numbers or If the number is followed by a "dot", the description of the number will be extracted.
In the example of FIGS. 8 and 17, the description content of the row-by-frame image data D12 located to the right of the row-by-frame image data D12 described as "radiotherapy" is followed by a "dot" after the number. Since it is a format, "0" which is a numerical part of the description content of the frame C is extracted.
Further, in the example of FIG. 9, since the description content of the frame image data D7 located below the frame image data D7 described as "radiotherapy" is in a format in which a "dot" follows a number. “0”, which is the numerical part of the description content of the frame C, is extracted.

[(7) Step S4-7: Extraction of medical product purchase points]
The control unit 11 extracts data related to the number of medical product purchases required for the medical expense deduction application from the frame image conversion result data D19 after the check. Specifically, it is as follows.

First, the control unit 11 extracts the data corresponding to the two-frame image data D16 from the frame image conversion result data D19 after the check.
Subsequently, the control unit 11 searches for the data corresponding to the item name frame image conversion result data D15 among the checked frame image conversion result data D19 corresponding to the extracted two-frame image data D16, and contains the data corresponding to the item name frame image conversion result data D15. Extracted item data of the storage unit 12 Data that matches the medical product purchase item data D4-8 stored in the extracted item data D4 stored in the storage unit 125, that is, "dispensing technology", "pharmaceutical management", "drug", When there is a character of "specified health medical material" or "medical material", the two-frame image data D16 is combined with the frame C related to the item name frame image data D14 corresponding to the item name frame image conversion result data D15. After checking the frame C, the frame image conversion result data D19 is extracted.

Further, the extracted checked frame image conversion result data D19 is compared with the medical product purchase point format data D5-6 of the extracted item format data D5 stored in the extracted item format data storage unit 126, and the formats match. In this case, the numerical portion of the frame image conversion result data D19 after the check is extracted as the medical product purchase point data D21-7.

In this case, for example, the description content of the top C synthesized with the top C described as "dispensing technology", "pharmacy management", "drug", "specified health care material" or "medical material" is, for example, a number. If the format is a chisel or a number followed by a "dot", the description of the number will be extracted.

[(8) Step S4-8: Extraction of long-term care insurance service points]
The control unit 11 extracts data related to the points of the long-term care insurance service required for the medical expense deduction application from the frame image conversion result data D19 after the check. Specifically, it is as follows.

First, the control unit 11 extracts the data corresponding to the two-frame image data D16 from the frame image conversion result data D19 after the check.
Subsequently, the control unit 11 searches for the data corresponding to the item name frame image conversion result data D15 among the checked frame image conversion result data D19 corresponding to the extracted two-frame image data D16, and contains the data corresponding to the item name frame image conversion result data D15. Extracted item data of the storage unit 12 Data matching the care insurance service item data D4-9 stored in the extracted item data D4 stored in the storage unit 125, that is, "home-visit care", "home help service", "visit bathing" When there are characters such as "nursing care", "visit rehabilitation", or "day service", the two-frame image data D16 is combined with the frame C related to the item name frame image data D14 corresponding to the item name frame image conversion result data D15. After the check, the frame image conversion result data D19 related to the frame C is extracted.

Further, the extracted checked frame image conversion result data D19 is compared with the long-term care insurance service score format data D5-7 of the extracted item format data D5 stored in the extracted item format data storage unit 126, and the formats match. In this case, the numerical portion of the frame image conversion result data D19 after the check is extracted as the long-term care insurance service score data D21-8.

In this case, for example, the description content of the frame C synthesized with the frame C described as "home-visit care", "home help service", "visit bathing care", "visit rehabilitation" or "day service" is, for example, If the number is the only number or the number is followed by a "dot", the description of the number will be extracted.

[(9) Step S4-9: Storage of extracted data]
Date data D21-1, name data D21-2, payee data D21-3, drug used data D21-4, payment amount data D21-5, medical / treatment score data D21-6, medical product purchase score data D21-7 When the medical expense deduction application extraction data D21 composed of the nursing care insurance service score data D21-8 is extracted, the control unit 11 applies the data to the medical expense deduction application extraction data storage unit 124 of the storage unit 12. It is stored in association with the user data D1 of the user U related to the data.

The content of the extracted data D21 for medical expense deduction application to be extracted is not limited to the above, and if there is an increase or decrease in the data required for the application due to the medical expense deduction application system, it will be dealt with. Therefore, it can be added or omitted as appropriate.
At this time, with respect to the extracted item data D4 and the extracted item format data D5, the data for new items is added or the data for unnecessary items is deleted according to the extracted data D21 for medical expense deduction application. Will be done.

[(10) Step S4-10: Update of conversion presence / absence data]
When the steps S4-9 are completed, the control unit 11 updates the conversion presence / absence data D3 corresponding to the medical receipt image data D2 of the conversion target data storage unit 122 of the storage unit 12 from unconverted to already converted.

[Explanation of the third effect]
Conversion of the contents of the voucher to the character code using OCR software is usually done in order from left to right and top to bottom of the voucher, so it is a voucher in which the characters are regularly arranged. If there is, it is easy to improve the conversion accuracy, but it was difficult to improve the conversion accuracy for vouchers with irregular descriptions.
In addition, the accuracy of the input image to the OCR software improves as the amount of information other than the character string that causes noise is small and the area where the character string is described is larger than the entire image, but a plurality of entry fields are irregular. In general, there is a lot of information other than the character string and the area where the character string is described is small with respect to the entire image, so that it is difficult to improve the conversion accuracy from this point as well.

In this regard, according to the present embodiment, the medical receipt image data D2 is divided into a plurality of frame image data D7, and each frame image data D7 is converted into a character code by using the OCR software 129. ..
Even if the image data of the voucher (medical receipt image data D2) itself has a plurality of entry fields arranged irregularly, the description in each frame C is regular compared to the entire voucher. Also, when looking inside each frame C, there is less information other than the character string that causes noise, and the area where the character string is written is larger than the case of looking at the entire voucher. There are many.
Therefore, according to the present embodiment, the conversion accuracy by the OCR software 129 can be improved by converting each frame image data D7 into a character code.

Further, according to the present embodiment, the out-of-frame image data D9 excluding the frame image data D7 portion is created from the medical receipt image data D2, and the OCR software 129 is also used for this as well, separately from the frame image data D7. Since the conversion to the character code is performed, the conversion accuracy by the OCR software 129 can be improved also in the region outside the frame C by reducing noise and the like.

Further, based on the frame image conversion result data D10 which is the conversion result of the frame image data D7, the item name frame image data D14 which is the frame image data D7 including a predetermined item name is extracted, and the frames adjacent thereto are extracted. By synthesizing the frame image data D7 of C to create the two-frame image data D16, it is easy to determine the correspondence between the description of the predetermined item name and the corresponding description content in the description of the voucher. Can be done.

Further, based on the frame image conversion result data D10 which is the conversion result of the frame image data D7, the number of lines of the character string included in the frame image data D7 is specified, and the frame image data D7 includes a plurality of lines of character strings. In this case, the frame image data D7 is divided into rows to create row-to-frame image data D12, and the frame C divided into row-to-frame image data D12 is divided into row-by-line frame image data D12. By creating the frame image data D16, it is possible to easily determine the correspondence between the description of the predetermined item name and the corresponding description content even for the frame C including the character strings of a plurality of lines.

Further, the divided / combined frame image data D17, which is the data of the frame image after the predetermined division / compositing process, and the divided / combined frame image conversion result data D18, which is the corresponding conversion result data, are divided / combined. By transmitting each post-frame image data D17 to a separate worker terminal 2 and having the worker W confirm and correct it, the conversion accuracy is improved while concealing the entire image of the contents of the voucher from the worker W. Can be done.

Further, by using the data (extracted item data D4) for a predetermined item that needs to be extracted from the medical receipt image data D2, the data for the item that matches the data is extracted, and the medical receipt image data is obtained. From D2, it becomes easy to extract the data required for a predetermined application, etc. (in this embodiment, a medical expense deduction application).

At this time, the item name frame image data D14 and the frame image data D7 of the adjacent frame C are combined to create the two-frame image data D16, which is extracted into the item name frame image data D14. When an item name matching a specific item is included in the item data D4, data can be extracted from the frame image conversion result data D10 related to the frame image data D7 combined with the item name frame image data D14, and predetermined data can be extracted. It will be easier to extract the data required for the application, etc. (in this embodiment, the medical expense deduction application).

Further, when the data for a specific item among the extracted item data D4 is not extracted from the two-frame image data D16, the frame image conversion result data D10 related to the frame C for which the two-frame image data D16 is not created is included. By extracting the data for the item that matches the extracted item data D4, first, while trying to extract from the two-frame image data D16 whose extraction accuracy is higher than that of the data of other parts, the two-frame image It is possible to extract data related to items that could not be extracted from the data D16.

Further, when the extraction item data D4 is used to extract the data for the item that matches the data, it is clarified by extracting only the data in the format that matches the extraction item format data D5. It is possible to prevent the extraction of data whose formats do not match, and improve the accuracy of data extraction.

Further, when the data for a specific item among the extracted item data D4 is not extracted from the frame image conversion result data D10, the extracted item data D4 includes the out-of-frame image conversion result data D11 related to the out-of-frame image data D9. By extracting the data for the matching items, first, while trying to extract from the frame image data D7 whose extraction accuracy is higher than that of the part other than the frame C, it could not be extracted from the frame image data D7. It is also possible to extract data related to items.

[Fourth variant]
In the above embodiment, the case where the description content of the medical receipt image data D2 is converted into a character code has been described, but the voucher that is the subject of the present invention is not limited to the medical receipt.
If the voucher is provided with a plurality of entry fields, for example, a voucher other than a medical receipt such as a salary statement, a social insurance payment slip, etc., according to the present invention, the frame image data D7 and the out-of-frame image data D9 By performing the conversion by the OCR software 129 after the division, the conversion accuracy of the described content into the character code can be improved.

Further, in the above embodiment, the case of extracting the data related to the items required for the medical expense deduction application has been described, but the items to be extracted after converting the contents of the voucher into the character code are not limited to this. If the required items are clear data required for applications, etc., by configuring the extracted item data accordingly, data related to items required for applications other than medical expense deduction applications, etc. It is also possible to extract.

Further, in the above embodiment, the case where the check by the cloud worker (step S3) is included has been described. However, although the conversion accuracy is lowered, the conversion data storage unit of the storage unit 12 in step S2 does not go through the process. It is also possible to extract the medical expense deduction application data from the divided / combined frame image conversion result data D18 and the out-of-frame image conversion result data D11 stored in 123.

100 Accounting support system 1 Accounting support device 11 Control unit (frame identification means, frame image data creation means, conversion means, out-of-frame image data creation means, item name frame extraction means, two-frame image data creation means, line number specifying means, line Image data creation means for each frame, data extraction means)
12 Memory 127 Medical Receipt Conversion Engine (Accounting Support Program)
128 Data extraction engine for medical expense deduction application (accounting support program)
129 OCR software 13 Communication unit (acquisition means, transmission means, reception means)
2 Worker terminal D2 Medical receipt image data (voucher image data)
D4 Extracted item data D5 Extracted item format data D7 Frame image data D9 Out-of-frame image data D10 Frame image conversion result data D11 Out-of-frame image conversion result data D12 Line-by-frame image data D14 Item name Frame image data D16 Two-frame image data L Border C frame W worker

Claims (17)

An acquisition means for acquiring voucher image data, which is the image data of the voucher to be converted, and
A frame identification means for identifying a frame that is an area surrounded by a ruled line from the voucher image data,
A frame image data creation means for creating frame image data, which is image data obtained by extracting the frame portion from the voucher image data, and
A conversion means for converting the frame image data into a character code and acquiring the frame image conversion result data using OCR software, and
An accounting support device characterized by being equipped with. The frame identification means
After detecting a straight line from the voucher image data,
The accounting support device according to claim 1, wherein the area surrounded by the straight line is identified as the frame. An out-of-frame image data creation means for creating out-of-frame image data, which is image data obtained by deleting the description of the frame portion from the voucher image data, is provided.
The accounting support device according to claim 1 or 2, wherein the conversion means uses OCR software to convert the out-of-frame image data into a character code and obtains the out-of-frame image conversion result data. Using the frame image conversion result data, it is determined whether or not the frame image data includes a predetermined item name, and the item name frame image data which is the frame image data including the predetermined item name is extracted. The accounting support device according to any one of claims 1 to 3, further comprising an extraction means. The accounting support device according to claim 4, further comprising a two-frame image data creating means for creating two-frame image data by synthesizing frame image data of adjacent frames with the item name frame image data. The accounting according to any one of claims 1 to 5, further comprising a line number specifying means for specifying the number of lines of a character string included in the frame image data based on the frame image conversion result data. Support device. A claim characterized in that, when the frame image data includes a character string of a plurality of lines, a line-by-line image data creating means for creating the line-by-line frame image data obtained by dividing the frame image data into lines is provided. The accounting support device according to 6. Using the frame image conversion result data, it is determined whether or not the frame image data includes a predetermined item name, and the item name frame image data which is the frame image data including the predetermined item name is extracted. Equipped with extraction means
The item name frame extraction means
With respect to the frame in which the line-by-line frame image data is created, it is determined whether the line-by-line frame image data includes a predetermined item name, and the line-by-frame image data including the predetermined item name is used as the item. Extracted as name frame image data,
For frames for which frame-by-line image data has not been created, it is determined whether the frame image data includes a predetermined item name, and the frame image data including the predetermined item name is used as the item name frame image. The accounting support device according to claim 7, wherein the data is extracted as data. A two-frame image data creation means for creating two-frame image data by synthesizing frame image data of adjacent frames with the item name frame image data is provided.
The two-frame image data creation means
When the line-by-frame image data is extracted as the item name frame image data,
After determining whether the number of rows of the frame containing the frame-by-line image data matches the number of adjacent frames, if they match, the frame-by-line image data and the frame adjacent to the right are the same. Combine the line-by-line frame image data
When the frame image data is extracted as the item name frame image data,
After determining whether the widths of the frame related to the frame image data and the adjacent frame below match in the left-right direction, if they match, the frame image data and the frame image data of the adjacent frame below The accounting support device according to claim 8, wherein the data and the data are combined with each other. A transmission means for transmitting the frame image data and the frame image conversion result data corresponding to the frame image data to a separate worker terminal for each frame image data.
A receiving means for receiving the frame image conversion result data after confirmation and correction by the worker from the worker terminal, and
The accounting support device according to any one of claims 1 to 9, further comprising. Any one of claims 1 to 10, further comprising a data extraction means for extracting data for an item that matches the extraction item data, which is data related to a predetermined extraction item, from the frame image conversion result data. Accounting support device described in. A data extraction means for extracting data for items that match the extraction item data, which is data related to a predetermined extraction item, from the frame image conversion result data is provided.
When the frame image conversion result data corresponding to the item name frame image data of the two-frame image data includes an item name matching the extracted item data, the data extraction means synthesizes the item name frame image data. The accounting support device according to claim 5 or 9, wherein the frame image conversion result data related to the frame image data is extracted. When the frame image conversion result data corresponding to the item name frame image data of the two-frame image data does not include an item name matching the extracted item data, the data extraction means all the frame image conversion results. The accounting support device according to claim 12, wherein data about an item matching the extracted item data is extracted from the data. The data extraction means selects data whose description format matches the extraction item format data, which is the data related to the description format of the data extracted by the data extraction means, among the data about the items matching the extraction item data. The accounting support device according to any one of claims 11 to 13, characterized in that it is extracted. From the frame image conversion result data, data for items that match the extraction item data, which is data related to a predetermined extraction item, is extracted.
Data extraction means for extracting data for items that match the extracted item data from the out-of-frame image conversion result data when data for items that match the extracted item data is not extracted from the frame image conversion result data. The accounting support device according to claim 3, wherein the data is provided. Computer,
Acquisition means for acquiring voucher image data, which is the image data of the voucher to be converted,
A frame identification means for identifying a frame that is an area surrounded by a ruled line from the voucher image data.
A frame image data creation means for creating frame image data which is image data obtained by extracting the frame portion from the voucher image data.
A conversion means for converting the frame image data into a character code and acquiring the frame image conversion result data using OCR software.
Accounting support program to function as. The acquisition step to acquire the voucher image data, which is the image data of the voucher to be converted,
A frame identification step for identifying a frame that is an area surrounded by a ruled line from the voucher image data,
A frame image data creation step for creating frame image data, which is image data obtained by extracting the frame portion from the voucher image data, and
A conversion step of converting the frame image data into a character code and acquiring the frame image conversion result data using OCR software, and
Accounting support methods including.

Images