KR102455536B1 - Receipt recognition device, a method for automatically recognizing medical items from scanned images of receipts, and computer programs - Google Patents

Abstract

The present specification includes the steps of: obtaining a scanned image of a receipt by a receipt recognition device for claiming insurance for medical expenses; extracting, by the receipt recognition device, one or more detailed item areas from the scanned image of the receipt using an object detection algorithm; determining, by the receipt recognition device, a name of a detailed item included in a detailed item area using a name detection algorithm; determining, by the receipt recognition device, a medical treatment amount area for each detailed item name by using the previously registered detailed item related information; Disclosed is a method for automatically recognizing a medical treatment item from a scanned image of a receipt, further comprising; extracting, by the receipt recognizing device, a medical treatment amount for a detailed item included in the receipt using an amount detection algorithm.

Description

Receipt recognition device, a method for automatically recognizing medical items from scanned images of receipts, and computer programs

The present disclosure relates to a receipt recognition device, a method for automatically recognizing a medical treatment item from a scanned image of a receipt, and a computer program.

In general, a user who claims insurance pays for insurance by directly entering the amount of medical treatment and attaching a scanned image of the receipt as proof. When an insurance claim input is received from the user, the manager of the insurance company confirms the input insurance claim. The manager of the insurance company checks the amount of the attached receipt to see if it matches the billed amount, and checks the claim details to see if it is subject to insurance claim.

As the number of insurance claims increases, the amount of work to confirm the amount of claims increases, and the work of insurance company managers increases. By recognizing the text of the attached receipt using text reading technology (OCR), the problem was secured. However, since the quality of the attached receipts varies widely, there are many errors in recognizing the amount of treatment and treatment item information from the scanned image of the attached receipt. In this regard, we propose a technology for automatically recognizing the medical treatment item and the medical treatment amount of the scanned image of the receipt.
Prior document KR 10-1139801 (published on April 30, 2012)

An object of the present invention is to provide a receipt recognition device, a method for automatically recognizing a medical treatment item from a scanned image of a receipt, and a computer program.

Receipt recognition method according to embodiments of the present invention comprises the steps of: obtaining, by a receipt recognition device for claiming insurance money for medical expenses, a scanned image of the receipt; extracting, by the receipt recognition device, one or more detailed item areas from the scanned image of the receipt using an object detection algorithm; determining, by the receipt recognition device, a name of a detailed item included in a detailed item area using a name detection algorithm; determining, by the receipt recognition device, a medical treatment amount area for each detailed item name by using the previously registered detailed item related information; The receipt recognition device may automatically recognize the medical treatment item from the scanned image of the receipt, further comprising a; extracting the medical treatment amount for the detailed item included in the receipt by using the amount detection algorithm.

The determining of the detailed item name may include extracting a feature value by recognizing the text of the detailed item area as an image, and determining a detailed item name corresponding to the detailed item area based on the extracted feature value.

In the step of extracting the amount of medical expenses, in the detailed item area, the treatment amount area is detected, one or more numbers included in the treatment amount area are respectively detected as a number area, and after each number in each number area is recognized, You can determine the amount of treatment by adding up each recognized number.

In the method, after obtaining the scanned image, the horizontal axis or vertical axis of the scanned image is extracted, and the extracted horizontal axis and vertical axis are compared with a reference horizontal axis and a reference vertical axis, respectively, to determine the degree of inclination of the scanned image, and The method may further include determining a degree of compensation for compensating the degree, and correcting the direction of the scanned image by the degree of compensation.

In the step of extracting the one or more detailed item areas, a format of the receipt is determined using one or more identifiers included in the scanned image, and the receipt is scanned based on a format corresponding to the receipt. The method may further include detecting a patient classification area and a treatment item area in the image, detecting a 'payment amount area' in the treatment item area, and determining a total amount in the 'payment amount area area' .

After detecting the patient division area and the treatment item area, in the patient division area, the item area of 'patient name' is detected, and the item area of 'patient name' is recognized using a character reading algorithm It may further include the step of

The method according to an embodiment of the present invention determines whether the total amount and the billing amount input from the user match, and if the total amount matches the input billing amount, the treatment amount for the detailed items included in the extracted receipt may further include; registering as the medical treatment amount for the receipt.

The method according to the embodiments of the present invention determines whether the total amount and the billing amount input from the user match, and if the total amount does not match the input billing amount, the extracted detailed items included in the receipt The method may further include transmitting a request for confirmation of the amount of treatment to the manager account.

Receipt recognition apparatus according to embodiments of the present invention is a receipt acquisition unit for acquiring a scanned image of the receipt; a detailed item detection unit that extracts one or more detailed item areas from the scanned image of the receipt using an object detection algorithm, and determines a name of a detailed item included in the detailed item area using a name detection algorithm; A treatment amount recognition unit for determining the treatment amount area for each detailed item name using the previously registered detailed item related information, and extracting the treatment amount for the detailed item included in the receipt using an amount detection algorithm; may include

The detailed item detection unit may recognize the text of the detailed item area as an image, extract a feature value, and determine a detailed item name corresponding to the detailed item area based on the extracted feature value.

The treatment amount recognition unit detects a treatment amount area from among the detailed item areas, detects one or more numbers included in the treatment amount area as a number area, recognizes each number in each number area, and then recognizes each You can determine the amount of treatment by adding up the numbers.

The area detection unit extracts the horizontal or vertical axis of the scanned image, compares the extracted horizontal and vertical axes with a reference horizontal axis and a reference vertical axis, respectively, to determine the degree of inclination of the scanned image, and to determine the degree of compensation for compensating for the degree of inclination. , the direction of the scanned image may be corrected by the compensation degree.

The area detection unit determines a format of the receipt using one or more identifiers included in the scanned image, and a patient classification area in the scanned image of the receipt based on a format corresponding to the receipt. and detecting a treatment item area, detecting a 'payment amount area' in the treatment item area, and determining a total amount in the 'payment amount area area'.

After detecting the patient division area and the treatment item area, the area detection unit detects an item area of 'patient name' from among the patient division area, and uses a character reading algorithm to convert the item area of 'patient name' to the patient can recognize the name of

The area detection unit determines whether the total amount and the billing amount input from the user match, and if the total amount matches the input billing amount, the extracted medical treatment amount for the detailed items included in the receipt for the receipt You can register for the treatment amount.

The area detection unit determines whether the total amount and the billing amount input from the user match, and if the total amount does not match the input billing amount, check the treatment amount for the detailed items included in the extracted receipt You can forward the request to the admin account.

A computer program according to an embodiment of the present invention may be stored in a medium to execute any one of the methods according to an embodiment of the present invention using a computer.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium for recording a computer program for executing the method are further provided.

Other aspects, features and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to the present invention, the legality of the insurance claim can be easily determined by converting the medical treatment items and the medical treatment amount included in the scanned image of the receipt attached to the insurance claim into a digital format with 99% accuracy.

1 is a block diagram of an insurance claim system 10 according to embodiments of the present invention.
2 is a block diagram of an insurance claim apparatus 100 according to embodiments of the present invention.
3 to 6 are flowcharts of a receipt recognition method according to embodiments of the present invention.
7 to 9 are diagrams for explaining a separator, a horizontal axis, a vertical axis, a patient division area, a treatment item area, etc. extracted with respect to a scanned image of a receipt.
10 is a diagram for explaining the operation of the amount detection algorithm.
11 is a diagram for explaining an operation of a name detection algorithm.

Hereinafter, the configuration and operation of the present invention will be described in detail with reference to the embodiments of the present invention shown in the accompanying drawings.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and a method for achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when described with reference to the drawings, the same or corresponding components are given the same reference numerals, and the overlapping description thereof will be omitted. .

In this specification, terms such as "learning" and "learning" are not intended to refer to mental actions such as human educational activities, but are terms referring to performing machine learning through computing according to procedures. interpret

In the following embodiments, terms such as first, second, etc. are used for the purpose of distinguishing one component from another, not in a limiting sense.

In the following examples, the singular expression includes the plural expression unless the context clearly dictates otherwise.

In the following embodiments, terms such as include or have means that the features or components described in the specification are present, and the possibility that one or more other features or components may be added is not excluded in advance.

In the drawings, the size of the components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily indicated for convenience of description, the present invention is not necessarily limited to the illustrated bar.

In cases where certain embodiments are otherwise practicable, a specific process sequence may be performed different from the described sequence. For example, two processes described in succession may be performed substantially simultaneously, or may be performed in an order opposite to the order described.

1 is a block diagram of an insurance claim system 10 according to embodiments of the present invention.

The insurance claim system 10 may include an insurance claim device 100 , a claimant terminal 200 , a manager terminal 300 , and a database 400 .

The insurance claim device 100 is a device that electronically implements a procedure necessary for claiming insurance money, such as hospital expenses and medical expenses, and provides it to the claimant's terminal 200 . The insurance claim device 100 may be a desktop computer, but is not limited thereto, and may be any type of electronic device capable of executing a program for performing a method for claiming insurance money.

The insurance claim device 100 automatically recognizes the billing amount, the treatment amount for each treatment item, etc. from the scanned image of the receipt using the item detection model and the item recognition model learned through the scanned image of the receipt, the item areas for the scanned image, and the like. can do.

The insurance claim device 100 recognizes the detailed item area as an image by recognizing the horizontal axis and the vertical axis included in the receipt, and uses the detailed item name detection model learned from training data of the detailed item areas and the detailed item name. The item name can be detected.

The insurance claim device 100 may automatically recognize the scanned image of the receipt to determine whether the claim amount of the insurance claim is the same as the amount of the receipt.

In this way, the insurance company can save the trouble of determining the suitability of the claim amount by comparing the amount of the receipt with the claimed insurance amount.

A claimant making a claim for insurance may access the insurance claim device 100 through the claimant terminal 200 to perform an insurance claim procedure.

If the insurance claim device 100 detects the claim details from the scanned image of the receipt and does not match the claim amount in the insurance claim input, the insurance claim device 100 transmits the scanned image of the receipt to the manager terminal 300 to receive the receipt. You can request confirmation of the scanned image of

The manager may determine whether the insurance claim amount is appropriate by checking the insurance claim input and the attached receipt in the manager terminal 300 , and may input the determined suitability through the manager terminal 300 .

The database 400 may store and manage a scanned image of a receipt attached to an insurance claim. The database 400 may store area information of the patient classification area and area information of the treatment item area (position value, coordinate value, size value, etc.) included in the scanned image of the receipt for each format of the receipt. The database 400 may store information (position, detailed item name, etc.) on the identifier used to determine the format of the receipt. The database 400 may recognize each number of medical expenses included in the receipt as an image, and store the corresponding table by matching the image of each number with each number. The database 400 may store a character reading algorithm that reads characters by recognizing the 'patient name' of the receipt as text. When the format of the corresponding text, such as 'patient name', is not defined in a certain number of formats but is determined in one of an infinite number of formats, the corresponding content can be read using a character reading algorithm. When it is determined as one of a plurality of predetermined formats, the content can be read using a detection algorithm. The detection algorithm may be an algorithm learned from training data of a scanned image of a receipt and corresponding information thereto, and may be continuously learned through continuously input training data. The database 400 may manage information about the manager terminal 300 that determines suitability of claiming insurance money.

Although the claimant terminal 200 and the manager terminal 300 are illustrated in the singular in FIG. 1 , the present invention is not limited thereto and may be included in a plurality.

2 is a block diagram of an insurance claim apparatus 100 according to embodiments of the present invention.

The processor 110 is a configuration for controlling the insurance claim device 100 as a whole. Specifically, the processor 110 controls the overall operation of the insurance claim device 100 by using the receipt recognition device 150 of the insurance claim device 100 or various programs stored in the storage medium. For example, the processor 110 may include a CPU, a RAM, a ROM, and a system bus. Here, the ROM is a configuration in which an instruction set for system booting is stored, and the CPU copies the stored operating system of the insurance claim device 100 to the RAM according to the instructions stored in the ROM, and executes an O/S to boot the system. When booting is complete, the CPU can perform various operations by copying various stored applications to RAM and executing them. Although it has been described above that the insurance claim device 100 includes only one CPU, it may be implemented with a plurality of CPUs (or DSPs, SoCs, etc.).

According to an embodiment of the present invention, the processor 110 may be implemented as a digital signal processor (DSP) processing a digital signal, a microprocessor, or a time controller (TCON). However, the present invention is not limited thereto, and a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), or It may include one or more of a communication processor (CP) and an ARM processor, or may be defined by a corresponding term. In addition, the processor 110 may be implemented as a system on chip (SoC), large scale integration (LSI), or a field programmable gate array (FPGA) type having a built-in processing algorithm.

The output unit 140 may display an interface generated by the insurance claim device 100 by the receipt recognition device 150 . According to an embodiment of the present invention, the output unit 140 may display a user interface for a user input input through the input unit 130 . The output unit 140 may output the stored graphic data, visual data, auditory data, and vibration data under the control of the processor 110 or execution of instructions stored in the receipt recognition device 150 or other storage medium.

The input unit 130 may include a user interface for inputting various information into the insurance claim device 100 .

The communication unit 120 is configured to transmit/receive data to and from devices such as a server and other electronic devices. The communication unit 120 includes a Bluetooth communication unit, a BLE (Bluetooth Low Energy) communication unit, a near field communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared (IrDA, infrared Data Association) communication unit, a WFD ( It may include a Wi-Fi Direct) communication unit, an ultra wideband (UWB) communication unit, a short-distance communication unit such as an Ant+ communication unit, a mobile communication network, or a wired Ethernet network.

The insurance claim device 100 may further include a receipt recognition device 150 storing various data for overall operations, such as a program for processing or controlling the processor 110 . The receipt recognition device 150 may specifically store a plurality of application programs (application programs or applications) driven in the insurance claim device 100 , data for the operation of the insurance claim device 100 , and commands. . At least some of these application programs may be downloaded from an external server through wireless communication. Also, at least some of these application programs may exist on the insurance claim device 100 from the time of shipment for a basic function of the insurance claim device 100 . The application program may be stored in a storage medium and driven to perform an operation (or function) of the insurance claim device 100 by the processor 110 .

When an insurance claim with a receipt is received, the insurance claim device 100 may transmit the attached receipt to the receipt recognition device 150 to extract detailed data related to the receipt. The receipt recognition device 150 may determine whether the information input to the insurance claim signal matches the detailed data, and if they match, the insurance claim may be processed without additional confirmation. If they do not match, the scan image of the attached receipt may be transmitted to the manager terminal 300 so that an additional confirmation operation is performed. The insurance claim device 100 may process an insurance claim signal and/or detailed data confirmation of a receipt by using a program installed in the claimant terminal 200 and/or the manager terminal 300 . The receipt recognition device 150 may firstly determine whether the total billing amount is the same, and secondarily, determine whether the treatment amount of the detailed treatment details matches. If the firstly checked content does not match, a confirmation request may be made to the manager terminal 300 , or if the secondly checked content does not match, a confirmation may be requested from the manager terminal 300 .

The receipt recognition device 150 includes a receipt acquisition unit 151, an area detection unit 152, a total confirmation unit 153, and a detailed item detection unit ( 154 ), a medical treatment amount recognition unit 155 , and an insurance claim registration unit 156 .

The receipt acquisition unit 151 acquires an insurance claim signal input from the claimant terminal 200 and a scanned image of the receipt. The insurance claim signal may include a disease name, an insurance claim amount, a treatment date, a treatment hospital, and a patient name. The claimant may input the disease name, insurance claim amount, treatment date, treatment hospital, patient name, etc., which are information necessary for claiming insurance through the claimant terminal 200 .

The area detection unit 152 determines the format of the receipt by using one or more identifiers included in the scanned image of the receipt. Here, the identifier refers to an item essential to a hospital receipt. The delimiter may be designated by the position of the item or the name of the item. For example, when 'patient name' is used as a delimiter, the delimiter may be searched for from a scanned image of a receipt as a position value of 'patient name'. The area detection unit 152 may scan the entire scanned image of the receipt to extract a previously registered identifier. In this case, the algorithm for extracting the delimiter may be an object detection algorithm for extracting an object from the image. The area detection unit 152 may determine the format of the receipt based on the position value of the separator detected in the scanned image of the receipt. Information on the format of the receipt may be recorded for each position of the separator.

The area detection unit 152 may detect a patient classification area (see PDA of FIG. 8 ) and a treatment item area (see DDA of FIG. 8 ) in the scanned image of the receipt based on a format corresponding to the receipt. The locations of the patient identification area (see PDA of FIG. 8 ) and treatment item area (see DDA of FIG. 8 ) may be stored in a table corresponding to the formats of receipts of insurance claims. A patient classification area (refer to the PDA of FIG. 8 ) and a treatment item area (refer to the DDA of FIG. 8 ) in the scan image may be detected using the table.

The sum check unit 153 extracts the total amount area (see TAA of FIG. 8) from the scanned image of the receipt, recognizes each number included in the total amount area, and adds each recognized number to detect the total amount. . The total check unit 153 may determine the total amount area (refer to TAA of FIG. 8 ) by using the patient classification area and the treatment item area of the receipt. Also, the total check unit 153 may determine the total amount area (refer to TAA of FIG. 8 ) based on the format of the receipt. The total amount area may be determined using a table in which position values of the total amount area are recorded for each format of the receipt. The amount detection model , which is an algorithm for detecting an amount in the total amount area, will be described in detail with reference to FIG. 10 .

The total check unit 153 may compare the detected total amount with the input billing amount to determine whether they match. The sum check unit 153 may return whether the match is true or false. In an optional embodiment, when it is determined by the total check unit 153 that the total amount and the claim amount match, the corresponding insurance claim signal may be processed as a 'examination complete' state for claiming insurance money.

The total check unit 153 may determine an area corresponding to 'patient name'. The sum check unit 153 may determine whether the inputted name matches the contents corresponding to the 'patient name'. The 'patient name' can be read through a character-reading algorithm because there can be an infinite number of combinations of contents.

After detecting the patient division area and the treatment item area, the sum check unit 153 detects the item area of 'patient name' from among the patient division area, and uses a character reading algorithm for the item area of 'patient name' Can read the patient's name. 'Patient name' is an area where the object detection algorithm cannot be used because it is a combination of various texts. Therefore, for 'patient name', the patient name can be extracted using a character reading algorithm. In addition to the 'patient name', the names of items that are not selected as one of the candidate data may be extracted using a character reading algorithm.

The detailed item detection unit 154 may detect the detailed item area from the scanned image of the receipt and determine the detailed name of the detailed item area. The detailed item detection unit 154 may extract detailed item areas from the medical treatment item area (see DDA of FIG. 8 ) of the scanned image of the receipt, and determine a corresponding detailed item name from the detailed item area. The sub-item detection unit 154 may detect sub-item areas as shown in DDA-P1 to DDA-P15 of FIG. 9 . The detailed item areas may be extracted based on information on the previously registered detailed item areas for each format by scanning the entire detailed item area in the scanned image of the receipt.

The sub-item detection unit 154 may determine sub-item names corresponding to the sub-item areas by using a name detection learning model learned by recognizing one or more texts included in the sub-item areas as a whole as a single object. As shown in FIG. 11, the name detection learning model can learn the input data of the detailed item area obtained from the scan images of various receipts, such as TD1, TD2, TD3, and TD4, as the result data of the NAME. The name detection learning model may be implemented to output the detailed item name by extracting the feature value of the detailed item through an object detection algorithm, which is an image processing algorithm. The name detection learning model may be implemented to output the detailed item name using a classification model based on the feature value of the object rather than OCR-based.

The treatment amount recognition unit 155 reads the treatment amount for each detailed item of the receipt. The treatment amount recognition unit 155 may read the treatment amount for each detailed item by using the amount detection algorithm . The treatment amount recognition unit 155 may detect the area on the right side of the detailed item area as the treatment amount area, detect areas of numbers in the treatment amount area, and read the detected areas using an amount detection algorithm. The treatment amount recognition unit 155 may read the treatment amount for each detailed item through the above process. The read medical treatment amount may be recorded for each detailed item.

The amount detection algorithm may be implemented to scan the entire treatment amount areas, select areas in which the number '0' is recorded, and read the treatment amount in the remaining areas except for areas in which '0' is recorded. Whether '0' is a recorded area may be determined through a learning model that repeatedly learns the medical treatment amount areas in which '0' is recorded.

The insurance claim registration unit 156 may register the insurance claim signal from the claimant terminal 200, the scanned image of the receipt, and the treatment amount for each detailed item obtained from the detailed item detection unit 154 and the treatment amount recognition unit 155. .

The insurance claim registration unit 156 reads the insurance claim signal, the scanned image of the receipt, and the detailed item detection unit 154 and the medical treatment amount to the manager terminal 300 when the entered claim amount and the total amount obtained from the scanned image do not match. The unit 155 may request confirmation of the treatment amount for each detailed item obtained. When receiving the confirmation signal from the manager terminal 300 , the insurance claim registration unit 156 may register the treatment amount for each detailed item input by the manager terminal 300 together with the insurance claim signal.

3 to 6 are flowcharts of a receipt recognition method according to embodiments of the present invention.

In S110, the receipt recognition device 150 acquires a scanned image of the receipt.

In S120, the receipt recognizing apparatus 150 may scan the entire scan image and detect one or more detailed item areas. An algorithm for detecting the sub-item regions may use an image analysis algorithm, and an algorithm based on an object detection algorithm in an image may be used.

In S130, the receipt recognizing apparatus 150 may determine the corresponding detailed item names by applying a name detection algorithm to the detected one or more detailed item areas.

In S140 , the receipt recognizing apparatus 150 may determine the medical treatment amount related to the detailed item area using the amount detection algorithm. The amount detection algorithm may be implemented to scan the entire treatment amount areas, select areas in which the number '0' is recorded, and read the treatment amount in the remaining areas except for areas in which '0' is recorded. Whether '0' is a recorded area may be determined through a learning model that repeatedly learns the medical treatment amount areas in which '0' is recorded.

In S150, the receipt recognition device 150 may extract a medical treatment amount for a detailed item included in the receipt. The receipt recognition device 150 may request confirmation by generating the treatment amounts for the extracted detailed items as a table and transmitting them to the manager terminal 300 . The manager terminal 300 may compare the scanned image of the receipt and the treatment amounts for detailed items to determine whether they match, record whether they match, and reply.

The insurance claim device 100 may manage the medical treatment amounts for detailed items of the receipt obtained through the receipt recognition device 150 together with the insurance claim signal. The insurance claiming apparatus 100 may manage whether or not the correspondence returned from the manager terminal 300 corresponds to the receipt. The medical treatment amounts for the detailed items of the receipt obtained through the receipt recognition device 150 and whether or not the amount of treatment returned from the manager terminal 300 match may be used to determine the legitimacy of the insurance claim signal.

In S210, the receipt recognizing apparatus 150 may detect detailed item areas from the scanned image of the receipt by using an object detection algorithm that is an image processing method. The sub-item areas may be detected based on the feature value of the sub-item areas.

In S220, the receipt recognition apparatus 150 may extract a feature value by recognizing the text of the detailed item area as a whole. In this case, the feature values may be feature values corresponding to name detection in the name detection algorithm. The name detection algorithm can be learned using machine learning and deep learning, and can be designed to be continuously updated by learning from training data. As shown in FIG. 11 , the name detection algorithm may learn the input data of the detailed item area obtained from the scan images of various receipts, such as , TD2, TD3, and TD4, as the result data of the fee (NAME).

In S230, the receipt recognizing apparatus 150 may determine a detailed item name corresponding to the detailed item area from among the candidate detailed item names based on the extracted feature values.

In S310, the receipt recognizing apparatus 150 may detect a detailed item area from the scanned image of the receipt by using an object detection algorithm.

In S320, the receipt recognizing apparatus 150 may detect the medical treatment amount area in relation to the detailed item area.

In S330, the receipt recognition apparatus 150 may re-classify one or more digits included in the area related to medical expenses into each number area. For example, in relation to the medical treatment amount of '123,000', the numerical fields of 1, 2, 3, 0, 0, and 0 may be individually distinguished. Each digit can be extracted from the divided areas using an amount detection algorithm.

In S340, the receipt recognizing apparatus 150 may determine a medical treatment amount by recognizing each number in each number area, and adding each recognized number. As described above, after extracting 1, 2, 3, 0, 0, and 0 from each area, '123000' may be determined as the treatment amount.

In S410, the receipt recognition device 150 detects a region corresponding to a previously registered identifier in the scanned image of the receipt.

In S420, the receipt recognition device 150 may detect a previously registered identifier (see INX1, INX2, INX3 in FIG. 7) and determine the format of the receipt based on the position of the detected identifier. Receipts input at the time of claiming insurance money are input by the claimant and have different formats depending on receipt output devices provided in pharmacies, hospitals, and the like. Accordingly, it is possible to create a table in which coordinates, locations, names, etc. of areas corresponding to the same item in receipts obtained through insurance claims are recorded. The format of the identifier and receipt can be determined through the table created in this way.

In S430, the receipt recognizing apparatus 150 may extract a patient classification area and a medical treatment item area in the scanned image of the receipt by using the area information registered with respect to the format of the receipt. Before extracting the patient classification area and the treatment item area, the body part of the receipt may be determined first. Areas related to the amount of treatment, treatment history, and patient information may be designated in advance by designating one body part except for areas in which parking permits, treatment reservation information, and examination-related information included in the receipt are recorded.

In S440, the receipt recognizing apparatus 150 may detect the 'payable amount area' in the medical treatment item area and determine the total amount related to the 'payable amount area' using the amount detection algorithm.

The 'payable amount area' may be replaced with another area, and may be an area registered as the total amount area.

In an optional embodiment, before S410, the receipt recognition device 150 may further perform a process of changing the direction of the scanned image of the receipt or correcting the wrong part. More specifically, axes in the x-axis direction and the y-axis direction may be extracted from the scanned image of the receipt. 7, the receipt recognition device 150 extracts P1 and P2 in the x-axis direction, extracts X1 by connecting P1 and P2, extracts P1, P3 in the y-axis direction, and extracts P1, P3 Y1 connected to can be extracted. Based on the inclination of X1 and the reference x-axis and the inclination of Y1 and the reference y-axis, the degree of inclination of the scanned image of the receipt is measured, and the measured degree of inclination is corrected. For example, if Y1 is tilted by +5 degrees with respect to the y-axis, the scanned image of the receipt can be corrected by -5 degrees with respect to the y-axis. An algorithm such as Efficient Detection or YOLO may be used to correct the orientation of the scanned image of the receipt.

As shown in FIG. 10 , according to the amount detection algorithm according to an embodiment of the present invention, it is possible to extract the amount into the treatment amount area DiA. It is possible to extract from the treatment amount area DiA into areas Di1, Di2, Di3, Di4, Di5, and Di6 including one number. Each number can be read by recognizing the corresponding regions Di1, Di2, Di3, Di4, Di5, and Di6 as an image. Regions corresponding to '1', Regions corresponding to '2', Regions corresponding to '3', Regions corresponding to '4', Regions corresponding to '5', Regions corresponding to '6' regions corresponding to '7', regions corresponding to '8', regions corresponding to '9', regions corresponding to '0' and corresponding numbers (1, 2, 3, 4, 5, 6, 7, 8, 9, 0) as training data, the amount detection algorithm may be trained.

11 , according to the name detection algorithm according to an embodiment of the present invention, the extracted sub-item areas and the sub-item name corresponding thereto can be trained as training data.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

10: insurance claim system 100: insurance claim device
110: processor 120: communication unit
130: input unit 140: output unit
150: receipt recognition device 151: receipt acquisition unit
152: area detection unit 153: total confirmation unit
154: detailed item detection unit 155: medical treatment amount reading unit
156: Insurance Claims Register
200: claimant terminal 300: manager terminal
400: database

Claims (13)

obtaining, by a receipt recognition device for claiming insurance money for medical expenses, a scanned image of the receipt;
extracting, by the receipt recognition device, one or more detailed item areas from the scanned image of the receipt using an object detection algorithm;
determining, by the receipt recognition device, a name of a detailed item included in a detailed item area using a name detection algorithm;
determining, by the receipt recognition device, a medical treatment amount area for each detailed item name by using the previously registered detailed item related information;
The receipt recognition device further comprises a;
The amount detection algorithm is
Extracts areas including one or more numbers included in the extracted treatment amount area, recognizes areas as images and reads each number, areas corresponding to '1', areas corresponding to '2', ' Regions corresponding to 3', Regions corresponding to '4', Regions corresponding to '5', Regions corresponding to '6', Regions corresponding to '7', Regions corresponding to '8' Areas, areas corresponding to '9', areas corresponding to '0' and corresponding numbers 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 as training data A method for automatically recognizing medical care items from scanned images of receipts. According to claim 1,
The step of determining the name of the detailed item is
A method of automatically recognizing a medical item from a scanned image of a receipt, extracting a feature value by recognizing the text of the detailed item area as an image, and determining the name of the detailed item corresponding to the detailed item area based on the extracted feature value. According to claim 1,
The step of extracting the treatment amount is
Among the detailed item areas, a treatment amount area is detected, and one or more numbers included in the treatment amount area are respectively detected as a number area,
A method of automatically recognizing a medical item from a scanned image of a receipt, in which each number in each number area is recognized, and then the number of each recognized number is combined to determine the amount of treatment. According to claim 1,
The step of extracting the one or more detailed item areas is
Determining the format of the receipt using one or more identifiers included in the scanned image,
Detecting a patient classification area and a treatment item area in the scanned image of the receipt based on a format corresponding to the receipt,
Further comprising the step of determining a total amount area using the patient classification area and the treatment item area, recognizing each number included in the total amount area, and summing each recognized number to determine the total amount, the scanned image of the receipt How to automatically recognize medical items in . 5. The method of claim 4,
It is determined whether the total amount and the billing amount input by the user match, and
If the total amount matches the input billing amount, registering the treatment amount for the detailed item included in the extracted receipt as the treatment amount for the receipt; how to recognize it as 5. The method of claim 4,
Determine whether the total amount and the billing amount input from the user match,
If the total amount does not match the entered billing amount,
The method of automatically recognizing a medical treatment item from the scanned image of the receipt, further comprising the step of transmitting a confirmation request for the medical treatment amount for the detailed item included in the extracted receipt to the manager terminal. Receipt acquisition unit for acquiring a scanned image of the receipt;
a detailed item detection unit that extracts one or more detailed item areas from the scanned image of the receipt using an object detection algorithm, and determines a name of a detailed item included in the detailed item area using a name detection algorithm;
A treatment amount recognition unit for determining the treatment amount area for each detailed item name using the previously registered detailed item related information, and extracting the treatment amount for the detailed item included in the receipt using an amount detection algorithm; including,
The amount detection algorithm is
Extracts areas including one or more numbers included in the extracted treatment amount area, recognizes areas as images and reads each number, areas corresponding to '1', areas corresponding to '2', ' Regions corresponding to 3', Regions corresponding to '4', Regions corresponding to '5', Regions corresponding to '6', Regions corresponding to '7', Regions corresponding to '8' Areas, areas corresponding to '9', areas corresponding to '0' and corresponding numbers 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 as training data being, a receipt recognition device. 8. The method of claim 7,
The detailed item detection unit
A receipt recognition device for recognizing the text of the detailed item area as an image, extracting a feature value, and determining a detailed item name corresponding to the detailed item area based on the extracted feature value. 8. The method of claim 7,
The treatment amount recognition unit
Among the detailed item areas, a treatment amount area is detected, one or more numbers included in the treatment amount area are detected as a number area, respectively, a number in each number area is recognized, and then each recognized number is combined to provide treatment A receipt recognition device that determines the amount. 8. The method of claim 7,
A format of the receipt is determined using one or more identifiers included in the scanned image, and a patient classification area and a treatment item area in the scanned image of the receipt based on a format corresponding to the receipt an area detection unit that detects
Receipt recognition device further comprising: a sum confirmation unit for determining a total amount area using the patient division area and the treatment item area, recognizing each number included in the total amount area, and summing each recognized number to determine the total amount . 11. The method of claim 10,
The total check unit
It is determined whether the total amount and the billing amount input from the user match, and if the total amount is the same as the input billing amount, the extracted treatment amount for detailed items included in the receipt is registered as the treatment amount for the receipt. which is a receipt recognition device. 11. The method of claim 10,
The total check unit
It is determined whether the total amount and the billing amount input from the user match, and if the total amount does not match the input billing amount, a request for confirmation of the treatment amount for the detailed item included in the extracted receipt is sent to the manager terminal Passed to, receipt recognition device. A computer program stored in a computer-readable storage medium for executing the method of any one of claims 1 to 6 using a computer.

Images