JP7398526B2 - Character recognition method and character recognition system for recognizing information contained in a table - Google Patents

Description

The present invention relates to a character recognition method and a character recognition system for recognizing information contained in a table.

The dictionary meaning of artificial intelligence is a technology that realizes human learning, reasoning, perceptual, and natural language understanding abilities through computer programs. This kind of artificial intelligence has developed dramatically through deep learning, which combines machine learning with neural networks that imitate the human brain.

Deep learning is a technology that enables computers to judge and learn like humans, thereby grouping or classifying objects and data.In recent years, it has become possible to analyze not only text data but also image data. It is now actively used in a wide variety of industrial fields.

With the development of artificial intelligence, various types of automation are being carried out in the field of office automation. In particular, in the field of office automation, much effort is being made to convert content printed on paper into data based on image data analysis technology that utilizes artificial intelligence. As part of this, in the office automation field, the content contained in documents is converted into data using image analysis technology (or image data analysis technology) that converts paper documents into images and analyzes the content contained in the images. , a technique is needed to analyze images according to the characteristics of the content contained in the document.

For example, when converting a document containing a table into data, the accuracy of various elements related to the table, such as the format of the table, the content of the text included in the table, and the position of the text included in the table, is required. A thorough analysis is required.

Therefore, Patent Document 1 (Table generation device and method for format automation) discloses a method for recognizing a table from an image and reproducing the recognized table. Since the table is reproduced based on (line), there is a limit to the ability to accurately analyze the contents included in the table.

Therefore, there is a need for a character recognition method that can accurately generate the contents included in a table.

Korean Registered Patent No. 10-1907029

The present invention is intended to provide a character recognition method and a character recognition system that are robust against character recognition errors and are capable of converting content (or content or information) contained in a table into data.

Furthermore, the present invention provides a character recognition method and character recognition system that are robust against character recognition errors and can ensure that the contents contained in the table are accurate as data when converting the contents contained in the table into data. It is intended to provide.

Furthermore, the present invention provides a character recognition method and character recognition system that are robust against character recognition errors and can convert the contents of a table into data by considering the organic relationship between the contents of the table. It is intended to provide

Furthermore, the present invention provides a character recognition method and a character recognition system that are robust against character recognition errors and can convert the contents included in a table into data while minimizing the amount of data processing. belongs to.

A character recognition method according to the present invention includes the steps of: receiving an image including a table; recognizing text included in a plurality of cells constituting the table; a step of identifying a cell to be corrected; a step of identifying at least one related cell related to the cell to be corrected from the plurality of cells; The method may include the step of correcting text included in the correction target cell using the vector of the correction target cell.

Further, the character recognition system according to the present invention includes a storage unit, a receiving unit that receives an image including a table, and a control unit that recognizes text included in a plurality of cells constituting the table included in the image. , the control unit specifies a correction target cell from the plurality of cells based on a preset criterion, specifies at least one related cell related to the correction target cell from the plurality of cells, and performs the correction. The text contained in the correction target cell may be corrected using a vector of the correction target cell calculated using the text contained in the target cell and the related cell.

Further, a computer program including a plurality of instructions according to the present invention includes the steps of: receiving an image including a table when the instructions are executed; recognizing text contained in a plurality of cells constituting the table; identifying a correction target cell from the plurality of cells based on a set criterion; identifying at least one related cell related to the correction target cell from the plurality of cells; and the correction target cell and the correction target cell. The step of correcting the text included in the correction target cell using the vector of the correction target cell calculated using the text included in the related cell may be executed by a computer.

As described above, the character recognition method and character recognition system according to the present invention recognize text from a table included in an image and verify the recognized text, thereby more accurately identifying the contents of the table included in the image. It can be converted into data.

More specifically, the character recognition method and character recognition system according to the present invention corrects when text included in a table is incorrectly recognized from a table included in an image. The contents can be converted into data more accurately.

1 is a conceptual diagram for explaining a character recognition system according to the present invention. It is a conceptual diagram for explaining the constituent elements of a table. 3 is a flowchart for explaining a character recognition method according to the present invention. FIG. 2 is a conceptual diagram for explaining a method of recognizing information included in a table using a character recognition system. FIG. 2 is a conceptual diagram for explaining a method of recognizing information included in a table using a character recognition system. FIG. 2 is a conceptual diagram for explaining a method of recognizing information included in a table using a character recognition system. FIG. 2 is a conceptual diagram for explaining a method of recognizing information included in a table using a character recognition system. FIG. 2 is a conceptual diagram for explaining a method of recognizing information included in a table using a character recognition system. FIG. 2 is a conceptual diagram for explaining a method of recognizing information included in a table using a character recognition system. FIG. 2 is a conceptual diagram for explaining a method of recognizing information included in a table using a character recognition system.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components will be denoted by the same reference numerals regardless of the drawing numbers, and redundant explanation thereof will be omitted. The suffixes "module" and "part" used in the following explanations are given or mixed to facilitate the preparation of specifications, and they themselves have significance and usefulness. It's not something you have. Furthermore, when describing the embodiments of the present invention, if it is determined that detailed explanation of related known techniques would obscure the gist of the embodiments of the present invention, the detailed explanation will be omitted. Furthermore, the attached drawings are only for helping understanding of the embodiments of the present invention, and the technical idea of the present invention is not limited by the attached drawings, and any changes that fall within the idea and technical scope of the present invention may be made. , should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as "first", "second", etc. are used to describe various components, but the components are not limited to the above terms. These terms are only used to distinguish one component from another.

When a component is referred to as being "coupled" or "connected" to another component, it may be directly coupled or connected to the other component, and there may be other components in between. It should be understood that it is permissible to do so. In contrast, when an element is referred to as being "directly coupled" or "directly connected" to another element, there are no intermediate elements present.

Singular expressions include plural expressions unless otherwise specified.

As used herein, terms such as "comprising" and "having" are intended to specify the presence of features, numbers, steps, acts, components, parts, or combinations thereof described herein. It should be understood that this does not exclude in advance the presence or possibility of adding one or more other features, figures, steps, acts, components, parts or combinations thereof.

As mentioned above, with the development of artificial intelligence, office automation is being carried out in various forms, and for business efficiency, tables contained in paper documents are There is an increasing need to digitize (digitize) table formats.

For example, documents such as various types of receipts contain tables, and insurance companies and hospitals need to process the tables contained in large volumes of paper documents as digital data and computerize it.

In response to such needs, technology for generating or reproducing a table having the same configuration as a table included in an image is being actively developed.

On the other hand, in technology that recognizes the content (or content, information) included in a table included in an image and converts the recognized content into data, various elements related to the table (for example, text, text position, cell It is very important to accurately recognize the configuration of cells, cell locations, relationships between cells, etc.

To this end, various efforts have been made to accurately recognize the tables contained in images by accurately recognizing the various components that make up the table, and by making corrections for incorrectly recognized elements. ing.

Hereinafter, a method for correcting content (or content or information) that differs from the content of a table included in an image when it is extracted will be described in more detail with reference to the accompanying drawings. FIG. 1 is a conceptual diagram for explaining the character recognition system according to the present invention, and FIG. 2 is a conceptual diagram for explaining the constituent elements of a table. Further, FIG. 3 is a flowchart for explaining the character recognition method according to the present invention, and FIGS. 4, 5, 6A, 6B, 6C, 7, and 8 are information included in the table by the character recognition system. FIG. 2 is a conceptual diagram for explaining a method of recognizing.

The character recognition system 100 according to the present invention recognizes the content included in the table 1100 and the components making up the table 1100 from the table 1100 included in the image 1000, and based on the recognized content and the relationship between the components, The contents included in the table 1100 can be accurately converted into data.

Here, the contents included in the table 1100 refer to all meaningful code systems such as letters, numbers, symbols, and operators. In the present invention, for convenience of explanation, "all meaningful code systems such as letters, numbers, symbols, operators, etc." are collectively named "text."

For example, as shown in the figure, the table 1100 includes texts such as "patient name," "resident registration number," and "medical fee (medication fee) breakdown."

Further, the constituent components that make up the table 1100 are composed of constituent elements such as cells containing text and lines or lines that partition the cells. These components have positional and/or semantic relationships with each other.

Here, the positional relationship indicates the mutual positional relationship between cells included in the table 1100, and for each cell, which cells are located around or next to it, and which cells are the same or It means the positional relationship (or arrangement relationship) regarding whether they are located in adjacent rows or columns.

Further, the semantic relationship means that at least some of the cells included in the table 1100 and another part of the cells include contents that are related to each other. Such a semantic relationship means connection information based on the content (text) included in each cell. For example, the contents contained in one cell and the contents contained in another cell may i) be the same or similar concept, ii) belong to the same category or the same group, or iii) be mutually exclusive. On the other hand, the content may correspond to a superordinate concept or a subordinate concept, and in this case, it can be expressed that these cells are connected.

Note that the image 1000 may include one or more tables, and in the present invention, all tables included in the image 1000 can be recognized regardless of the number of tables included in the image 1000. can.

In the present invention, the image 1000 is an image obtained by scanning a paper document, an image obtained by taking a photograph, or an image obtained by various other methods.

Meanwhile, as shown in FIG. 1, the character recognition system 100 according to the present invention includes at least one of a receiving unit 110, a storage unit 120, an optical character reader (OCR) unit 130, and a control unit 140.

First, the receiving unit 110 is a means for receiving the image 1000 including the table 1100, and may include at least one of a communication unit, a scanning unit, and an input unit, and may include other means for receiving the image 1000. You can also do this.

The character recognition system 100 can recognize the table 1100 included in the image 1000 received via the receiving unit 110 and convert the contents included in the table 1100 included in the image 1000 into data. As a result of the recognition, the character recognition system 100 can secure data that matches the content included in the table 1100.

In the present invention, the character recognition system 100 recognizes the content (or content, information) of the table 1100 included in the image 1000, and if there is an error in the recognized content, corrects it. It is possible to provide a method for accurately converting the contents into data.

Note that the content recognized from the table 1100 included in the image 1000 according to the present invention can be used to generate table data having the same or similar structure to the table 1100 included in the image 1000.

The table data consists of formatted or structured data based on at least one of the cell structure and cell relationships (such as the relationship between category cells and data cells) constituting the table 1100 included in the image 1000. It's okay.

The table data includes data corresponding to the content recognized by the character recognition system 100 according to the present invention.

In the present invention, the positional relationship of the constituent elements includes, in particular, the positional relationship of the cells included in the table, and the positional relationship according to the row or column relationship, and the semantic relationship of the constituent elements includes It includes the relationships formed between the meanings of the included texts.

Next, the storage unit 120 may store various information according to the present invention. The types of storage units 120 are very diverse, and at least some of them refer to external servers 150 (at least one of a cloud server 151 and a database (DB) 152). That is, the storage unit 120 may be any space in which information related to the present invention is stored, and there is no physical space restriction.

The storage unit 120 includes information regarding various components that make up the table. The storage unit 120 stores i) an image 1000 including the table 1100 and data related thereto, ii) cell information of the table 1100 recognized from the image 1000, constituent components (for example, lines, corners, etc.) and data related thereto; At least one of the following is saved: iii) content (eg, text, images, etc.) data included in table 1100; and iv) data sets related to the content included in table 1100. Here, the data set may be data used to verify or correct the content included in the table 1100.

Next, the OCR unit 130 is a means for recognizing content (or information) included in the image 1000, and can recognize the content included in the image 1000 using at least one of various content recognition algorithms. The OCR unit 130 can recognize content using an algorithm based on artificial intelligence (or a deep learning algorithm). Here, the content may include text (characters). The OCR section can also be named "OCR API".

The OCR unit 130 can recognize the text included in the image 1000 and the position information of the text. Here, the text position information includes at least one of information regarding the position of the text within the image 1000 input via the receiving unit 110 and information regarding the position of the text within the table 1100.

The OCR unit 130 can recognize the text included in each cell based on the text included in the image 1000.

The table 1100 is composed of a plurality of cells, and the OCR unit 130 can recognize the text contained in each cell.

For example, as shown in FIG. 1, when the text "Patient Name" is recognized from the first cell 1101 of the table 1100, the identification information in the first cell 1101 and the text "Patient Name" are matched, Such matching information is stored in the storage unit 120.

In the present invention, the OCR unit 130 can recognize the text contained in each cell separately. Therefore, in the present invention, the cell identification information and the corresponding text are matched to determine which text is included in which cell. Such matching information is stored in the storage unit 120.

In this way, the text recognized from each cell is matched and stored with the respective cell's identification information, where such cell identification information includes the cell's location information with respect to at least one of the image 1000 and the table 1100. is included.

Such matching information may be generated by the OCR unit 130 during the process of recognizing text from the image 1000, or may be generated under the control of the control unit 140.

Next, the control unit 140 controls the overall operation of the character recognition system 100 related to the present invention. The control unit 140 may include a processor (or an artificial intelligence processor, a deep learning processor) that processes an artificial intelligence algorithm (or a deep learning algorithm). The controller 140 may recognize the table 1100 from the image 1000 and recognize at least one cell forming the table 1100 based on an artificial intelligence algorithm.

Further, the control unit 140 may analyze the relationship between cells and the meaning of text included in the cells based on an artificial intelligence algorithm.

Further, the control unit 140 can correct at least a portion of the text recognized by the OCR unit 130 based on the relationship between cells and the meaning of the text included in the cells.

Here, the control unit 140 may correct the text recognized by the OCR unit 130 based on the data set stored in the storage unit 120.

Note that a cell in the present invention refers to a rectangular box defined by a plurality of lines forming a table.

Prior to explaining the present invention, the constituent elements of the table will be explained. A "table" in the present invention means something that shows certain contents in a predetermined format or procedure, and is also called a table. Describing the components of the table with reference to FIG. 2, the table 200 includes at least one cell (or space, region) 210. That is, table 200 includes at least one cell 210 and is configured to include information within cell 210.

Cells 210 included in table 200 may be defined by at least four line segments. The table 200 and the cells provided within the table 200 have a rectangular shape.

That is, a cell included in the table may be a rectangle surrounded by four line segments. A "line segment" in the present invention refers to a line or at least a portion of a line. The line segments constituting each cell extend into lines or lines.

In this way, at least four line segments a, b, c, and d are required to form one rectangular cell, which corresponds to the two horizontal lines (horizontal lines) that make up the table. Alternatively, it may consist of at least a portion of lines (or edges) in the row direction 201 and 202 and two vertical lines (or lines or edges in the column direction) 203 and 204. More specifically, a rectangular cell may be formed by at least a portion of two horizontal lines 201, 202 and two vertical lines 203, 204 intersecting each other.

Therefore, the table 200 includes at least four lines, and the number of cells, cell size, cell position, etc. are defined by the number of horizontal lines, the number of vertical lines, the spacing between lines, and the arrangement position of the lines. Ru.

On the other hand, the plurality of lines constituting the table 200 are divided into different types, either a first type or a second type, and this is specified depending on which direction the line extends.

As shown, horizontally extending lines 201, 202 are defined as a first type, and vertically extending lines 203, 204 are defined as a second type.

The first type of line may be variously named by terms having the same or similar meanings, such as horizontal line, horizontal line, row line, horizontal line, etc.

In addition, the second type of line may be variously named by terms having the same or similar meanings, such as a vertical line, a vertical line, a column line, a vertical line, etc.

On the other hand, the number of cells, cell size, cell position, etc. included in the table 200 can be variously modified based on the number and arrangement of horizontal lines and vertical lines, and can also be varied depending on the combination of cells. It can be transformed into.

In addition, the constituent components constituting the table in the present invention are the corners (or corners, vertices) (symbol "e , f, g, h").

All tables have at least four corners, which may be the corners contained in the outermost frame of the table. Also, the number of corners included in the table varies depending on the number of cells included in the table.

Hereinafter, a method for generating a table will be described in more detail based on the configuration of the character recognition system according to the present invention described above.

As shown in FIG. 3, in the character recognition method according to the present invention, first, a process of receiving an image 1000 (see FIG. 1) including a table 1100 is performed (S310).

As previously discussed, image 1000 including table 1100 may be received via a variety of routes. For example, the image 1000 can be received by being transmitted by a communication unit, scanned by a scanning unit, input by an input unit, etc.

Once image 1000 is received, a process of recognizing text from table 1100 contained in image 1000 is performed.

More specifically, in the present invention, a process of recognizing text included in a plurality of cells constituting the table 1100 is performed (S320). Such recognition is performed by the OCR unit 130, and the information or content included in the table 1100 is recognized by the OCR unit 130. Here, "recognizing the information or contents included in table 1100" means that text included in table 1100 is recognized.

As a result of the recognition by the OCR unit 130, data corresponding to the contents included in the table 1100 can be secured. For example, the OCR unit 130 recognizes the text included in the table 1100, and selects text that corresponds to the content included in the table 1100, such as "patient name", "resident registration number", "medical fee (medicine cost) breakdown", etc. text can be secured as data. The OCR unit 130 can recognize text from the image 1000 using a text recognition model trained to recognize each text (or character) included in the image 1000 and the position of the text. Such text recognition models may include artificial intelligence-based algorithms (eg, deep learning algorithms).

As shown in FIG. 4(a), the OCR unit 130 recognizes text from the table 1100, and as shown in FIG. 4(b), the recognized text is identified by the identification of each cell containing the text. Information KEY1, KEY2, KEY3. ．． ．． and are matched and stored in the storage unit 120.

In this manner, when text is recognized for cells forming the table 1100, a process of verifying the recognized text may be performed. Here, "verification" refers to the process of confirming whether the text included in each cell has been correctly recognized. For example, as shown in (a) of FIG. 4, a specific cell 401 of the table 1100 includes the text "cooperative contribution amount," but as shown in (b) of FIG. The text ``Gross combined burden amount'' 402 may be recognized.

In this case, content different from the original document is recognized (erroneously recognized), causing problems with the reliability and accuracy of the recognized data.

Therefore, the control unit 140 may verify the text recognized from the table 1100. In this case, in the present invention, the text included in the cells that require correction may be corrected in the verification process based on the mutual positional relationship and semantic relationship (or connection relationship) between the cells.

To this end, in the present invention, a process of identifying a correction target cell from a plurality of cells is performed (S330). The control unit 140 may specify at least one correction target cell from a plurality of cells based on preset criteria. Here, the predetermined criteria can be set in a wide variety of ways, and therefore, in the present invention, the process for specifying the correction target cell is very diverse.

For example, when text is recognized for a plurality of cells forming the table 1100, the control unit 140 may verify all the cells in which text has been recognized.

In this case, all cells in which text has been recognized are identified as cells to be corrected.

On the other hand, the control unit 140 may verify text included in a category cell among a plurality of cells configuring the table 1100.

In this case, the control unit 140 specifies each of the texts included in the category cell among the plurality of cells configuring the table 1100 as a correction target cell.

Here, the category cell is also called a key cell and is distinguished from a data cell (or value cell).

A category cell is also named a title cell, and is understood as a cell that includes text with a meaning that defines the category, type, affiliation, characteristics, etc. of data (text) included in the data cell.

A category is something that can be divided based on the same or similar properties or meanings, and is also called a category. Data belonging to the same category may be data that has the same meaning, the same type, or the same affiliation.

The category cell contains a category name or data related to the category name, and the data cell contains data belonging to the category name included in the category cell. Here, the category name is also called a category name.

As an example, as shown in (a) of FIG. 4, in the table 1100, a specific cell 412 containing the text "Medical treatment, dispensing date (medical treatment period)" is a category cell, and is the same as the specific cell 412. Cells 403 located in the column and each containing date information are data cells related to a particular cell 412 .

As another example, in the table 1100, a specific cell 411 that includes the text "Patient Name" is a category cell, is located in the same row as the specific cell 411, and includes name information "Hong Gil-dong." Cell 405 is a data cell associated with particular cell 411.

As shown in FIG. 4, table 1100 includes one or more category cells 411, 412, 413, 414, 415, 416.

Based on the structure of the table 1100 and the meaning of the text included in the cells, the control unit 140 divides the types of the plurality of cells making up the table 1100 into a first type (category cell type) and a second type (data cell type). It can be specified as either.

Further, the control unit 140 can specify each text included in a category cell among the plurality of cells configuring the table 1100 as a correction target cell.

As another example, the control unit 140 can specify at least some of the category cells among the plurality of cells making up the table 1100 as correction target cells. Here, the preset standard for identifying the cells to be corrected may be related to the recognition rate for text included in the plurality of cells. The control unit 140 can specify, as a correction target cell, a specific cell that has a low text recognition rate at the time of text recognition and extraction, among the texts extracted from the category cells.

As shown in FIG. 5, when text is recognized for each cell by the OCR unit 130, the reliability of recognition of each text can be specified.

Such confidence is also named confidence score or confidence score.

As shown in FIG. 5, the reliability score may be matched with at least one of each cell constituting the table 1100 and the text recognized from the cell and stored in the storage unit 120.

As shown in FIG. 5, the control unit 140 selects category cells whose text reliability satisfies preset standard conditions (for example, cells whose text reliability is less than a standard score (standard probability, etc.)) to be corrected. It can be specified as a cell. According to Figure 5, the closer the confidence score is to 1, the higher the probability that the recognized text was correctly recognized, and the closer the confidence score is to 0, the lower the probability that the recognized text was correctly recognized. means.

In the present invention, the control unit 140 reduces the amount of data processing calculations and accurately extracts the information included in the table 1100 by correcting the text included in cells where the probability of the text being accurately recognized is low. can do. In this way, the control unit 140 does not specify all recognized texts as texts to be corrected, but rather specifies texts with a low probability of being accurately recognized as texts to be corrected based on the reliability. Therefore, since the control unit 140 performs the correction according to the present invention only on the text specified as the text to be corrected, the amount of calculation for data processing can be reduced compared to when correction is performed on all texts. .

For example, if the preset standard condition is set to correct a cell whose text confidence score is less than 0.5, the control unit 140 corrects the cell corresponding to "KEY7" in FIG. Specify as the target cell. In this case, the text corresponding to "(1) Gross combined burden" included in cell "KEY7" is corrected.

In this way, when a correction target cell is specified based on various methods or criteria, the control unit 140 selects at least one related cell related to the correction target cell from among the plurality of cells included in the table 1100 and the cells included therein. The text included in the correction target cell can be corrected using the content (text) contained in the correction target cell.

To this end, in the present invention, a process of identifying related cells related to the correction target cell from a plurality of cells included in the table 1100 is performed (S340).

For convenience of explanation, the following description will be made assuming that the cell to be corrected is a specific category cell 601 that includes the text "(1) Association contribution amount" shown in FIG. 6A. The following explanation can be commonly applied to all category cells.

As shown in FIG. 4, FIG. 6A, and FIG. 6B, in the image 1000, a specific category cell 601 includes text corresponding to "(1) Association contribution amount," but as explained in FIG. Due to an error, the text "(1) Gross contribution amount" may be recognized. Therefore, the control unit 140 uses the specific category cell 601 that includes the incorrectly recognized text and at least one related cell related to the specific category cell 601 to identify the incorrectly recognized text (for example, (1) )The gross combined contribution amount) can be corrected.

The control unit 140 can specify at least one related cell using the correction target cell 601 as a reference.

The control unit 140 can identify related cells based on at least one of the meaning of the text included in the correction target cell 601 and the position of the correction target cell 601.

Here, the related cell of the correction target cell 601 may be a category cell. The control unit 140 can specify only the category cells among the category cells and data cells included in the table 1100 as cells related to the correction target cell 601.

Furthermore, the control unit 140 specifies, among the category cells, a category cell that includes text that is semantically related to the correction target cell 601 as a related cell, or identifies a category cell that is positionally adjacent to the correction target cell 601. It can be identified as a related cell.

When specifying a cell related to the correction target cell 601, the control unit 140 can specify the related cell based on any of the following cases.

As a first case, the control unit 140 can specify a category cell that includes text that is semantically related to the correction target cell 601 as a related cell.

Furthermore, as a second case, the control unit 140 can specify a category cell that is positionally related to the correction target cell 601 as a related cell.

Furthermore, as a third case, the control unit 140 combines the first case and the second case, identifies a category cell containing text semantically related to the correction target cell 601 as a related cell, and A category cell related to the correction target cell 601 can be specified as a related cell. All of the cells identified as related cells in the third case, first case, and second case can be identified as related cells of the correction target cell 601.

Hereinafter, as a specific example of the first case, a method of identifying related cells based on the meaning of the text included in the correction target cell 601 will be described.

First, a method for identifying related cells based on the meaning of the text included in the correction target cell 601 will be explained. One category cell can be identified as a related cell.

The control unit 140 analyzes the meaning of the text included in the correction target cell 601 and the meanings of the texts included in other category cells based on an artificial intelligence algorithm, and selects at least one text that includes texts with mutually related meanings. Category cells can be identified as related cells.

The text having a meaning related to the correction target cell 601 may be a text having a meaning of a superordinate concept or a subordinate concept of the text included in the correction target cell 601.

Since the control unit 140 specifies the correction target cell 601 and related cells for the category cell, due to the characteristics of the category cell, the correction target cell 601 has a meaning that corresponds to a superordinate concept or a subordinate concept with respect to the related cell. Contains text. For example, the correction target cell 601 and related cells are composed of texts having meanings that fall under the same or related categories (types).

For example, as shown in FIG. 6A, the text included in the first category cell 611 (“Medical expenses (drug expenses) breakdown”) is changed to the second category cell 612 (“Total amount (1) + (2) + (3)”). ”), third category cell 613 (“Salary”), fourth category cell 615 (“(2) Patient burden amount”), fifth category cell 618 (“Patient burden total amount (2) + (3)”), 6th category cell 601 (cell to be corrected, "(1) Association contribution amount", recognized as "(1) Gross contribution amount" when OCR is recognized), 7th category cell 617 ("(3) Patient contribution amount") ”), which has a superordinate meaning for the text contained in the eighth category cell 616 (“non-salary”).

Furthermore, a category cell that is a subordinate concept to the first category cell 611 is also a category cell that is a superordinate concept to other category cells. For example, the text “salary” included in the third category cell 613 is a subordinate concept to the text “Medical expenses (medication costs) breakdown” included in the first category cell 611, and the text “salary” included in the fourth category cell 615 The text “(2) Patient burden amount” included in the text “(2) Patient burden amount” and the text “(1) Union burden amount” included in the sixth category cell 601 (cell to be corrected) (recognized as “(1) Gross combined burden amount” when OCR is recognized) ) is a superordinate concept.

On the other hand, among the category cells, the control unit 140 can specify a category cell that includes text that is semantically related to the correction target cell 601 as a related cell. A category cell containing a text that has a meaning of a superordinate concept or a subordinate concept of the text can be specified as a related cell.

In the first case, cells related to the correction target cell 601 are a first category cell 611 (“Medical expenses (medication costs) breakdown”) and a third category cell 613 (“ ``Salary'').

Hereinafter, as a specific example of the second case, a method of specifying a category cell that is positionally related to the correction target cell 601 as a related cell will be described. More specifically, in consideration of the characteristics of the table 1100, the control unit 140 selects a category cell that has a preset positional relationship (or arrangement relationship) with the correction target cell 601, and converts the text included in the correction target cell 601 into a category cell that has a preset positional relationship (or arrangement relationship) with the correction target cell 601. It can be determined as a category cell related to the meaning of , that is, a related cell.

Here, the preset positional relationship may mean being located adjacent to the correction target cell 601. For example, the second category cell 612 (“total amount (1)+(2)+(3)”), the third category cell 613 (“salary”), and the fourth category cell 615 (“(2) patient burden amount”) The specific data cell 622 is located adjacent to the correction target cell 601, and can be said to have a preset positional relationship with the correction target cell 601.

In this way, cells having a preset positional relationship with the correction target cell 601 are located side by side with the correction target cell 601 in the column direction a or the row direction b, with the correction target cell 601 as a reference. This applies to cells located adjacent to.

Here, the cells located side by side in the column direction a of the correction target cell 601 are cells corresponding to symbols 621, 611, 613, 601, and 622 in FIG. 6A, and the cells located side by side in the row direction b of the correction target cell 601 The located cells are cells corresponding to 641, 642, 612, 601, 615, 617, 618, 643, 644, and 645 in FIG. 6A.

On the other hand, since the control unit 140 specifies only the category cell as the related cell of the correction target cell 601, the control unit 140 specifies only the category cell as the cell related to the correction target cell 601. +(3)''), the third category cell 613 (``salary''), the fourth category cell 615 (``(2) patient burden amount''), and the specific data cell 622) are related cells. excluded from.

As a result, as shown in FIG. 6B, the control unit 140 selects the second category cell 612 corresponding to the category cell from the cells having the preset positional relationship ("total amount (1) + (2) + (3) ”), the third category cell 613 (“salary”), and the fourth category cell 615 (“(2) patient burden amount”) can be specified as related cells.

According to the second case, the cells related to the correction target cell 601 are the second category cell 612 (“total amount (1)+(2)+(3)”), the third category cell 613 (“salary”), and the third category cell 613 (“salary”). A 4-category cell 615 (“(2) Patient burden amount”) is specified.

On the other hand, in order to specify a related cell having a positional relationship with the correction target cell 601, the control unit 140 calculates the median value of the corners (vertices) of the correction target cell 601 to the correction target cell on the table 1100 or the image 1000. 601, and category cells 612, 613, and 615 having median values located within a preset distance from this position can be identified as related cells 612, 613, and 615.

Here, the preset distance may be based on L2 distance information, and the L2 distance information can also be called Euclidean distance information.

In this way, in the table 1100, the control unit 140 calculates the median value located within a preset distance from the median value of the corner (vertex) of the correction target cell 601 in each of the left, right, upper, and lower directions. It is possible to check whether there is a category cell with the same category. Further, as a result of the confirmation, if there is a category cell having a median value located within a preset distance, it can be identified as a related cell.

As described above, the control unit 140 can specify the related cells related to the correction target cell 601 according to the first case or the second case.

Further, the control unit 140 can specify all of the related cells specified by the first case and the related cells specified by the second case as related cells of the correction target cell 601. This case corresponds to the third case mentioned above.

As described above, the control unit 140 specifies, among the category cells included in the table 1100, a category cell that includes text that is semantically related to the correction target cell 601 in the first case as a related cell, and In the second case, a category cell that is located adjacent to the correction target cell 601 can be specified as a related cell. As a result, as shown in FIGS. 6A and 6B, the cells related to the correction target cell 601 are the first category cell 611 (“Medical expenses (drug expenses) breakdown”) and the second category cell 612 (“Total amount (1) +(2)+(3)''), the third category cell 613 (``salary''), and the fourth category cell 615 (``(2) patient burden amount'').

Here, at least one category cell (for example, the third category cell 613 (“salary”)) is identified as a related cell because it overlaps semantically and positionally.

It goes without saying that the category cells 611, 612, 613, and 615 identified as cells related to the correction target cell 601 may also be named "related cells," and the same reference numerals are used.

In the following, as an example of the third case, a method of correcting the text included in the correction target cell 601 using related cells identified in consideration of both semantic relationships and positional relationships will be explained. do.

As described above, when the correction target cell 601 and at least one related cell are specified in the third case, the control unit 140 specifies the correction target cell 601 and the related cells 611, 612, 613, as shown in FIG. 6C. , 615 to embed the correction target cell 601 and calculate (or calculate, derive) a dense vector, which can be used to correct the correction target cell 601. . In the present invention, in order to show the relationship between the text contained in the correction target cell and related cells, the text contained in the correction target cell and related cells is embedded, and the text contained in the correction target cell and related cells is embedded. It can be shown as a vector.

Here, embedding is a method of expressing information to be embedded as a vector, and in the present invention, the control unit 140 uses embedding to express information of the correction target cell 601 and related cells 611, 612, 613, and 615. It can be expressed in the form of a dense vector.

More specifically, the control unit 140 can calculate a dense vector indicating the correction target cell 601 using text included in each of the correction target cell 601 and related cells 611, 612, 613, and 615.

Further, the control unit 140 can use the dense vector calculated using the correction target cell 601 and the related cells 611, 612, 613, and 615 to correct the correction target cell 601.

In the present invention, the algorithm for performing embedding is not particularly limited, and any algorithm that can express text as a dense vector can be utilized in the present invention. For example, the control unit 140 calculates a dense vector indicating the correction target cell 601 based on the text included in the correction target cell 601 and related cells 611, 612, 613, and 615 based on an algorithm such as FastText or BERT. Can be done.

On the other hand, in order to perform embedding, the control unit 140 arranges texts 631, 632, 633, 634, and 635 included in the correction target cell 601 and related cells 611, 612, 613, and 615, as shown in FIG. 6C. Can be done. Here, the control unit 140 determines the arrangement order of the texts 631, 632, 633, 634, and 635 included in the correction target cell 601 and related cells 611, 612, 613, and 615 based on preset criteria. be able to.

For example, the control unit 140 can also arrange the texts 632 and 634 included in the related cells 611 and 613 that have a semantic relationship closer to the text 631 included in the correction target cell 601; It is also possible to arrange texts 633, 634, 635 included in related cells 612, 613, 615 that have a physical relationship closer to text 631 included in the correction target cell 601.

Further, the control unit 140 can also place the text 634 included in the related cell 613 having both semantic and positional relationships at the position closest to the text 631 included in the correction target cell 601.

In this way, by embedding, the control unit 140 can indicate the correction target cell 601 as a vector based on the text included in the correction target cell 601 and related cells 611, 612, 613, and 615. In this way, when embedding is completed for the correction target cell 601 based on the texts 631, 632, 633, 634, and 635 included in the correction target cell 601 and related cells 611, 612, 613, and 615, in the present invention, , a process of correcting the correction target cell is performed (S350).

The control unit 140 corrects the text included in the correction target cell 601 using the vector of the correction target cell calculated using the texts 631, 632, 633, 634, and 635 included in the correction target cell 601 and related cells. It can be performed. Here, the vector of the correction target cell 601 may correspond to the above-mentioned dense vector.

More specifically, the control unit 140 selects a word having a vector expression most similar to the vector of the correction target cell 601 from a data set specified in advance (for example, a specific vector located at the closest distance to the correction target cell 601). ) can be extracted as the correction target text.

The control unit 140 performs the pre-identification based on the distance between the vector of the correction target cell calculated using the correction target cell 601 and related cells 611, 612, 613, and 615 and the vector included in the pre-specified data set. Text to be corrected can be extracted from the dataset. Here, the pre-specified data set may include candidate texts that may be included in a plurality of cells included in the table 1100, particularly category cells, and vectors corresponding to the candidate texts. Such a dataset may be pre-built, and candidate texts that may be included in the dataset may be further included in or removed from the dataset after the dataset is built. It is possible to be removed.

Further, such a data set includes the result of embedding based on at least one of the positional relationship of a plurality of cells configuring the table 1100 and the semantic relationship between candidate texts included in each cell. Good too. That is, the dataset may include vector information regarding each of the candidate texts.

The control unit 140 can extract the text 806 (see FIG. 7) corresponding to the specific vector closest to the vector of the correction target cell 601 from the data set as the correction target text.

Further, the control unit 140 can change the text included in the correction target cell 601 to the text 806 (see FIG. 7) corresponding to a specific vector closest to the vector of the correction target cell 601.

In this manner, the control unit 140 uses the vector of the correction target cell 601 derived using the texts 801 to 805 included in the correction target cell 601 and related cells 611, 612, 613, and 615 to Correction target text 806 can be extracted.

As a result, the erroneously recognized text "(1) Combined contribution amount" for the correction target cell 601 is corrected to "(1) Union contribution amount".

In this manner, when the correction target text 806 is extracted from the data set, the control unit 140 can change the text 801 included in the correction target cell 601 to the correction target text 806.

On the other hand, as described above, when the correction target text 806 is extracted, the control unit 140 can utilize the extracted correction target text 806 to convert the table 1100 included in the image 1000 into data. The control unit 140 can include the correction target text 806 on the data obtained as a result of converting the table 1100 included in the image 1000 into data.

Further, the control unit 140 can not only convert the table 1100 into data, but also use the data obtained as a result of data conversion to generate a table having a structure corresponding to the table 1100 included in the image 1000. .

For example, as shown in (a) of FIG. 8, the control unit 140 controls the text 930 (“Other than Arasawa Medical Department”) that corresponds to at least part of the data extracted from the table 910 included in the scanned image. If there is an error, by correcting it using the method described above, it is possible to secure data that includes the corrected text 940 (“other than selected medical fees”), as shown in FIG. 8(b). . Further, if necessary, a table 920 including the corrected text 940 can be generated, and such a table 920 is configured with the same contents as the table 910 that was the object of the scan.

As described above, the character recognition method and character recognition system according to the present invention recognize text from a table included in an image and verify the recognized text, thereby more accurately identifying the contents of the table included in the image. It can be converted into data.

More specifically, the character recognition method and character recognition system according to the present invention corrects when text included in a table is incorrectly recognized from a table included in an image. The contents can be converted into data more accurately. To this end, in the present invention, the text included in the cell that is the target of correction is corrected by taking into account the arrangement relationship of cells composing the table and the semantic relationship of the text included in the cell. It is possible to provide a highly accurate character recognition method and system that can directly convert the contents of a table into data. As a result, the character recognition method and system according to the present invention can generate a table containing the same contents as the table included in the image.

In addition, the character recognition method and character recognition system according to the present invention corrects only specific cells with a low text recognition rate, thereby reducing the amount of data processing while accurately converting the contents of the table included in the image into data. can be minimized.

On the other hand, the present invention described above can be implemented as a program that is executed by one or more processes on a computer and can be stored on a computer-readable medium (or recording medium).

Further, the present invention described above can be implemented as computer readable codes or commands on a program recording medium. That is, the present invention can be provided in the form of a program.

On the other hand, the computer-readable medium includes any type of recording device on which data that can be read by a computer system is recorded. Examples of computer readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. It will be done.

The computer-readable medium also includes storage and can be a server or cloud storage that the electronic device can access via communication. In this case, the computer can download the program according to the invention from the server or cloud storage by wired or wireless communication.

Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, that is, a central processing unit (Central Processing Unit, CPU), and its type is not particularly limited.

On the other hand, the detailed description of the present invention is illustrative and should not be construed as restrictive in any respect. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all modifications within the range of equivalency of the present invention are included within the scope of the present invention.

100 Character recognition system 110 Receiving unit 120 Storage unit 130 OCR unit 140 Control unit 150 External server 151 Cloud server 152 Database (DB)
200 Table 201, 202 Horizontal line 203, 204 Vertical line 210 Cell 1000 Image 1100 Table

Claims (14)

A character recognition method performed by a character recognition system including a receiving unit and a control unit,
the receiving unit receiving an image including a table;
The control unit,
Recognizing text contained in a plurality of cells constituting the table;
identifying a correction target cell from the plurality of cells based on preset criteria;
identifying at least one related cell related to the correction target cell from the plurality of cells based on a related relationship formed between meanings indicated by texts included in each cell ;
The present invention is characterized by comprising the step of correcting the text included in the correction target cell using a vector of the correction target cell calculated using the text included in the correction target cell and the related cell. Character recognition method. In the step of identifying the related cells,
2. The character recognition method according to claim 1, wherein the related cell is identified based on the meaning of text included in the correction target cell. At least some of the plurality of cells are identified as category cells, and other parts are identified as data cells that include data corresponding to at least one of the category cells,
In the step of identifying the related cells,
3. The character recognition method according to claim 2, wherein a specific category cell related to the meaning of the text included in the correction target cell is specified as the related cell among the category cells. Specific category cells related to the meaning of the text contained in the correction target cell include:
4. The character recognition method according to claim 3, wherein the text included in the correction target cell includes a text having a meaning of a higher-level concept or a lower-level concept of the text included in the correction target cell. A specific category cell related to the meaning of the text included in the correction target cell is:
5. The character recognition method according to claim 4, wherein the character recognition method is arranged parallel to the correction target cell in a column direction or a row direction with the correction target cell as a reference. The specific category cell is a cell located adjacent to the correction target cell in the row or column direction on the table among the category cells, with the correction target cell as a reference. The character recognition method according to claim 3. In the step of performing the correction,
The character recognition method according to claim 1, wherein the text included in the correction target cell is corrected based on a distance between the vector of the correction target cell and a vector included in a data set specified in advance. . The step of performing the correction includes:
extracting a correction target text from the data set based on a distance between the vector of the correction target cell and a vector included in the data set specified in advance;
8. The character recognition method according to claim 7, further comprising the step of changing the text included in the correction target cell to the correction target text. The dataset includes:
Candidate text that may be included in the plurality of cells constituting the table, and
Claim 8, characterized in that vector information regarding each of the candidate texts is included, which corresponds to a result of embedding based on at least one of a positional relationship between the plurality of cells and a semantic relationship between the candidate texts. Character recognition method described in. In the step of performing the correction,
identifying a vector closest to the vector of the correction target cell from the data set;
10. The character recognition method according to claim 9, wherein a text corresponding to the closest vector is extracted as the correction target text. the correction target cell contains the correction target text;
11. The character recognition method according to claim 10, further comprising the step of generating a table having a structure corresponding to a table included in the image. The character recognition method according to claim 1, wherein the preset standard for identifying the correction target cell is related to reliability of recognition of text included in the plurality of cells. . preservation department,
a receiving unit that receives an image including a table;
a control unit that recognizes text included in a plurality of cells configuring the table included in the image;
The control unit includes:
identifying a correction target cell from the plurality of cells based on preset criteria;
identifying at least one related cell related to the correction target cell from the plurality of cells based on a related relationship formed between meanings indicated by text included in each cell ;
A character recognition system characterized in that the text contained in the correction target cell is corrected using a vector of the correction target cell calculated using the text contained in the correction target cell and the related cell. A computer program including a plurality of instructions,
When the instruction is executed,
receiving an image including a table;
Recognizing text contained in a plurality of cells constituting the table;
identifying a correction target cell from the plurality of cells based on preset criteria;
identifying at least one related cell related to the correction target cell from the plurality of cells based on a related relationship formed between meanings indicated by texts included in each cell ;
correcting the text included in the correction target cell using a vector of the correction target cell calculated using the text included in the correction target cell and the related cell; Featured computer program.