JP2023016031A - Character recognition method for recognizing information contained in table and character recognition system - Google Patents

Abstract

To provide a method and system for analyzing an image to generate a table.SOLUTION: A character recognition method according to the present invention comprises: receiving an image that contains a table; recognizing the texts contained in a plurality of cells constructing the table; specifying a to-be-corrected cell from the plurality of cells on the basis of a preset criterion; specifying at least one related cell, which is related to the to-be-corrected cell, from the plurality of cells; and correcting the text contained in the to-be-corrected cell using the vector of the to-be-corrected cell, which vector is calculated using the texts contained the to-be-corrected cell and the related cell.SELECTED DRAWING: Figure 1

Description

The present invention relates to a character recognition method and system for recognizing information contained in a table.

The dictionary meaning of artificial intelligence is technology that realizes human learning ability, reasoning ability, perceptual ability, natural language understanding ability, etc. by computer program. Such artificial intelligence has made dramatic progress through deep learning, which is a combination of machine learning and a neural network that mimics the human brain.

Deep learning is a technology that allows computers to make judgments and learn like humans, thereby grouping or classifying objects and data.In recent years, not only text data but also image data can be analyzed. and is actively used in a wide variety of industrial fields.

With the development of such artificial intelligence, various automations are being carried out in the field of office automation. In particular, in the field of office automation, much effort is being made to digitize content printed on paper based on image data analysis technology that utilizes artificial intelligence. As part of this, in the field of office automation, the content contained in the document is converted into data using image analysis technology (or image data analysis technology) that converts paper documents into images and analyzes the content contained in the images. Therefore, there is a need for techniques for analyzing images according to the characteristics of the content contained in the document.

For example, when digitizing a document that contains a table, the accuracy of various elements related to the table, such as the format of the table, the content of the text contained in the table, and the position of the text contained in the table. Further analysis is required.

Therefore, Patent Document 1 (table generation device and method for form automation) discloses a method of recognizing a table from an image and reproducing the recognized table. Since the table is reproduced on the basis of (line), there was a limit in accurately analyzing the contents included in the table.

Therefore, there is a need for a character recognition method that can accurately generate even the contents contained in the table.

Korean Patent No. 10-1907029

An object of the present invention is to provide a character recognition method and a character recognition system that are robust against character recognition errors and that can convert the contents (or contents, information) contained in a table into data.

Further, the present invention provides a character recognition method and a character recognition system that are robust against character recognition errors and that can ensure the contents contained in the table as data when converting the contents contained in the table into data. It is for providing.

Furthermore, the present invention provides a character recognition method and a character recognition system that are robust against character recognition errors, which can convert the contents contained in the table into data in consideration of the organic relationship between the contents contained in the table. It is intended to provide

Furthermore, the present invention is intended to provide a character recognition method and a character recognition system that are robust against character recognition errors and can convert the contents contained in a table into data while minimizing the amount of data processing. belongs to.

A character recognition method according to the present invention includes the steps of receiving an image containing a table, recognizing text contained in a plurality of cells constituting the table, and recognizing text contained in the plurality of cells based on preset criteria. identifying a correction target cell; identifying at least one related cell related to the correction target cell from the plurality of cells; and calculating using the text included in the correction target cell and the related cell and a step of correcting the text included in the correction target cell using the vector of the correction target cell.

A character recognition system according to the present invention includes a storage unit, a reception unit for receiving an image including a table, and a control unit for recognizing text included in a plurality of cells forming the table included in the image. , the control unit specifies a correction target cell from the plurality of cells based on a preset standard; specifies at least one related cell related to the correction target cell from the plurality of cells; The text included in the correction target cell may be corrected using the vector of the correction target cell calculated using the text included in the target cell and the related cell.

Further, a computer program comprising a plurality of instructions according to the present invention, upon execution of the instructions, comprises the steps of: receiving an image containing a table; recognizing text contained in a plurality of cells comprising said table; identifying a correction target cell from the plurality of cells based on set criteria; identifying at least one related cell related to the correction target cell from the plurality of cells; and the step of correcting the text included in the correction target cell using the vector of the correction target cell calculated using the text included in the related cell.

As described above, the character recognition method and character recognition system according to the present invention recognizes text from a table included in an image and verifies the recognized text to more accurately identify the contents of the table included in the image. It can be converted into data.

More specifically, the character recognition method and the character recognition system according to the present invention correct the text contained in the table contained in the image when the text contained in the table is erroneously recognized from the table contained in the image. can be converted into data more accurately.

1 is a conceptual diagram for explaining a character recognition system according to the present invention; FIG. 4 is a conceptual diagram for explaining components of a table; FIG. 4 is a flowchart for explaining a character recognition method according to the present invention; FIG. 4 is a conceptual diagram for explaining a method of recognizing information contained in a table by a character recognition system; FIG. 4 is a conceptual diagram for explaining a method of recognizing information contained in a table by a character recognition system; FIG. 4 is a conceptual diagram for explaining a method of recognizing information contained in a table by a character recognition system; FIG. 4 is a conceptual diagram for explaining a method of recognizing information contained in a table by a character recognition system; FIG. 4 is a conceptual diagram for explaining a method of recognizing information contained in a table by a character recognition system; FIG. 4 is a conceptual diagram for explaining a method of recognizing information contained in a table by a character recognition system; FIG. 4 is a conceptual diagram for explaining a method of recognizing information contained in a table by a character recognition system;

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The same or similar components are denoted by the same reference numerals regardless of the drawing numbers, and duplicate descriptions thereof will be omitted. The suffixes “module” and “part” used in the following explanation are given or used together to facilitate the preparation of the specification, and themselves have significance and usefulness. does not have In addition, in describing the embodiments of the present invention, detailed descriptions of related known techniques will be omitted if it is determined that they may obscure the gist of the embodiments of the present invention. Furthermore, the accompanying drawings are only for helping understanding of the embodiments of the present invention, and the technical ideas of the present invention are not limited by the accompanying drawings. , equivalents or alternatives.

Terms including ordinal numbers such as "first", "second", etc. are used to describe various components, but the components are not limited by the above terms. The above terms are only used to distinguish one component from another.

When a component is referred to as being "coupled" or "connected" to another component, it may be directly coupled or connected to the other component, with additional components in between. It should be interpreted as something that can be done. In contrast, when a component is referred to as being "directly coupled" or "directly connected" to another component, it should be understood that there are no additional components in between.

References to the singular include the plural unless specifically stated otherwise.

As used herein, terms such as "including" and "having" are intended to specify the presence of features, numbers, steps, acts, components, parts, or combinations thereof described herein. and does not preclude the presence or possibility of adding one or more other features, figures, steps, acts, components, parts or combinations thereof.

As mentioned above, with the development of artificial intelligence, office automation is being carried out in various forms. There is a growing need to digitize (data) data in a table format.

For example, various types of documents such as receipts contain tables, and in insurance companies and hospitals, it is necessary to process and computerize the tables contained in large amounts of paper documents as digitized data.

In response to such needs, techniques for generating or reproducing a table having the same configuration as the table included in the image are being actively developed.

On the other hand, in the technology of recognizing the content (or content, information) included in the table included in the image and converting the recognized content into data, various elements related to the table (for example, text, text position, cell configuration, cell locations, relationships between cells, etc.) is very important.

For this reason, various efforts have been made to accurately recognize the table contained in the image by accurately recognizing the various components that make up the table and correcting the erroneously recognized elements. ing.

Hereinafter, a method for correcting extracted content (or content, information) different from the content of the table included in the image will be described in more detail with reference to the accompanying drawings. FIG. 1 is a conceptual diagram for explaining a character recognition system according to the present invention, and FIG. 2 is a conceptual diagram for explaining constituent elements of a table. 3 is a flow chart for explaining the character recognition method according to the present invention, and FIGS. 4, 5, 6A, 6B, 6C, 7 and 8 show information contained in a table by the character recognition system. is a conceptual diagram for explaining a method of recognizing the .

The character recognition system 100 according to the present invention recognizes the contents included in the table 1100 and the components that make up the table 1100 from the table 1100 included in the image 1000, and based on the recognized contents and the relationship between the components, The contents contained in the table 1100 can be accurately converted into data.

Here, the content included in the table 1100 means all meaningful code systems such as letters, numerals, symbols, operators, and the like. In the present invention, for convenience of explanation, "all meaningful code systems such as letters, numerals, symbols, operators, etc." are collectively named "text".

For example, as shown, table 1100 includes text such as "Patient Name", "Resident Registration Number", and "Breakdown of Medical Expenses (Drug Expenses)".

Also, the components that make up the table 1100 are composed of components such as cells containing text, lines that partition the cells, and the like. These components have a positional and/or semantic relationship with each other.

Here, the positional relationship indicates the mutual positional relationship between the cells included in the table 1100. For each cell, which cell is located in the vicinity or next to it, which cell is the same or It means a positional relationship (or positional relationship) regarding whether or not they are located in adjacent rows or columns.

In addition, the semantic relationship means that at least some of the cells included in the table 1100 and some of the other cells include contents related to each other. Such semantic relationship means connection information based on the content (text) contained in each cell. For example, the content contained in one cell and the content contained in another cell are i) the same or similar concepts, ii) belong to the same category or the same group, iii) each other On the other hand, the content may correspond to a higher concept or a lower concept, and in this case, it can be expressed that these cells are connected.

Note that image 1000 may contain one or more tables, and the present invention can recognize all tables contained in image 1000 regardless of the number of tables contained in image 1000. can.

In the present invention, the image 1000 is an image obtained by scanning a paper document, an image obtained by photographing, or an image obtained by various other methods.

Meanwhile, the character recognition system 100 according to the present invention includes at least one of a receiver 110, a storage 120, an OCR (Optical Character Reader) 130, and a controller 140, as shown in FIG.

First, the receiving unit 110 is means for receiving the image 1000 including the table 1100, and may include at least one of a communication unit, a scanning unit, and an input unit. can be

The character recognition system 100 can recognize the table 1100 included in the image 1000 received via the receiving unit 110 and convert the contents included in the table 1100 included in the image 1000 into data. As a result of that recognition, the character recognition system 100 can secure data that matches what is contained in the table 1100 .

In the present invention, the character recognition system 100 recognizes the content (or content, information) of the table 1100 included in the image 1000, corrects any errors in the recognized content, and It is possible to provide a method for accurately digitizing the content that can be obtained.

In addition, the contents recognized from the table 1100 included in the image 1000 according to the present invention can be used to generate table data having the same or similar structure as the table 1100 included in the image 1000. FIG.

The table data should consist of data formatted or structured based on at least one of the cell structure and cell relationship (relationship between category cells and data cells, etc.) forming the table 1100 included in the image 1000. may

The table data includes data corresponding to the contents recognized by the character recognition system 100 according to the present invention.

In the present invention, the positional relationship of the components particularly includes the positional relationship of the cells included in the table, the positional relationship according to the row or column relationship, and the semantic relationship of the components includes It contains the relationships formed between the meanings implied by the contained text.

Next, the storage unit 120 may store various information according to the present invention. The storage unit 120 has a wide variety of types, and at least part of it means the external server 150 (at least one of the cloud server 151 and the database (DB) 152). That is, the storage unit 120 may be any space in which information related to the present invention is stored, and there is no physical space limitation.

The storage unit 120 contains information about various components that make up the table. The storage unit 120 stores i) an image 1000 including the table 1100 and data related thereto, ii) cell information of the table 1100 recognized from the image 1000, components (e.g., lines, corners, etc.) and data related thereto; At least one of iii) content (eg, text, image, etc.) data included in table 1100 and iv) data sets associated with the content included in table 1100 are stored. Here, the data set may be data used to verify or correct the contents included in the table 1100. FIG.

Next, the OCR unit 130 is a means for recognizing the content (or information) included in the image 1000, and can recognize the content included in the image 1000 by at least one of various content recognition algorithms. The OCR unit 130 can recognize content using algorithms based on artificial intelligence (or deep learning algorithms). Here, the content may include text (characters). The OCR part can also be named "OCR API".

The OCR unit 130 can recognize the text included in the image 1000 and the location information of the text. Here, the text position information includes at least one of information about the position of the text within the image 1000 input via the receiving unit 110 and information about the position of the text within the table 1100 .

OCR unit 130 can recognize the text contained in each cell based on the text contained in image 1000 .

The table 1100 is composed of a plurality of cells, and the OCR section 130 can recognize text included in each cell.

For example, as shown in FIG. 1, when the text "patient name" is recognized from the first cell 1101 of the table 1100, the identification information in the first cell 1101 and the text "patient name" are matched, Such matching information is stored in the storage unit 120 .

In the present invention, the OCR unit 130 can distinguish and recognize the text contained in each cell. Therefore, in the present invention, which cell contains which text is matched between the identification information of the cell and the corresponding text. Such matching information is stored in the storage unit 120 .

In this way, the text recognized from each cell is matched with the identification information of each cell and stored, and the identification information of the cell includes position information of the cell with respect to at least one of the image 1000 and the table 1100. is included.

Such matching information is generated by the OCR unit 130 while text is being recognized from the image 1000 or generated under the control of the control unit 140 .

Next, the control unit 140 controls general operations of the character recognition system 100 related to the present invention. The control unit 140 may include a processor (or artificial intelligence processor, deep learning processor) that processes an artificial intelligence algorithm (or deep learning algorithm). The control unit 140 can recognize the table 1100 from the image 1000 and recognize at least one cell forming the table 1100 based on an artificial intelligence algorithm.

In addition, the controller 140 can analyze the relationship between cells and the meaning of text included in the cells based on an artificial intelligence algorithm.

Additionally, the control unit 140 can correct at least a portion of the text recognized by the OCR unit 130 based on the cell relationships and the meaning of the text contained in the cells.

Here, the control unit 140 can correct the text recognized by the OCR unit 130 based on the data set stored in the storage unit 120 .

A cell in the present invention means a rectangular box defined by a plurality of lines forming a table.

Before describing the present invention, the constituent elements of the table will be described. A "table" in the present invention means a representation of certain contents in a predetermined format or procedure, and is also called a table. Referring to FIG. 2 to describe the components of the table, table 200 includes at least one cell (or space, area) 210 . That is, table 200 includes at least one cell 210 and is configured to include information within cell 210 .

Cells 210 included in table 200 may be defined by at least four line segments. The table 200 and the cells provided in the table 200 are rectangular.

That is, a cell included in the table may consist of a rectangle surrounded by four line segments. A "line segment" in the present invention refers to a line or at least part of a line. A line segment forming each cell extends to form a line or a line.

Thus, at least four line segments a, b, c, and d are required to form one square-shaped cell, which is equivalent to the two horizontal lines ( 201 , 202 in the row direction, and at least part of two vertical lines (or lines, edges in the column direction) 203 , 204 . More specifically, a rectangular cell may be formed by at least a portion of two horizontal lines 201, 202 and two vertical lines 203, 204 crossing each other.

Therefore, the table 200 includes at least four lines, and the number of cells, the size of cells, the positions of cells, etc. are defined by the number of horizontal lines, the number of vertical lines, the spacing between lines, and the arrangement position of the lines. be.

On the other hand, the plurality of lines forming the table 200 are classified into either a first type or a second type, which are specified by the direction in which the lines extend.

As shown, horizontally extending lines 201, 202 are defined as a first type and vertically extending lines 203, 204 are defined as a second type.

Lines of the first type are variously named with terms that have the same or similar meanings, such as horizontal lines, horizontal lines, row lines, and horizontal lines.

In addition, the second type lines are variously named with terms having the same or similar meanings, such as vertical line, vertical line, column line, and vertical line.

Meanwhile, the number of cells included in the table 200, the size of the cells, the positions of the cells, etc., can be variously modified based on the number of horizontal lines and vertical lines, the arrangement relationship, etc., and can be variously changed by combining the cells. can be transformed into

In addition, the components that make up the table in the present invention are the lines that make up the table (horizontal lines and vertical lines) and the corners (or corners, vertices) formed by the intersection of the lines that make up the table (symbol "e , f, g, h”).

All tables have at least four corners, which may be the corners included in the outermost frame of the table. Also, the number of corners included in the table depends on the number of cells included in the table.

Hereinafter, a method for generating a table based on the configuration of the character recognition system according to the present invention will be described in more detail.

As shown in FIG. 3, in the character recognition method according to the present invention, first, a process of receiving an image 1000 (see FIG. 1) including a table 1100 is performed (S310).

As mentioned above, the image 1000 containing the table 1100 can be received by various routes. For example, the image 1000 can be received in a manner transmitted by a communication unit, scanned by a scanning unit, input by an input unit, or the like.

When the image 1000 is received, the process of recognizing text from the table 1100 included in the image 1000 is performed.

More specifically, in the present invention, a process of recognizing texts included in a plurality of cells forming the table 1100 is performed (S320). Such recognition is performed by the OCR section 130 , which recognizes the information or content contained in the table 1100 . Here, recognizing the information or contents included in the table 1100 means that the text included in the table 1100 is recognized.

As a result of recognition by the OCR unit 130, data corresponding to the contents included in the table 1100 can be secured. For example, the OCR unit 130 recognizes the text included in the table 1100, and the text corresponding to the contents included in the table 1100, such as "patient name", "resident registration number", "medical expense (drug expense) breakdown", etc. text can be secured as data. The OCR unit 130 can recognize text from the image 1000 with a text recognition model trained to recognize each text (or character) contained in the image 1000 and the location of the text. Such text recognition models may include artificial intelligence-based algorithms (eg, deep learning algorithms).

As shown in FIG. 4(a), OCR unit 130 recognizes text from table 1100, and as shown in FIG. 4(b), the recognized text is an identification of each cell containing the text. Information KEY1, KEY2, KEY3 . . . are matched together and stored in the storage unit 120 .

As such, when the text is recognized for the cells forming the table 1100, a process of verifying the recognized text may be performed. Here, 'verification' means a process of checking whether the text included in each cell has been correctly recognized. For example, as shown in (a) of FIG. 4, a specific cell 401 of table 1100 contains the text "union contribution amount", but as shown in (b) of FIG. The text "Rough Total Cost" 402 may be recognized.

In this case, the recognition (erroneous recognition) of the content different from the original document causes problems of the reliability and accuracy of the recognized data.

Therefore, the control unit 140 may verify the text recognized from the table 1100 . In this case, the present invention may correct the text contained in the cells that need to be corrected in the verification process based on the mutual positional relationship and semantic relationship (or connection relationship) between the cells.

Therefore, in the present invention, a process of identifying a correction target cell from a plurality of cells is performed (S330). The control unit 140 may specify at least one correction target cell from a plurality of cells based on preset criteria. Here, the preset criteria can be set in various ways, and therefore, the process of specifying the correction target cell in the present invention is very diverse.

For example, when text is recognized in a plurality of cells forming the table 1100, the control unit 140 may verify all cells in which text recognition has been performed.

In this case, all cells in which text recognition has been performed are specified as correction target cells.

Alternatively, the control unit 140 may verify the text included in the category cell among the plurality of cells forming the table 1100 .

In this case, control unit 140 identifies each of the texts included in the category cells among the plurality of cells forming table 1100 as correction target cells.

Here, category cells are also called key cells and are distinguished from data cells (or value cells).

A category cell is also called a title cell, and is understood as a cell containing text that defines the category, type, affiliation, characteristics, etc. of the data (text) contained in the data cell.

Categories are divided based on the same or similar properties and meanings, and are also called categories. Data belonging to the same category may be data corresponding to the same meaning, the same type, or the same affiliation.

A category cell contains a category name or data related to the category name, and a data cell contains data belonging to the category name contained in the category cell. Here, the category name is also named category name.

As an example, as shown in FIG. 4A, in the table 1100, a specific cell 412 containing the text "medical care, dispensing date (medical care period)" is a category cell and is the same as the specific cell 412. Cells 403 located in columns and each containing date information are data cells associated with a particular cell 412 .

As another example, in the table 1100, a particular cell 411 containing the text "Patient Name" is a category cell, is located in the same row as the particular cell 411, and contains the name information "Hong Gil Dong". Cell 405 is a data cell associated with a particular cell 411 .

As shown in FIG. 4, table 1100 includes one or more category cells 411 , 412 , 413 , 414 , 415 , 416 .

Based on the structure of the table 1100 and the meaning of the text contained in the cells, the control unit 140 classifies the types of the plurality of cells forming the table 1100 into a first type (category cell type) and a second type (data cell type). can be specified.

Further, the control unit 140 can specify each of the texts included in the category cells among the plurality of cells forming the table 1100 as correction target cells.

As another example, the control unit 140 can specify at least some of the category cells among the plurality of cells forming the table 1100 as correction target cells. Here, the preset criteria for specifying the correction target cell may be related to the recognition rate for the text included in the plurality of cells. The control unit 140 may specify a specific cell having a low text recognition rate at the time of recognition and extraction of the text among the texts extracted from the category cells, as a correction target cell.

As shown in FIG. 5, when text is recognized for each cell by the OCR unit 130, the recognition reliability of each text can be specified.

Such confidence is also named confidence score or confidence score.

As shown in FIG. 5 , such reliability scores may be matched with at least one of each cell constituting the table 1100 and the text recognized from the cell, and stored in the storage unit 120 .

As shown in FIG. 5, the control unit 140 selects a category cell that satisfies a preset reference condition for text reliability (for example, a cell whose text reliability is less than a reference score (such as a reference probability)) as a correction target. can be identified as a cell. According to FIG. 5, the closer the confidence score is to 1, the higher the probability that the recognized text was recognized correctly, and the closer the confidence score is to 0, the lower the probability that the recognized text was recognized correctly. means

In the present invention, the control unit 140 corrects the text contained in the cells in which the probability of correct recognition of the text is low, thereby reducing the amount of data processing operations and accurately extracting the information contained in the table 1100. can do. In this way, the control unit 140 does not specify all recognized texts as correction target texts, but specifies texts with a low probability of being correctly recognized based on the reliability as correction target texts. Therefore, since the control unit 140 performs the correction according to the present invention only on the text specified as the text to be corrected, it is possible to reduce the amount of calculation for data processing compared to the case where all the texts are corrected. .

For example, when the preset reference condition is set to correct a cell whose text confidence score is less than 0.5, the control unit 140 corrects a cell corresponding to 'KEY7' in FIG. Identify as the target cell. In this case, the text corresponding to "(1) total cost" contained in the cell "KEY7" is corrected.

In this way, when the correction target cell is specified based on various methods or criteria, the control unit 140 selects at least one cell related to the correction target cell among the plurality of cells included in the table 1100 and The text contained in the correction target cell can be corrected using the content (text) included in the correction target cell.

To this end, in the present invention, a process of identifying related cells related to the correction target cell from a plurality of cells included in the table 1100 is performed (S340).

For convenience of explanation, the following description assumes that the cell to be corrected is the specific category cell 601 containing the text "(1) Co-payment amount" shown in FIG. 6A. The following description can be commonly applied to all category cells.

As shown in FIGS. 4, 6A, and 6B, in the image 1000, a specific category cell 601 contains the text corresponding to "(1) Association contribution amount", but as described in FIG. Due to an error, the text "(1) total cost" may be recognized. Therefore, the control unit 140 uses the specific category cell 601 containing the erroneously recognized text and at least one related cell related to the specific category cell 601 to identify the erroneously recognized text (for example, (1 ) can be corrected.

The control unit 140 can specify at least one related cell using the correction target cell 601 as a reference.

The control unit 140 can identify related cells based on at least one of the meaning of the text included in the correction target cell 601 and the position of the correction target cell 601 .

Here, the related cell of the correction target cell 601 may be a category cell. The control unit 140 can specify only the category cells among the category cells and data cells included in the table 1100 as related cells of the correction target cell 601 .

In addition, the control unit 140 identifies, among the category cells, category cells containing text semantically related to the correction target cell 601 as related cells, or identifies category cells that are positionally adjacent to the correction target cell 601. It can be identified as an associated cell.

When identifying a cell related to the correction target cell 601, the control unit 140 can identify the related cell based on any of the following cases.

As a first case, the control unit 140 can identify a category cell containing text semantically related to the correction target cell 601 as a related cell.

Also, as a second case, the control unit 140 can identify a category cell that is positionally related to the correction target cell 601 as the related cell.

Furthermore, as a third case, the control unit 140 combines the first case and the second case, identifies a category cell containing text semantically related to the correction target cell 601 as a related cell, and positionally Category cells related to the correction target cell 601 can be identified as related cells. All of the cells identified as related cells in the third case, first case, and second case can be identified as related cells of the correction target cell 601 .

Hereinafter, as a specific example of the first case, a method of specifying related cells based on the meaning of the text included in the correction target cell 601 will be described.

First, the method of identifying related cells based on the meaning of the text included in the correction target cell 601 will be described. One category cell can be identified as an associated cell.

The control unit 140 analyzes the meaning of the text contained in the correction target cell 601 and the meaning of the text contained in the other category cells based on an artificial intelligence algorithm, and analyzes at least one Category cells can be identified as related cells.

The text having a meaning related to the correction target cell 601 may be a text having a higher concept or a lower concept meaning than the text included in the correction target cell 601 .

Since the control unit 140 specifies the correction target cell 601 and related cells for the category cell, the correction target cell 601 has a meaning corresponding to a higher concept or a lower concept for the related cell due to the characteristics of the category cell. Contains text. For example, the correction target cell 601 and related cells are composed of texts having meanings corresponding to the same or related categories (kinds).

For example, as shown in FIG. 6A, the text contained in the first category cell 611 (“Medical Expenses (Drug Expenses) Details”) is the text in the second category cell 612 (“Total (1)+(2)+(3) ”), third category cell 613 (“salary”), fourth category cell 615 (“(2) patient burden”), fifth category cell 618 (“total patient burden (2) + (3)”), Sixth category cell 601 (correction target cell, ``(1) Association burden'', recognized as ``(1) Total burden'' at the time of OCR recognition), Seventh category cell 617 (``(3) Patient burden ”), which has a broader meaning for the text contained in the eighth category cell 616 (“Non-Salary”).

Also, the category cells of the lower concept with respect to the first category cell 611 are also the category cells of the higher concept with respect to other category cells. For example, the text “salary” included in the third category cell 613 is a subordinate concept to the text “medical expense (drug expense) breakdown” included in the first category cell 611, and is included in the fourth category cell 615. The text "(2) patient burden" contained in the text "(2) patient burden" and the text "(1) association burden" contained in the sixth category cell 601 (correction target cell) (recognised as "(1) rough total burden" at the time of OCR recognition) ) is a superordinate concept.

On the other hand, the control unit 140 can identify a category cell including a text semantically related to the correction target cell 601 among the category cells as a related cell. A category cell containing text having a superordinate or subordinate meaning of the text contained therein can be identified as a related cell.

In the first case, the related cells of the correction target cell 601 are the first category cell 611 (“medical expense (drug expense) breakdown”) and the third category cell 613 (“ salary”).

Hereinafter, as a specific example of the second case, a method of specifying category cells that are positionally related to the correction target cell 601 as related cells will be described. More specifically, in consideration of the characteristics of the table 1100, the control unit 140 selects a category cell having a preset positional relationship (or arrangement relationship) with the correction target cell 601 according to the text included in the correction target cell 601. can be determined as category cells related to the meaning of , ie related cells.

Here, the preset positional relationship may mean being positioned adjacent to the correction target cell 601 . For example, second category cell 612 (“total amount (1) + (2) + (3)”), third category cell 613 (“salary”), fourth category cell 615 (“(2) patient burden”) And the specific data cell 622 is positioned adjacent to the correction target cell 601 and can be said to have a preset positional relationship with the correction target cell 601 .

In this way, the cells having a preset positional relationship with the correction target cell 601 are positioned side by side with the correction target cell 601 in the column direction a or the row direction b with the correction target cell 601 as a reference. It corresponds to a cell located adjacent to the .

Here, the cells aligned in the column direction a of the correction target cell 601 are cells corresponding to reference numerals 621, 611, 613, 601, and 622 in FIG. The located cells are cells corresponding to 641, 642, 612, 601, 615, 617, 618, 643, 644 and 645 of FIG. 6A.

On the other hand, since the control unit 140 specifies only the category cells as related cells of the correction target cell 601, the cells having the previously set positional relationship (the second category cell 612 (“Total amount (1)+(2) +(3)”), the third category cell 613 (“salary”), the fourth category cell 615 (“(2) patient cost”), and the specific data cell 622), the specific data cell 622 is the related cell excluded from

As a result, as shown in FIG. 6B, the control unit 140 selects a second category cell 612 (“Total amount (1)+(2)+(3) ”), the third category cell 613 (“salary”), and the fourth category cell 615 (“(2) patient cost”) can be identified as related cells.

According to the second case, as related cells of the correction target cell 601, the second category cell 612 ("total amount (1)+(2)+(3)"), the third category cell 613 ("salary"), the third A 4-category cell 615 (“(2) patient cost”) is identified.

On the other hand, in order to specify related cells having a positional relationship with the correction target cell 601, the control unit 140 calculates the median value of the corners (vertices) of the correction target cell 601 from the correction target cell on the table 1100 or the image 1000. The position of 601 is specified, and category cells 612, 613, 615 having median values located within a preset distance from the position can be specified as related cells 612, 613, 615. FIG.

Here, the preset distance may be based on L2 distance information, and the L2 distance information can also be said to be Euclidean distance information.

In this way, the control unit 140 determines the median values located within a preset distance from the median values of the corners (vertexes) of the correction target cell 601 in each of the left, right, top, and bottom directions in the table 1100. It is possible to check whether there is a category cell with Also, if a category cell having a median located within a preset distance exists as a result of checking, it can be identified as a related cell.

As described above, the control unit 140 can identify related cells related to the correction target cell 601 depending on the first case or the second case.

Also, the control unit 140 can identify all of the related cells identified by the first case and the related cells identified by the second case as related cells of the correction target cell 601 . This case corresponds to the third case described above.

As described above, the control unit 140 identifies, from among the category cells included in the table 1100, category cells containing text semantically related to the correction target cell 601 in the first case as related cells, and According to the second case, category cells that are positionally adjacent to the correction target cell 601 can be identified as related cells. As a result, the related cells of the correction target cell 601 are, as shown in FIGS. +(2)+(3)”), the third category cell 613 (“salary”), and the fourth category cell 615 (“(2) patient burden”).

Here, at least one category cell (eg, the third category cell 613 ("salary") is semantically and positionally redundant and identified as a related cell.

The category cells 611, 612, 613, and 615 identified as the related cells of the correction target cell 601 may of course be named "related cells" and use the same reference numerals.

In the following, as an example of the third case, a method of correcting the text contained in the correction target cell 601 using the related cells identified by considering both the semantic relationship and the positional relationship will be described. do.

As described above, when the correction target cell 601 and at least one related cell are specified in the third case, the control unit 140 controls the correction target cell 601 and related cells 611, 612, and 613 as shown in FIG. 6C. , 615 is used to embed the correction target cell 601, and a dense vector calculated (or calculated or derived) can be used to correct the correction target cell 601. . In the present invention, in order to show the relationship between the texts contained in the correction target cell and related cells, the text contained in the correction target cell and related cells is embedded, and the text contained in the correction target cell and related cells is embedded. It can be represented by a vector.

Here, embedding is a method of expressing information to be embedded using a vector. It can be expressed in the form of a dense vector.

More specifically, the control unit 140 can calculate a dense vector indicating the correction target cell 601 using the text included in each of the correction target cell 601 and related cells 611 , 612 , 613 , and 615 .

Also, the control unit 140 can use the dense vector calculated using the correction target cell 601 and related cells 611 , 612 , 613 , and 615 to correct the correction target cell 601 .

In the present invention, the embedding algorithm is not particularly limited, and any algorithm that can express text as a dense vector can be used in the present invention. For example, the control unit 140 calculates a dense vector indicating the correction target cell 601 based on the text included in the correction target cell 601 and the related cells 611, 612, 613, and 615 based on algorithms such as FastText and BERT. can be done.

On the other hand, the control unit 140 arranges texts 631, 632, 633, 634, and 635 included in the correction target cell 601 and related cells 611, 612, 613, and 615 in order to embed them, as shown in FIG. 6C. can be done. Here, the control unit 140 determines the arrangement order of the texts 631, 632, 633, 634, and 635 included in the correction target cell 601 and the related cells 611, 612, 613, and 615 based on preset criteria. be able to.

For example, the control unit 140 can arrange the texts 632 and 634 included in the related cells 611 and 613 having semantic relevance closer to the text 631 included in the correction target cell 601. The texts 633 , 634 , 635 included in the related cells 612 , 613 , 615 that are related to each other can be arranged closer to the text 631 included in the correction target cell 601 .

The control unit 140 can also arrange the text 634 included in the related cell 613 having both semantic and positional relevance at the position closest to the text 631 included in the correction target cell 601 .

In this manner, the control unit 140 can indicate the correction target cell 601 by embedding vectors based on the texts included in the correction target cell 601 and related cells 611 , 612 , 613 , and 615 . In this way, when embedding into the correction target cell 601 is completed based on the texts 631, 632, 633, 634, and 635 contained in the correction target cell 601 and related cells 611, 612, 613, and 615, the present invention performs , a process of correcting the correction target cell is performed (S350).

The control unit 140 corrects the text included in the correction target cell 601 using the vector of the correction target cell calculated using the texts 631, 632, 633, 634, and 635 included in the correction target cell 601 and related cells. It can be performed. Here, the vector of the correction target cell 601 may correspond to the dense vector described above.

More specifically, the control unit 140 selects a word having a vector representation that is most similar to the vector of the correction target cell 601 (for example, a specific vector located closest to the correction target vector) from a data set specified in advance. ) can be extracted as correction target text.

Based on the distance between the vector of the correction target cell calculated using the correction target cell 601 and the related cells 611, 612, 613, and 615 and the vector included in the pre-specified data set, the control unit 140 pre-specifies The text to be corrected can be extracted from the data set. Here, the pre-specified data set may include candidate texts and corresponding vectors that may be included in a plurality of cells included in table 1100, particularly category cells. Such datasets may be pre-constructed and the candidate texts that may be included in the dataset may be further included in or extracted from the dataset after the dataset is constructed. It can be removed.

In addition, such a data set includes the result of embedding based on at least one of the positional relationship of a plurality of cells forming the table 1100 and the semantic relationship between candidate texts contained in each cell. good too. That is, the dataset may include vector information for each of the candidate texts.

The control unit 140 can extract the text 806 (see FIG. 7) corresponding to the specific vector closest to the vector of the correction target cell 601 from the data set as the correction target text.

Also, the control unit 140 can change the text included in the correction target cell 601 to a text 806 (see FIG. 7) corresponding to a specific vector closest to the vector of the correction target cell 601 .

In this way, the control unit 140 uses the vector of the correction target cell 601 derived using the texts 801 to 805 included in the correction target cell 601 and related cells 611, 612, 613, and 615, from the data set, Correction target text 806 can be extracted.

As a result, the erroneously recognized text "(1) Co-payment amount" for the correction target cell 601 is corrected to "(1) Co-payment amount".

When the correction target text 806 is extracted from the data set in this way, the control unit 140 can change the text 801 included in the correction target cell 601 to the correction target text 806 .

Meanwhile, as described above, when the correction target text 806 is extracted, the control unit 140 can utilize the extracted correction target text 806 to convert the table 1100 included in the image 1000 into data. The control unit 140 can include the correction target text 806 in the data obtained as a result of digitizing the table 1100 included in the image 1000 .

Moreover, the control unit 140 can not only convert the table 1100 into data, but also generate a table having a structure corresponding to the table 1100 included in the image 1000 using the data obtained as a result of the data conversion. .

For example, as shown in (a) of FIG. 8, the control unit 140 selects a text 930 (“Other than Kanzawa Department”) corresponding to at least a part of the data extracted from the table 910 included in the scanned image. If there is an error in the above, it is possible to secure data including the corrected text 940 (“Other than selected medical care fee”) as shown in FIG. . Also, if desired, a table 920 containing the corrected text 940 can be generated, and such a table 920 has the same content as the table 910 that was scanned.

As described above, the character recognition method and character recognition system according to the present invention recognizes text from a table included in an image and verifies the recognized text to more accurately identify the contents of the table included in the image. It can be converted into data.

More specifically, the character recognition method and the character recognition system according to the present invention correct the text contained in the table contained in the image when the text contained in the table is erroneously recognized from the table contained in the image. can be converted into data more accurately. For this purpose, in the present invention, the arrangement of cells constituting a table and the semantic relation of text contained in the cells are taken into consideration, and the text contained in the cells to be corrected is corrected so that the text contained in the image is corrected. It is possible to provide a highly accurate character recognition method and system that can convert the contents of a table stored in the table into data as they are. As a result, the character recognition method and system according to the present invention can generate a table containing the same content as the table contained in the image.

In addition, the character recognition method and character recognition system according to the present invention can reduce the amount of data processing while accurately converting the contents of a table included in an image into data by correcting only specific cells with a low text recognition rate. can be minimized.

On the other hand, the present invention described above can be implemented as a program that is executed by one or more processes on a computer and can be stored in a computer-readable medium (or recording medium).

Also, the present invention described above can be implemented as computer readable codes or commands on a program recording medium. That is, the present invention can be provided in the form of a program.

A computer-readable medium, on the other hand, includes any type of recording device on which data readable by a computer system is recorded. Examples of computer-readable media include HDDs (Hard Disk Drives), SSDs (Solid State Disks), SDDs (Silicon Disk Drives), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like. be done.

Computer-readable media also includes storage, which may be a server or cloud storage communicatively accessible by the electronic device. In this case, the computer can download the program according to the present invention from a server or cloud storage via wired or wireless communication.

Furthermore, in the present invention, the aforementioned computer is an electronic device equipped with a processor, that is, a central processing unit (CPU), and its type is not particularly limited.

On the other hand, the detailed description of the present invention is illustrative and should not be construed as restrictive in any aspect. The scope of the invention should be determined by reasonable interpretation of the appended claims, and all changes that come within the equivalent scope of the invention are included in the scope of the invention.

100 Character Recognition System 110 Reception Unit 120 Storage Unit 130 OCR Unit 140 Control Unit 150 External Server 151 Cloud Server 152 Database (DB)
200 table 201, 202 horizontal line 203, 204 vertical line 210 cell 1000 image 1100 table

Claims (14)

receiving an image containing a table;
recognizing text contained in a plurality of cells forming the table;
identifying a correction target cell from the plurality of cells based on preset criteria;
identifying at least one related cell related to the correction target cell from the plurality of cells;
and a step of correcting the text contained in the correction target cell using the vector of the correction target cell calculated using the text contained in the correction target cell and the related cell. character recognition method. In the step of identifying related cells,
2. The character recognition method according to claim 1, wherein the related cell is specified based on at least one of the meaning of the text included in the correction target cell and the position of the correction target cell. at least a portion of the plurality of cells are identified as category cells and another portion are identified as data cells containing data corresponding to at least one of the category cells;
In the step of identifying related cells,
3. The method according to claim 2, wherein, among the category cells, a specific category cell related to at least one of the meaning of the text included in the correction target cell and the position of the correction target cell is specified as the related cell. Described character recognition method. In a specific category cell related to the meaning of the text contained in the correction target cell,
4. The character recognition method according to claim 3, wherein text having a meaning of a higher concept or a lower concept of the text included in the correction target cell is included. A specific category cell related to the meaning of the text contained in the correction target cell is
5. The character recognition method according to claim 4, wherein the correction target cell is aligned with the correction target cell in a column direction or a row direction with the correction target cell as a reference. The specific category cell is a cell located adjacent to the correction target cell in a row direction or a column direction with respect to the correction target cell on the table, among the category cells. 4. The character recognition method according to claim 3. In the correcting step,
2. The character recognition method according to claim 1, wherein the text included in the correction target cell is corrected based on the distance between the vector of the correction target cell and the vector included in a data set specified in advance. . The correcting step includes:
extracting text to be corrected from the data set based on the distance between the vector of the cell to be corrected and a vector included in a previously specified data set;
8. The character recognition method according to claim 7, further comprising the step of changing the text included in the correction target cell to the correction target text. Said data set includes:
Candidate texts that can be included in the plurality of cells constituting the table; and
8. Vector information about each of said candidate texts corresponding to a result of embedding based on at least one of a positional relationship of said plurality of cells and a semantic relationship between said candidate texts is included. The character recognition method described in . In the correcting step,
Identifying the vector closest to the vector of the correction target cell from the data set,
10. The character recognition method according to claim 9, wherein the text corresponding to said closest vector is extracted as said text to be corrected. 11. The character recognition method according to claim 10, further comprising the step of generating a table having a structure corresponding to a table included in the image and including the text to be corrected in the cell to be corrected. 2. The character recognition method according to claim 1, wherein the preset criteria for specifying the correction target cell relate to reliability of recognition of text included in the plurality of cells. . a storage unit;
a receiver for receiving an image containing a table;
a control unit for recognizing text contained in a plurality of cells forming the table contained in the image;
The control unit
Identifying a correction target cell from the plurality of cells based on preset criteria;
identifying at least one related cell related to the correction target cell from the plurality of cells;
A character recognition system, wherein the text included in the correction target cell is corrected using a vector of the correction target cell calculated using the text included in the correction target cell and the related cell. A computer program comprising a plurality of instructions,
When the instruction is executed
receiving an image containing a table;
recognizing text contained in a plurality of cells forming the table;
identifying a correction target cell from the plurality of cells based on preset criteria;
identifying at least one related cell related to the correction target cell from the plurality of cells;
and a computer executing a step of correcting the text contained in the correction target cell using the vector of the correction target cell calculated using the text contained in the correction target cell and the related cell. A computer program characterized by:

Images