KR102673900B1 - Table data extraction system and the method of thereof - Google Patents

Abstract

A table data extraction system and method are disclosed. A table data extraction system according to an embodiment of the present invention includes an image input unit into which at least one image document data is input; a table type determination unit that determines whether the input image document data includes at least one table data and determines a type of the table data; Based on the type of table data determined by the table type determination unit, a table data area included in the image document data is extracted, the extracted table data area is divided into pixels to extract a line area, and the extraction a cell area designation unit that designates a cell area based on the drawn line; and a data extraction unit that extracts text data included in the cell area designated by the cell area designation unit.

Description

Table data extraction system and method {TABLE DATA EXTRACTION SYSTEM AND THE METHOD OF THEREOF}

The present invention relates to a system and method for extracting table data included in image data, and more specifically, to receive document image data, determine the type of table included in the document image data, extract table data, and convert it into data. It relates to a table data extraction system and method.

Recently, various optical character recognition (OCR)-related technologies that can recognize characters from images are emerging.

For example, there are services that recognize characters from images and provide an automatic translation function for the characters, or services that recognize characters in signage images of nearby objects and search for and provide related information about nearby objects through the Internet.

Meanwhile, when people receive treatment at a hospital and receive a receipt and prescription, they generally receive the receipt and prescription in the form of printed paper documents. Additionally, after taking the test at school, students receive a test score report in the form of a printed paper document.

However, if paper documents have been stored for a long time, there is a risk of losing the location where they were stored, and if the document needs to be found later, there is the inconvenience of not being able to check it immediately through an electronic device such as a smartphone.

In order to solve this problem, technologies for scanning documents and extracting text information from scanned images are emerging, and the conventional technology, Korean Patent Publication No. 10-2010-0120753, uses image processing of images captured through an image sensor. Discloses an image sensor and image sensing method for character recognition that removes non-text information in a specific image area and converts information in the image area determined as text into binary information, Korean Patent Publication No. 10-2189373 discloses a device and method for generating text from images using a neural network.

More specifically, a neural network is a network structure that processes information by imitating the human brain's neural network and refers to a technology for inferring images or text by applying a learned algorithm.

However, when a table is inserted into a scanned image, the area corresponding to the table is saved in a simple image file format rather than in the form of an electronic document. Therefore, if you want to record it in an electronic document format that can be standardized and edited, you must create the tables one by one and There is an inconvenience in having to enter text into the multiple cells that make up or go through the process of entering them one by one.

If it were possible to create an electronic document with a table inserted from the scanned image by extracting the lines that make up the table from the scanned image with the table inserted and extracting the text written inside the table using OCR, it would be possible to create an electronic document with a table inserted. will be able to promote.

Therefore, research is needed on technology that can convert images with tables inserted into electronic documents and build a knowledge database based on table data inserted in the converted electronic documents.

The present invention provides a table data extraction system and method that can easily convert information existing in paper form into digital data by extracting a table from input image data, determining the type of table, and extracting table data. The purpose is to do this.

In addition, by dividing the image data into pixel units and extracting the line area included in the table area to specify the cell area, and extracting the feature area of the table where no lines exist and specifying the cell area, the table data is stored regardless of the type of table. The purpose is to provide a table data extraction system and method that can accurately extract.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems to be solved by the present invention that are not mentioned herein can be explained to those skilled in the art from the description below. You will be able to understand it clearly.

The table data extraction system according to an embodiment of the present invention includes an image input unit where at least one image document data is input, determines whether the input image document data includes at least one table data, and determines the type of table data. Table type determination unit, extracts the table data area included in the image document data based on the type of table data determined by the table type determination unit, and extracts the line area by dividing the extracted table data area in pixel units. and a cell area designation unit that designates a cell area based on the extracted line, and a data extraction unit that extracts text data included in the cell area designated by the cell area designation unit.

In addition, the table type determination unit determines the table type included in the image document data as either a standard type or a non-standard type by comparing the image document data input to the image input unit with the training image data stored in a predefined learning DB. is classified as a standard type, first training image data whose cell areas are separated by continuous lines, classified as a non-standard type, and second training image data whose cell areas are not separated by continuous lines and classified into a non-standard type, , the cell area includes third training image data including both an area separated by continuous lines and an area not separated by lines, and the cell area designation unit is a table for extracting the table data area included in the image document data. If the image document data is determined to be a standard type in the area extraction unit and the table type determination unit, the extracted table data area is divided into pixels, the color of the pixel is determined to be black and white, and the pixel determined to be black is A line area extraction unit that recognizes the line area by determining whether the number is more than a preset number, and a feature area extraction unit that extracts at least one feature area from the extracted table data area when the table type determination unit determines that the image document data is a non-standard type. It is characterized in that it includes a cell area generation unit that calculates the number of rows and columns and generates a cell area based on the line area recognized by the part and line extraction unit and the feature area extracted by the feature area extraction unit.

In addition, when the table type determination unit determines that the image document data is the non-standard type by comparing it with the second training image data, the feature area extraction unit divides the extracted table data area into pixel units and pixels within the table data area. It is characterized by comprising a no-line feature extraction unit that detects a change and extracts a feature region, calculates the number of rows and columns of the feature region, and merges the feature region and the feature region based on the number of rows and columns.

In addition, when the table type determination unit determines that the image document data is a non-standard type compared to the third training image data, at least one line area and a feature area included in the image document data are extracted, and the intersection of the line areas is determined at least. It includes a broken line feature extraction unit that extracts one, creates a group area by grouping the feature areas based on the intersection of the line areas, and determines the table structure by calculating the number of rows and columns in the group area and the number of rows and columns in the feature area. It is characterized by

In addition, it further includes a data verification unit that verifies the accuracy of the text data extracted from the data extraction unit, wherein the data verification unit includes at least one of special characters, consonants, and vowels included in the text data extracted from the data extraction unit. A first extraction unit for extracting single characters, a second extraction unit for extracting at least one word data corresponding to text data from a predetermined learning DB, and text based on the data extracted from the first extraction unit and the second extraction unit. It includes an error verification unit that determines whether the data contains errors, and the learning DB applies the word data stored in the input word storage unit and the input word storage unit to a predetermined learning algorithm where word data input from a predetermined administrator terminal is stored. It is characterized in that it includes a vector data generation unit that generates vector data and maps vector data and word data.

The table data extraction system and method of the present invention has the effect of easily converting information existing in paper form into digital data by extracting a table from input image data, determining the type of table, and extracting table data. has

In addition, by dividing the image data into pixel units and extracting the line area included in the table area to specify the cell area, and extracting the feature area of the table where no lines exist and specifying the cell area, the table data is stored regardless of the type of table. It has the effect of accurately extracting.

1 is a configuration diagram of a table data extraction system according to an embodiment of the present invention.
Figure 2 is a diagram for explaining the cell area designation unit of the table data extraction system according to an embodiment of the present invention.
Figure 3 is a diagram for explaining standard type table data extraction of the table data extraction system according to an embodiment of the present invention.
Figure 4 is a diagram for explaining the feature region extraction unit of the table data extraction system in one embodiment of the present invention.
5 and 6 are diagrams for explaining non-standard table data extraction of the table data extraction system according to an embodiment of the present invention.
Figure 7 is a diagram for explaining the data verification unit of the table data extraction system according to an embodiment of the present invention.

Specific details, including the problems to be solved by the present invention, the means for solving the problems, and the effects of the invention, are included in the examples and drawings described below. The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings.

Hereinafter, the present invention will be described in more detail with reference to the attached drawings.

FIG. 1 is a configuration diagram of a table data extraction system according to an embodiment of the present invention, FIG. 2 is a diagram illustrating a cell area designation unit of the table data extraction system according to an embodiment of the present invention, and FIG. 3 is a diagram showing the configuration of a table data extraction system according to an embodiment of the present invention. It is a diagram for explaining the standard type of table data extraction of the table data extraction system according to an embodiment of the present invention, and Figure 4 is a diagram for explaining the feature area extraction unit of the table data extraction system according to an embodiment of the present invention. Figures 5 and 6 are diagrams for explaining non-standard table data extraction of the table data extraction system according to an embodiment of the present invention, and Figure 7 is a diagram illustrating data verification of the table data extraction system according to an embodiment of the present invention. This is a drawing to explain wealth.

<Example 1>

Referring to FIG. 1, the table data extraction system 100 according to an embodiment of the present invention includes an image input unit 110, a table type determination unit 120, a cell area designation unit 130, and a data extraction unit 140. At least one image document data can be input to the image input unit 110.

At this time, the image document data may be input in a file format such as jpg, jpeg, png, or pdf in which the document is scanned, and may be converted into a preset format through an image format conversion unit included in the image input unit 110.

The table type determination unit 120 may determine whether at least one table data is included in the input image document data and determine the type of the table data.

More specifically, the table type determination unit 120 determines the table type included in the image document data by comparing the image document data input to the image input unit 110 with the training image data stored in a predetermined learning DB. It can be judged as either a standard type or a non-standard type.

The learning DB includes first training image data classified as the standard type and the cell area being divided by continuous lines, and second learning image data classified as the non-standard type and having the cell area not divided by continuous lines. It may include data and third training image data that is classified into the non-standard type and includes both an area divided by a continuous line in the cell area and an area that is not divided by a line.

Therefore, the table type determination unit 120 scans the entire image document data input to the image input unit 110 and determines whether table data matching the first to third learning image data stored in the learning DB is included. And the table type of the table data can be determined.

Meanwhile, the cell area designator 130 extracts a table data area included in the image document data based on the type of table data determined by the table type determination unit 120, and the extracted table data area A line area can be extracted by dividing into pixel units, and a cell area can be designated based on the extracted line.

More specifically, as shown in FIG. 2, the cell area designation unit 130 includes a table area extractor 131 that extracts the table data area included in the image document data, and the table type determination unit 120. ), if the image document data is determined to be the standard type, the extracted table data area is divided into pixels, the colors of the pixels are determined to be black and white, and the pixels determined to be black are preset. A line area extractor 132 that recognizes the line area by determining whether it is abnormal, and when the table type determination unit 120 determines that the image document data is the non-standard type, at least one feature in the extracted table data area. Calculate the number of rows and columns based on the line region recognized by the feature region extractor 133 and the line extractor 132 for extracting the region and the feature region extracted by the feature region extractor 133, and calculate the number of rows and columns in the cell region. It may include a cell area generator 134 that generates.

For example, referring to FIG. 3, when the image document data included in the image input unit 110 includes a standard type table, the table area extractor 131 extracts the table data from the image document data. The included area can be extracted.

The line area extractor 132 may extract at least one line from the table data area extracted by the table area extractor 131. At this time, the line area extractor 132 may divide the table data area into pixel units and divide the pixels into white and black.

For example, each pixel divided by the line area extractor 132 is assigned a unique coordinate (x1, y1), and when a white pixel is set to 0 and a black pixel is set to 1, the x-axis or y-axis A preset number (ex. 10) or more consecutive black pixels can be searched, and the extracted area can be recognized as a line.

Therefore, as shown in FIG. 3, the cell area generator 134 generates at least one cell area 310 in the table data area based on the line recognized by the line area extractor 132. can do.

Accordingly, the data extraction unit 140 extracts text data included in the cell area designated by the cell area designation unit 130, and performs OCR on at least one cell area generated by the cell area creation unit 134. By applying an algorithm, text data corresponding to the cell area can be extracted.

At this time, if the table type determined by the table type determination unit 120 is a standard type, the work process of the feature area extraction unit 133 of the cell area designation unit 130 is omitted, and the cell area creation unit 134 may generate the cell area based on the line area extracted by the line area extractor 132.

On the other hand, if the table type determined by the table type determination unit 120 is a non-standard type, the work process of the line area extraction unit 132 is omitted, and the cell area creation unit 134 is operated by the feature area extraction unit. The cell region can be created based on the feature region extracted in (130).

More specifically, as shown in FIG. 4, the feature area extractor 133 compares the image document data with the second training image data in the table type determination unit 120 to determine the non-standard type. If determined, the extracted table data area is divided into pixel units, a feature area is extracted by detecting pixel changes in the table data area, the number of rows and columns of the feature area is calculated, and the number of rows and columns of the feature area and the rows and columns are calculated. When the image document data is determined to be the non-standard type by comparison with the third training image data in the no-line feature extraction unit 1331 and the table type determination unit 120 for merging the feature areas based on the image Extracting at least one line area and feature area included in document data, extracting at least one intersection of the line areas, grouping the feature areas based on the intersection of the line areas to create a group area, and generating the group area. It may include a broken line feature extractor 1332 that determines the table structure by calculating the number of rows and columns in the feature area.

More specifically, the no-line feature extraction unit 1331 extracts a feature area from a table type that does not include a line area in the table data, and analyzes the table data area extracted from the table area extraction unit 132 to determine the pixel At least one feature region can be extracted by searching an area where values change, and the maximum number of rows and columns of the extracted feature region can be calculated.

For example, as shown in Figure 5, the maximum number of columns of the no-line table data is 4, the maximum number of rows can be calculated as 10, and the initial structure of the table data is a structure of 4 columns and 10 rows. can be created.

At this time, in the case of a column where the maximum number of rows is not 10, it is determined that a merged cell exists, and the no-line feature extractor 1331 can merge the cell areas.

For example, in the initial structure created by the feature area, row 1 is recognized as an item name and is automatically excluded, and a cell merging process can be performed on the cell areas included in rows 2 to 9 excluding row 1. there is.

Referring to Figure 5, it can be seen that the no-line feature extraction unit 1331 extracted a total of 5 feature regions, excluding the first column Name. Since the maximum number of calculated rows is 10, a cell merging process is performed for columns 1 and 3, but for the cell 510 located below the item name, merging is performed in the up and down directions based on the feature area. For other cells 520, 530, 540, and 550, merging can be performed only in the downward direction.

Accordingly, after the cell merging operation is completed, the final structure of the table data is determined, and the cell area creation unit 143 may generate at least one cell area based on the final structure of the table data.

Meanwhile, as shown in FIG. 6, the broken line feature extractor 1332 can extract at least one feature area and a line area from a table type that includes both a line area and a no-line area in the table data. .

The process of extracting the line area and the process of extracting the feature area are performed separately, but the process of first extracting the line area is the same as that of the line area extractor 132, and the table area extractor 132 ), the extracted table data area can be divided into pixels, and at least one line area can be extracted by distinguishing the colors of the pixels.

Additionally, at least one feature area can be extracted by analyzing the table data area extracted by the table area extractor 132 and searching for an area in which pixel values change.

For example, the broken line feature extraction unit 1332 may search for intersections of the extracted lines and determine the initial structure of the table data based on the line area and intersection.

Thereafter, the feature areas are grouped based on the initial structure to create group areas 610 and 620, the number of rows and columns of the group area are calculated, and the number of rows and columns of the group area is calculated, and the number of rows and columns of the group area is calculated. The final structure of the table data can be determined by calculating the number of rows and columns.

Accordingly, the cell area generator 143 may generate at least one cell area based on the final structure of the table data.

Meanwhile, the table data extraction system 100 may further include a data verification unit 150 that verifies the accuracy of the data extracted from the data extraction unit 140.

More specifically, as shown in FIG. 7, the data verification unit 150 includes at least one of special characters, consonants, and vowels included in the text data extracted by the data extraction unit 140. A first extraction unit 151 for extracting a single character, a second extraction unit 152 for extracting at least one word data corresponding to the text data from a predetermined learning DB, and the first extraction unit 151 and the It may include an error verification unit 153 that determines whether the text data contains an error based on the data extracted by the second extraction unit 152.

For example, when at least one special character or at least one short vowel is extracted in the first extraction unit 151, the error verification unit 153 detects an error in the text data extracted by the data extraction unit 140. It can be judged that there is.

Additionally, the text extraction process from the image document data may be re-performed, and if the same error occurs as a result of the re-performance, a notification signal requesting modification of the image document data may be transmitted to a predetermined administrator terminal.

Meanwhile, the learning DB generates vector data by applying an input word storage unit in which word data input from the manager terminal is stored and the word data stored in the input word storage unit to a predetermined learning algorithm, and the vector data and a vector data generator that maps the word data.

More specifically, synonyms and related words of word data are learned by the learning algorithm, and the vector data may include information on the synonyms and related words of the mapped word data.

For example, if a special character or short vowel is not extracted in the first extraction unit 151, the second extraction unit 152 extracts word data that matches the text data among the word data stored in the input word storage unit. You can determine whether there is a , and extract matching word data.

In addition, if there is no data among the word data that matches the text data, the second extraction unit 152 may extract synonyms and related words of the text data based on the vector data generated by the vector data generation unit. there is.

At this time, if there is no word data extracted through the second extraction unit 152, the error verification unit 153 may determine that there is an error in the text data extracted by the data extraction unit 140.

Meanwhile, when verification of the text data extracted through the data verification unit 150 is completed, the text data can be stored in a predetermined server, and the text data that has been verified as free from errors generates the vector data. It can be used as learning data for the learning algorithm.

<Example 2>

The table data extraction method of the table data extraction system 100 according to an embodiment of the present invention includes a first step of inputting at least one image document data into the image input unit 110, and the table type determination unit 120. ) determines whether at least one table data is included in the input image document data, and if the table data is included, a second step of determining the type of the table data, the cell area designator 130 A third step of extracting the table area of the image document data and extracting at least one line area and feature area included in the table data based on the table type, the cell area designation unit 130 A fourth step of generating at least one cell area based on the area and the feature area, a fifth step of the data extraction unit 140 extracting text information included in the cell area, and verifying the extracted text information. , The verified text information may include a sixth step in which the verified text information is stored in a predetermined server.

According to the effect of the present invention as described above, table data that can easily convert information existing in paper form into digital data by extracting a table from input image data, determining the type of table, and extracting table data An extraction system and method may be provided.

In addition, by dividing the image data into pixel units and extracting the line area included in the table area to specify the cell area, and extracting the feature area of the table where no lines exist and specifying the cell area, the table data is stored regardless of the type of table. A table data extraction system and method that can accurately extract can be provided.

Additionally, the control method of the table data extraction system according to an embodiment of the present invention may be recorded on a computer-readable medium including program instructions for performing various computer-implemented operations. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. The medium may have program instructions specifically designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

As described above, although one embodiment of the present invention has been described with limited examples and drawings, one embodiment of the present invention is not limited to the above-described embodiment, which is based on common knowledge in the field to which the present invention pertains. Anyone who has the knowledge can make various modifications and variations from this description. Accordingly, one embodiment of the present invention should be understood only by the scope of the claims set forth below, and all equivalent or equivalent modifications thereof shall fall within the scope of the spirit of the present invention.

110: image input unit
120: Table type judgment unit
130: Cell area designation unit 131: Table area extraction unit
132: Line area extraction unit
133: Feature region extraction unit
134: Cell area creation unit
140: data extraction unit
150: data verification unit

Claims (5)

an image input unit where at least one image document data is input;
a table type determination unit that determines whether the input image document data includes at least one table data and determines a type of the table data;
Based on the type of table data determined by the table type determination unit, a table data area included in the image document data is extracted, the extracted table data area is divided into pixels to extract a line area, and the extraction a cell area designation unit that designates a cell area based on the drawn line; and
A data extraction unit that extracts text data corresponding to the cell area by applying a preset OCR algorithm to the cell area designated by the cell area designation unit,

It further includes a data verification unit that verifies the accuracy of the text data extracted from the data extraction unit,

The table type determination unit,
By comparing the image document data input to the image input unit with learning image data stored in a predetermined learning DB, the table type included in the image document data is determined as one of a standard type and a non-standard type,
The learning DB is,
first training image data classified into the standard type, and the cell areas are divided by continuous lines;
second training image data classified as the non-standard type, and in which the cell area is not divided by continuous lines; and
Third training image data classified as the non-standard type and including both an area divided by a continuous line in the cell area and an area not divided by a line;

The cell area designator,
a table area extractor that extracts a table data area included in the image document data;
When the table type determination unit determines that the image document data is the standard type, the extracted table data area is divided into pixels, the color of the pixel is determined to be black and white, and the pixel determined to be black is determined. a line area extraction unit that recognizes the line area by determining whether the number is greater than or equal to a preset number;
a feature area extraction unit that extracts at least one feature area from the extracted table data area when the table type determination unit determines that the image document data is of the non-standard type; and
It includes a cell area generator that calculates the number of rows and columns and generates a cell area based on the line area recognized by the line extractor and the feature area extracted by the feature area extractor,

The cell area generator,
If the table type determined by the table type determination unit is the standard type, the process of extracting the feature area through the feature area extractor is omitted, and the table type is based on the line area extracted by the line area extractor. Create a cell area,
If the table type determined by the table type determination unit is the non-standard type, the process of recognizing the line area through the line area extractor is omitted, and the process of recognizing the line area is performed based on the feature area extracted by the feature area extractor. Create a cell area,

The feature region extraction unit,
When the table type determination unit determines that the image document data is a non-standard type by comparing it with the second learning image data, the extracted table data area is divided into pixels and a pixel change in the table data area is detected. a no-line feature extraction unit that extracts a feature region, calculates the number of rows and columns of the feature region, and merges the feature region based on the number of rows and columns of the feature region; and
When the table type determination unit determines that the image document data is a non-standard type by comparing it with the third learning image data, extract at least one line area and a feature area included in the image document data, and extract the line area and the feature area of the line area. Broken extracts at least one intersection, creates a group area by grouping the feature areas based on the intersection of the line areas, and determines the table structure by calculating the number of rows and columns in the group area and the number of rows and columns in the feature area. Includes a line feature extraction unit,

The no-line feature extraction unit,
Determine an initial structure of the table data based on the maximum number of rows and the maximum number of columns among the number of rows and columns in the feature area,
If there is a row and column that is less than the maximum number of rows and the maximum number of columns, it is determined that at least one integrated cell exists in the row and column, and cell merging of the cell area is performed,
In the initial structure of the table data, row 1 is recognized as an item name, and cell merging of cell areas of rows other than row 1 is performed,
If the remaining row is located below and adjacent to the first row, the cell area of the remaining row is merged in the upper and lower directions, and if the remaining row is not adjacent to the first row and is located below, the remaining row The cell areas of the rows are subjected to downward merging to determine the final structure of the table data,

The broken line feature extraction unit,
Determine an initial structure of the table data based on the line area and the intersection of the line area,
Determine the final structure of the table data by calculating the number of rows and columns of the group area and the number of rows and columns of at least one feature area included in the group area,

The data verification unit,
a first extraction unit that extracts special characters included in the text data extracted by the data extraction unit and single characters including at least one of a consonant and a vowel;
a second extraction unit that extracts at least one word data corresponding to the text data from a predetermined learning DB; and
It includes an error verification unit that determines whether the text data contains an error based on the data extracted from the first extraction unit and the second extraction unit,

The learning DB is,
an input word storage unit that stores word data input from a predetermined manager terminal; and
A vector data generator that generates vector data by applying the word data stored in the input word storage unit to a predetermined learning algorithm and maps the vector data to the word data,

The vector data generator,
A table characterized in that, based on the learning algorithm, the synonyms and related words of the word data input from the manager terminal are learned, and the vector data containing data corresponding to the synonyms and the related words of the word data are generated. Data extraction system.
delete delete delete delete