KR102629133B1 - Document recognition device using optical character recognition and document structuring tags for building ai learning dataset - Google Patents

Abstract

The present invention extracts text through OCR, identifies the format and structure of the document, attaches a document structuring tag, and builds an artificial intelligence learning dataset that allows the content of the document and the table to be identified as the original document according to the format of the document. It relates to a document recognition device using optical character recognition and document structuring tags, comprising: an OCR analysis unit that identifies format items according to the type of an input target document and extracts text corresponding to the format items; a document structuring unit that generates document structured data including at least one format item identified in the target document and relationship information about text of the format item; a table recognition unit that determines the structure of a table included in the target document, extracts text included in a cell area of the table, and generates table format data; and a document tagging unit that attaches a document structuring tag identifying the document structure to the document structured data and converts it into text data in a predetermined format.

Description

Document recognition device using optical character recognition and document structuring tags for building artificial intelligence learning dataset {DOCUMENT RECOGNITION DEVICE USING OPTICAL CHARACTER RECOGNITION AND DOCUMENT STRUCTURING TAGS FOR BUILDING AI LEARNING DATASET}

The present invention receives an image file of a scanned document, extracts text through optical character recognition (OCR), determines the format and structure of the document, and attaches a document structuring tag to the content of the document and the contents of the table according to the format of the document. This relates to a document recognition device that utilizes optical character recognition and document structuring tags to build an artificial intelligence learning dataset that allows the original document to be recognized and edited by reproducing the format and tables of the document.

As optical character recognition (OCR) technology develops, companies are automating image and document processing tasks to maximize process efficiency.

In order to internalize the collected images, it has become possible to automatically extract text by applying OCR technology instead of having people manually input the text contents into the system. Recently, artificial intelligence-based optical character recognition (OCR) technology has been developed. In this way, more sophisticated text extraction became possible through context recognition from recognized letters and words.

OCR technology extracts text from the image by identifying letters from a scanned image, recognizing and extracting the identified letters as text, and arranging the text according to the image coordinates.

In general, OCR technology is implemented by reading the entire document and recognizing characters. However, in some cases, unnecessary parts that are repeated within the document, such as headers, footers, and page numbers, are recognized, which causes the inconvenience of having to manually remove or classify them. This still exists.

Additionally, since document formatting is excluded when extracting text, documents containing formatting or tables are often not understood as intended in the original document.

Therefore, when performing OCR, in addition to text extraction, there is a need for technology to recognize documents so that the format of the document can be reflected and the structure of the document can be identified.

In order to solve the above problem, the present invention identifies the structure of the document according to the type of document when performing OCR, extracts text according to the format of the document, and attaches a structure tag to the document to recognize the document as intended and turn it into data. The purpose is to provide a device that does this.

A document recognition device using optical character recognition and document structuring tags for building an artificial intelligence learning dataset according to an embodiment of the present invention identifies format items according to the type of the input target document and corresponds to the format items. OCR analysis unit that extracts text; a document structuring unit that generates document structured data including at least one format item identified in the target document and relationship information about text of the format item; a table recognition unit that determines the structure of a table included in the target document, extracts text included in a cell area of the table, and generates table format data; and a document tagging unit that attaches a document structuring tag identifying the document structure to the document structured data and converts it into text data in a predetermined format.

In addition, the target document includes at least one of an image and a PDF file containing text, and the OCR analysis unit applies optical character recognition (OCR) technology to recognize text in a document in the form of a text image. It is characterized by extraction.

In addition, the OCR analysis unit determines the document type through the meta information of the target document, checks the template data of the document type, and determines the format in the target document according to the location area and pattern rule of the format item defined in the template data. It is characterized by identifying the location area of the item.

In addition, the OCR analysis unit is characterized in that for each format item identified in the target document, the text included in the position area of the format item is extracted to generate format item-text content pair data.

In addition, the table recognition unit identifies each cell area of the table from the image data of the area where the table is located in the target document, determines the matrix structure of the entire table, extracts text in the identified cell area, and identifies the cell. It is characterized by attaching a table tag that can be used to connect to document structured data.

In addition, the table tag is characterized in that the relative position information of cells within the table is formatted and displayed in a matrix structure.

In addition, the table tag includes a cell tag, and the cell tag is characterized in that it includes and displays information that a specific cell has been added or a plurality of cells have been merged.

In addition, the document tagging unit is characterized in that it writes and stores the document structured data in markup language using a predefined document structuring tag.

When extracting text from PDF documents or scanned images of documents, the format of the document is identified and tags are attached according to the format, allowing editing and processing by selecting only the text in the required format.

Additionally, when using text extracted from PDF documents and scanned images of documents as learning data, unnecessary text such as headers, footers, and page numbers that appear repeatedly can be easily excluded.

In addition, the structure of the table and the contents of each cell can be connected and understood in the document, and the table can be reproduced through tagged document data.

Figure 1 is an overall relationship diagram of a document recognition device using optical character recognition and document structuring tags for building an artificial intelligence learning dataset according to an embodiment of the present invention.
Figure 2 is a block diagram of the function of a document recognition device using optical character recognition and document structuring tags for building an artificial intelligence learning dataset according to an embodiment of the present invention.
Figure 3 shows a document recognition device using optical character recognition and document structuring tags for building an artificial intelligence learning dataset according to an embodiment of the present invention, where tables included in the document are displayed in a matrix structure and table tags are attached. This is an example drawing of one table format data.
Figure 4 shows a document recognition device using optical character recognition and document structuring tags for building an artificial intelligence learning dataset according to an embodiment of the present invention, recognizes document structure and text in an image file containing text, and documents This is a diagram showing data converted into text data in a defined format by attaching a structure tag.
Figure 5 is a diagram showing the hardware structure of a document recognition device using optical character recognition and document structuring tags for building an artificial intelligence learning dataset according to an embodiment of the present invention.

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings. However, the spirit of the present invention is not limited to the presented embodiments, and those skilled in the art who understand the spirit of the present invention may add, change, or delete other components within the scope of the same spirit, or create other degenerative inventions or this invention. Other embodiments that are included within the scope of the invention can be easily proposed, but this will also be said to be included within the scope of the invention of the present application.

In addition, the terms described below are terms set in consideration of the function in the present invention, and may vary depending on the inventor's intention or custom, so the definition should be made based on the content throughout the specification, and in this specification, the terms related to the present invention In cases where it is determined that detailed descriptions of well-known configurations or functions may obscure the gist of the present invention, detailed descriptions thereof will be omitted.

Hereinafter, a document recognition device 100 using optical character recognition and document structuring tags for constructing an artificial intelligence learning dataset according to the present invention will be described with reference to the drawings.

Figure 1 is an overall relationship diagram of a document recognition device (hereinafter referred to as a document recognition device) using optical character recognition and document structuring tags for building an artificial intelligence learning dataset according to an embodiment of the present invention.

Referring to FIG. 1, the document recognition device 100 is connected to at least one user terminal 200 and at least one application server 300 over a network and can communicate with each other.

The network referred to in the present invention may be a core network integrated with a wired public network, wireless mobile communication network, or mobile Internet, etc., and may include the TCP/IP protocol and various services existing in its upper layer, such as HTTP (Hyper Text Transfer Protocol), It can refer to a global open computer network structure that provides HTTPS (Hyper Text Transfer Protocol Secure), Telnet, and FTP (File Transfer Protocol), etc., and is not limited to these examples, but is a data communication network that can transmit and receive data in various forms. It means comprehensively.

The document recognition device 100 of the present invention extracts text from the input document and extracts the text from the input document because the format is excluded when text is extracted from a PDF document or an image document, making it impossible to understand the original document as intended. , determine the structure of the document such as footer, page number, and body and attach a document structure tag to the extracted text. If the body includes a table, link information between the structure of the table and the contents of each cell is formatted into a matrix structure to tag the table. is attached to convert the contents of the table into data so that it can be understood as in the original document.

For this purpose, the user terminal 200 receives a PDF document or a scanned document (image) from at least one of the external server 300, identifies form items according to the type of the received target document, and enters the form items into the form items. The corresponding text is extracted, and document structured data is generated including at least one format item identified in the target document and relationship information about the text of the format item.

In addition, the structure of the table included in the target document is identified, the text included in the cell area of the table is extracted, table format data is generated, and linked to document structured data.

When document structuring is completed, a document structuring tag that identifies the document structure is attached to the document structured data and converted into text data in a specified format.

In the present invention, the user terminal 200 or the external server 300 can register the target document through the user interface or linkage interface provided by the document recognition device 100.

Here, the target document is at least one of an image containing text and a PDF file, such as an image file scanned from a paper document, a file saved as a PDF, and a file converted to PDF from an electronic document such as Word, PPT, or Korean document. may include.

The external server 300 may be a server that produces documents or processes collected documents, or may be a system that stores and stores documents.

Figure 2 is a block diagram of the function of the document recognition device 100 using optical character recognition and document structuring tags for building an artificial intelligence learning dataset according to an embodiment of the present invention.

Referring to FIG. 2, the document recognition device 100 may include an OCR analysis unit 110, a document structuring unit 120, a mark recognition unit 130, and a document tagging unit 140.

The OCR analysis unit 110 identifies format items according to the type of the input target document and extracts text corresponding to the format items.

Target documents input to the OCR analysis unit 110 may be images containing text, PDF documents, papers, reports, press releases, administrative documents, etc. containing formats and tables.

The OCR analysis unit 110 applies optical character recognition (OCR) technology to recognize and extract text from the input target document in the form of a text image.

For example, optical character recognition can proceed in the following steps.

First, if the target document is damaged by noise or the image is tilted or rotated, preprocessing can be performed to restore the image to a form suitable for analysis.

Afterwards, the task of detecting text from the preprocessed image is carried out. Since documents contain not only text but also various objects such as pictures, graphs, and lines, text recognition is performed to recognize the text and identify the characters in the detected area.

The process of recognizing text can be done by applying deep learning-based OCR models such as CNN (Convolutional neural network), RNN (Recurrent neural network), and CRNN method that combines CNN and RNN.

The OCR analysis unit 110 includes a format identification unit 111 and a text extraction unit 112.

The format identification unit 111 identifies the document type through meta information of the target document, checks template data of the document type, and extracts text according to the location area and pattern rules of the format item defined in the template data.

For example, a general document may consist of a title, body, header, footer, page, etc.

Additionally, as an example, if the input target document is a paper, it may consist of title, abstract, introduction, research method, results, discussion, acknowledgment, and references.

The format identification unit 111 registers and manages a template defining the document format used in each domain, and upon receiving a document subject to OCR analysis, checks the meta information of the document received together to determine the type of the document.

In addition, the format items defined in the template data are checked according to the type of document, and the extraction rules for format identification are checked.

Format identification rules included in template data may include location area (coordinates) within the document image, character style (size, font type), inclusion of special characters or symbols, etc.

When the format area of the corresponding form is confirmed in the format identification unit 111, the text in the corresponding area is recognized and extracted through the text extraction unit 112.

The text extraction unit 112 may extract text for a specific area from the image.

The extracted text is mapped and stored as format item-text content pair data, and is structured and stored according to the relationship between document formats through the document structuring unit 120.

The document structuring unit 120 generates document structured data including relationship information about at least one form item identified through the OCR analysis unit 110 and the extracted text.

For example, document structured data may be in the form of pair data consisting of at least one attribute name and attribute value.

Additionally, document structured data may include information indicating relationships between a plurality of pair data.

Meanwhile, the format identification unit 111 can recognize the table included in the target document and extract the location area of the table.

The location area of the extracted table is analyzed by the table recognition unit 130 to structure the table and extract text within the cell.

The table recognition unit 130 determines the structure of the table included in the target document, extracts text included in the cell area of the table, and generates table format data.

The table recognition unit 130 includes a table structure identification unit 131 and a cell text extraction unit 132.

The table structure identification unit 131 analyzes the location area of the table extracted by the format identification unit 111 to identify the entire table area, and identifies each cell area of the table from the image data of the area where the table is located.

At this time, the relative position information of cells within the table is formatted and displayed in a matrix structure.

Figure 3 shows a document recognition device 100 using optical character recognition and document structuring tags for constructing an artificial intelligence learning dataset according to an embodiment of the present invention, where a table included in the document is displayed in a matrix structure, and the table This is an example drawing of table format data with tags attached.

Referring to FIG. 3, the process of recognizing tables included in a document and formulating them into a matrix structure will be described.

Figure 3(a) is an example of a table included in a document.

The table structure identification unit 131 identifies the entire area of the table included in the document by identifying the outline of the table, and determines the area of the cell by recognizing cell lines within the table area.

At this time, the entire structure of the table can be formalized into a matrix structure like (b).

Additionally, the table structure identification unit 131 defines a table tag for each identified cell that can check the location of the cell and whether it is merged.

In particular, the part where at least one cell is merged defines the table tag including the range of the merged cells.

For example, the structure of table (a) in Figure 3 is determined to have 2 rows and 4 columns, and can be defined with the table tag '<table2*4>'.

In the first cell in table (a) of Figure 3, that is, the cell containing the text 'Separation', when defining the matrix structure, the (1,1) cell and the (1,2) cell are recognized as merged, The corresponding table tag can be defined as '<data:(1,1)(1,2)>'.

Figure 3 (c) is an example of recognizing table (a) and attaching a table tag.

For example, a table tag may be a markup language structure similar to HTML and XML that includes a start tag, an end tag, an element, and an attribute, as shown in (c) of FIG. 3.

However, the tag is not limited to this and may be a tag indicating a text format document format including various types of tags.

By formulating and defining the table structure as a matrix structure in this way, tagged table format data can be accurately reproduced as a table, and table editing such as merging cells and adding cells in the reproduced table becomes possible.

Additionally, when cell addition and cell merging are performed on the edited table, the cell tag can be displayed including information about a specific cell being added or a plurality of cells being merged.

The cell text extractor 132 extracts text within a cell area recognized and defined by a table tag.

At this time, if the text in the cell area consists of multiple lines, it would be desirable to recognize the cell's outline within the cell area and change the lines to read the text so that it can be read the same as the original text.

Meanwhile, the table recognition unit 130 connects table format data with a table tag attached to document structured data.

For example, if the structure of the title, header, footer, and body is recognized and stored in the document structured data, and a table is located within the body, text with a table tag attached as a substructure of the body may be included in the document structured data.

The document tagging unit 140 attaches a document structuring tag that identifies the document structure to the document structured data and converts it into text data in a predetermined format.

In other words, document structured data can be written and stored in markup language using predefined document structuring tags.

Here, the markup language includes start tags, end tags, elements, and attributes, such as html and xml, and may include the structure of the document.

Figure 4 shows a document recognition device 100 using optical character recognition and document structuring tags for building an artificial intelligence learning dataset according to an embodiment of the present invention, recognizing document structure and text in an image file containing text. This is a drawing showing data that has been converted into text data in a given format by attaching a document structure tag. The document structure tag for each format defined in advance is attached to the generated document structured data and displayed in a markup language. .

As shown in Figure 4, a document with a tag attached and converted into a markup language form can be flexibly stored in text form without being affected by the structure of the document, and can also be converted to structured data and used.

In addition, the structure of the document can be defined in advance as a rule (schema, schema) for each document type, and the tagged document text can be parsed according to the rule (schema) and converted into document structured data. You can selectively extract and edit/process only the required format text.

For example, headers, footers, and pages repeatedly included in a document can be identified and excluded when processing the document.

Also, as an example, when extracting keywords for a document, text recognized as the title can be used without analyzing the entire document content.

Figure 5 is a diagram showing the hardware structure of a document recognition device 100 using optical character recognition and document structuring tags for building an artificial intelligence learning dataset according to an embodiment of the present invention.

Referring to FIG. 5, the hardware structure of the document recognition device 100 includes a central processing unit 1000, a memory 2000, a user interface 3000, a database interface 4000, a network interface 5000, and a web server ( 6000), etc.

The user interface 3000 provides an input and output interface to the user by using a graphical user interface (GUI).

The database interface 4000 provides an interface between the database and the hardware structure.

The network interface 5000 provides network connections between devices owned by users.

The web server 6000 provides a means for users to access the hardware structure through a network. Most users can use the document recognition device 100 by remotely accessing a web server.

Each step of the above-described configuration or method may be implemented as computer-readable code on a computer-readable recording medium or transmitted through a transmission medium. A computer-readable recording medium is a data storage device capable of storing data that can be read by a computer system.

Examples of computer-readable recording media include, but are not limited to, databases, ROM, RAM, CD-ROM, DVD, magnetic tape, floppy disk, and optical data storage devices. Transmission media may include carrier waves transmitted over the Internet or various types of communication channels. The computer-readable recording medium may also be distributed through a network-coupled computer system such that the computer-readable code is stored and executed in a distributed manner.

In addition, at least one or more components applied to the present invention may include or be implemented by a processor such as a central processing unit (CPU) or microprocessor that performs each function, and two or more of the components may be implemented as a single It can be combined into components and perform all operations or functions of two or more components combined. Additionally, part of at least one or more components applied to the present invention may be performed by other components among these components. Additionally, communication between the components may be performed through a bus (not shown).

100: Document recognition device
110: OCR analysis department
111: Format identification unit 112: Text extraction unit
120: Document structuring unit
130: Mark recognition unit
131: Table structure identification unit 132: Cell text extraction unit
140: Document tagging unit
200: user terminal
300: external server

Claims (8)

an OCR analysis unit that identifies format items according to the type of the input target document and extracts text corresponding to the format items;
a document structuring unit that generates document structured data including at least one format item identified in the target document and relationship information about text of the format item;
a table recognition unit that determines the structure of a table included in the target document, extracts text included in a cell area of the table, generates table format data, and connects the generated table format data to document structured data; and
It includes a document tagging unit that attaches a document structuring tag identifying the document structure to the document structured data and converts it into text data in a predetermined format,
The target document includes at least one of an image and a PDF file containing text,
The OCR analysis unit,
The document type is identified through the meta information of the target document, and optical character recognition (OCR) technology is applied to identify the format item located in the format item area according to the format item identification rules defined in the template data of the target document type. Characterized by recognizing and extracting text,
The identification rules of format items defined in the template data include location coordinates and character styles on the document image,
The document tagging unit,
Based on a schema that defines the structure of each document for each target document type, an XML-based document text is created by attaching a document structuring tag to the document structured data,
The XML-based document text can be parsed according to the schema and converted into document structured data, and text of specific format items can be extracted.
A document recognition device utilizing optical character recognition and document structuring tags.
delete delete According to paragraph 1,
The OCR analysis unit,
Characterized in that, for each format item identified in the target document, text included in the location area of the format item is extracted to generate format item-text content pair data,
A document recognition device utilizing optical character recognition and document structuring tags.
According to paragraph 1,
The mark recognition unit,
Identify each cell area of the table from the image data of the area where the table is located in the target document, identify the matrix structure of the entire table, extract the text of the identified cell area, and attach a table tag to identify the cell. Characterized by linking to document structured data,
A document recognition device utilizing optical character recognition and document structuring tags.
According to clause 5,
The table tag above is,
Characterized in that the relative position information of cells within the table is formatted and displayed in a matrix structure.
A document recognition device utilizing optical character recognition and document structuring tags.
According to clause 6,
The table tag above is,
Includes a cell tag, wherein the cell tag displays information including information on a specific cell being added or a plurality of cells being merged.
A document recognition device utilizing optical character recognition and document structuring tags.
delete