KR102649429B1 - Method and system for extracting information from semi-structured documents - Google Patents

Abstract

The present invention relates to a method and system for extracting information from documents. The present invention includes the steps of performing OCR (Optical character recognition) on a target document, extracting a plurality of words and location information for each of the plurality of words, and generating a plurality of tokens corresponding to each of the plurality of words. and arranging the plurality of tokens so that the data structure between the plurality of words included in the target document is reflected, and in the arranging of the plurality of tokens, the meaning of the plurality of words and the plurality of words are included. A method of extracting information from a semi-structured document can be provided, characterized by arranging the plurality of tokens using location information for each word.

Description

Method and system for extracting information from semi-structured documents {METHOD AND SYSTEM FOR EXTRACTING INFORMATION FROM SEMI-STRUCTURED DOCUMENTS}

The present invention relates to a method and system for extracting information from documents.

The dictionary meaning of artificial intelligence is a technology that realizes human learning ability, reasoning ability, perception ability, and natural language understanding ability through computer programs. This artificial intelligence has made rapid progress due to deep learning, which adds a neural network that mimics the human brain to machine learning.

Deep learning is a technology that enables computers to judge and learn like humans, and to cluster or classify objects or data. Recently, it has become possible to analyze not only text data but also video data, enabling a wide variety of applications. It is actively used in industrial fields.

With the development of artificial intelligence, various automations are being implemented in the office automation field. In particular, in the field of office automation, much effort is being made to convert content printed on paper into data based on video data analysis technology using artificial intelligence. As part of this, in the office automation field, paper documents are converted into images and the content contained in the document is converted into data through image analysis technology (or video data analysis technology) that analyzes the content contained in the image. In this case, the document Technology is needed to analyze images according to the type of content included.

For example, when converting a document containing a receipt into data, an accurate analysis of various elements surrounding the receipt, such as the format of the receipt, the content of the text included in the receipt, and the location of the text included in the receipt, is required. need.

Accordingly, various technologies are being developed to process information contained in images into data that can be processed by electronic equipment. For example, Republic of Korea Patent No. 10-1181209 discloses a method of building an OCR database.

However, since the methods developed to date are at the level of classifying data according to rules established empirically by humans, there is a continued need for the development of applications that can accurately reproduce the structure of the data contained in the target document and the relationships between text data. do.

The present invention relates to a method and system for extracting information from a target document by reflecting the data structure between a plurality of words included in the target document.

Furthermore, the present invention provides a method and system for extracting data from a target document in a form that facilitates training of a machine learning model that extracts data from the target document.

Specifically, the present invention implements the format of the learning data used to determine the accuracy of a machine learning model that extracts data from a target document and the data output from the machine learning model in a form that is easy for humans to process, thereby creating a machine learning model. Provides a data extraction method and system that reduces the time and cost required to determine the accuracy of.

Furthermore, the present invention is intended to provide a data extraction method and system that can provide relationships between data included in target documents.

In order to solve the problems described above, the present invention performs OCR (Optical character recognition) on a target document, extracting a plurality of words and location information for each of the plurality of words, corresponding to each of the plurality of words. generating a plurality of tokens and arranging the plurality of tokens so that a data structure between the plurality of words included in the target document is reflected, and in the step of arranging the plurality of tokens, the plurality of tokens are A method of extracting information from a semi-structured document can be provided, characterized by arranging the plurality of tokens using the meaning of the word and location information for each of the plurality of words.

In addition, the present invention includes an OCR unit that performs OCR (Optical character recognition) on a target document to extract a plurality of words and location information for each of the plurality of words, and a plurality of tokens corresponding to each of the plurality of words. and a control unit that generates and arranges the plurality of tokens so that a data structure between the plurality of words included in the target document is reflected, wherein the control unit determines the meaning of the plurality of words and each of the plurality of words. It is possible to provide a system for extracting information from a semi-structured document, characterized by arranging the plurality of tokens using location information.

In addition, the present invention is a program that is executed by one or more processes in an electronic device and can be stored in a computer-readable recording medium, wherein the program performs OCR (Optical character recognition) on the target document, Extracting a word and location information for each of the plurality of words, generating a plurality of tokens corresponding to each of the plurality of words, and reflecting a data structure between the plurality of words included in the target document, Includes instructions for performing the step of arranging a plurality of tokens, and in the step of arranging the plurality of tokens, using the meaning of the plurality of words and location information for each of the plurality of words, the plurality of tokens It is possible to provide a program that can be stored in a computer-readable recording medium, characterized by arranging .

As seen above, the present invention uses the meaning of the plurality of words extracted from the target document and the location information for each of the plurality of words, and includes the generated token in the target document by using the generated token and the arrangement of the generated token. Data that reflects the data structure between multiple words can be generated.

Additionally, machine learning of the information extraction model according to the present invention is performed by calculating the accuracy of a plurality of tokens output from the information extraction model. Since the output tokens are not data that completely reflects the relationship between words, it becomes easy to generate learning data to calculate the accuracy of the information extraction model.

Specifically, when generating learning data, tokens arranged in a certain order are generated according to predetermined rules (token generation rules), and then the accuracy of the information extraction model is determined by comparing them with the tokens output from the information extraction model. And, training on the information extraction model can be carried out in the direction of improving the accuracy of the information extraction model. Accordingly, the time required to generate learning data for machine learning of the information extraction model can be greatly reduced.

1 is a conceptual diagram illustrating an information extraction system according to the present invention.
Figure 2 is a flowchart for explaining the information extraction method according to the present invention.
Figure 3 is a conceptual diagram for explaining the information extraction model in the present invention.
Figure 4 is a conceptual diagram showing the data output from the information extraction model according to the present invention converted into a tree.
Figure 5 is a conceptual diagram showing a machine learning method for an information extraction model according to the present invention.
Figure 6 is a conceptual diagram showing calculating the accuracy of data output from the information extraction model according to the present invention.
Figure 7 is a conceptual diagram showing a conventional machine learning method for information extraction.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings. However, identical or similar components will be assigned the same reference numbers regardless of drawing symbols, and duplicate descriptions thereof will be omitted. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves. Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

As seen above, office automation is being implemented in various forms due to the development of artificial intelligence, and for work efficiency, certain types of information contained in paper documents are digitized in the format contained in the paper documents ( The need for data conversion is gradually increasing.

For example, a document such as a receipt shown in Figure 1 includes multiple categories related to sellers and consumers, such as store name, store address, ordered product name, ordered product quantity, and order amount. In order to process data efficiently, it is necessary to computerize the data contained in paper documents by linking them to the same categories.

As used herein, “category” means an item at any level within a defined classification. The standards for data belonging to a specific category and data belonging to another category are not absolute and may vary depending on arbitrary rules defining the categories. These rules may apply differently to paper documents and data digitized from paper documents.

For example, a particular receipt consisting of a paper document may include five categories: “Store Name,” “Store Address,” “Order Product Name,” “Order Quantity,” and “Order Product Amount.” When digitizing the data contained in documents, it can be reduced to two categories: “Store” and “Ordered Products.”

Meanwhile, in this specification, words representing attributes of a specific category are classified as first type words. For example, type 1 words may include words that define higher-level concepts of data belonging to the category (“store name: “, “quantity: “, “store address”), etc.

Meanwhile, in this specification, words representing values of a specific category are classified as type 2 words. For example, type 2 words may be categorical data (“Happy store”, “105 h-street”, “Carrot”). In this specification, a first type word classified as the first type may be referred to as a “field name”, and a second type word classified as the second type may be referred to as a “field value.”

Meanwhile, from the perspective of data in a format that can be processed by a machine, the first type word may be referred to as an “attribute (key)” and the second type word may be referred to as a “value.”

According to the above definition of terms, paper documents such as receipts may contain data of “attribute-value” pairs contained in the same category. However, the document is not limited to this, and paper documents such as paper receipts may include only field values without field names for specific categories. In this case, the field name is simply omitted from the paper document, and the field value corresponding to the omitted field name contains the meaning related to the field name.

Meanwhile, paper documents can only contain field names without field values. In this case, the value assigned to the specific item does not exist, and even if the value assigned to the specific item does not exist, the specific item may exist in the paper document.

Hereinafter, the present invention will be described in detail based on the above-mentioned definitions of terms.

The above-described paper document maintains the relationship between data of the same category included in the document and can be converted into machine-understandable data.

In one embodiment, words included in a paper document can be extracted using an optical character reader (OCR). In the case of documents containing data of multiple categories, such as receipts, unrelated digitized words can be classified into the same category through a parsing process for the extracted words. Accordingly, data including a plurality of attribute-value pairs can be formed.

Research is continuing to use machine learning to ensure that unrelated digitized words have a certain data structure through OCR. Referring to FIG. 7, the conventional information extraction model extracts a plurality of words and location information from the target document 10 through OCR 711 for the target document 10, and based on this, creates a serialization model (Serializer, 712). ), some of the extracted words were serialized, the categories of the serialized text were classified through the tagging model (713), and parsing was performed through the Tag2parse model (714).

Afterwards, data in JSON format is generated through a normalization model (parse normalizer, 715).

Training of the conventional information extraction model was conducted in the direction of minimizing the error by calculating the error between the data output from the Tag2parse model 714 and the original data. At this time, in order to calculate the error, original data (hereinafter referred to as learning data) must be generated. The original data displays annotations for each of the words included in the target document 10 (721), serializes the text fragments included in the target document 10 (722), and uses a tagging model (723) to connect the words. Generated by performing parsing. Since the above-described processes are all performed through human work, generating training data for machine learning takes a very long time.

The present invention provides an information extraction system and method that can minimize the cost of machine learning of an information extraction model by shortening the time required to generate the above-described learning data.

The present invention provides a system for extracting data from a target document by reflecting the data structure between a plurality of words included in the target document.

Below, we will look at the system for extracting data from the target document in more detail along with the attached drawings.

1 is a conceptual diagram illustrating an information extraction system according to the present invention.

The information extraction system 100 according to the present invention may be implemented in the form of an application or software. According to the software implementation of the information extraction system 100 of the present invention, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

The software implementation according to the present invention is implemented by the information extraction system 100 shown in FIG. 1, and the configuration of the information extraction system 100 will be looked at in more detail below.

The information extraction system 100 according to the present invention can generate OCR data through OCR on images. In this specification, OCR data may include text extracted from an image and location information corresponding to the extracted text. Here, the location information defines the location of the extracted text within the image (or paper document).

Here, the image may be an image obtained by scanning a paper document, or may be an image obtained through various other methods.

Meanwhile, as shown in FIG. 1, the information extraction system 100 according to the present invention may include at least one of a communication unit 110, a storage unit 120, an OCR unit 130, and a control unit 140. .

The communication unit 110 is a means for receiving an image 10 obtained by scanning a paper document, and may include at least one of a communication unit, a scanning unit, and an input unit, or may be comprised of other means capable of receiving the image 10. .

The information extraction system 100 may extract data using the image 10 received through the communication unit 110.

Next, the storage unit 120 can be configured to store various information according to the present invention. The types of storage unit 120 may be very diverse, and at least some of them may refer to an external server 150, at least one of a cloud server and a database (DB). In other words, the storage unit 120 is sufficient as a space where information related to the present invention is stored, and it can be understood that there are no restrictions on physical space.

The storage unit 120 may store at least one of i) a scanned paper document image 10 and data related thereto, ii) learning data used for machine learning of an information extraction model, and iii) extracted data.

The storage unit 120 may store information related to the above-mentioned categories. Specifically, the information related to the category may store conceptual information about the category, text indicating the category name corresponding to the category, and information defining the value corresponding to the category.

Here, the category name may also be referred to as a field name or an attribute corresponding to the category.

Meanwhile, a value corresponding to a category may include at least one text belonging to the category.

Meanwhile, the information related to the category can be updated as the information extraction model according to the present invention is trained. Specifically, as training for the information extraction model progresses, the number of categories defined in the storage unit 120 may increase, and information related to the category includes conceptual information about the category and the category name corresponding to the category. At least one of the information defining the value corresponding to the representing text and category may be updated.

Next, the OCR (Optical Character Reader, 130) unit is a means for recognizing content included in the image 10, and can recognize content included in the image 10 through at least one of various content recognition algorithms. The OCR unit 130 can recognize content using an algorithm based on artificial intelligence. Here, the content may include text (characters).

The OCR unit can extract text included in the image and location information of the text. Here, the location information of the text may include information about where the text is located within the image 10.

Next, the control unit 140 may control the overall operation of the information extraction system 100 related to the present invention. The control unit 140 may include a processor (or artificial intelligence processor) capable of processing artificial intelligence algorithms. The control unit 140 may recognize text from the image 10 and extract information about the recognized text based on a deep learning algorithm.

Furthermore, the control unit 140 provides a work area for information extraction, and this work area may also be called a “user environment” or “user interface” for performing information extraction or machine learning for information extraction. You can.

This work area can be output (or provided) on the display unit of the electronic device. Furthermore, the control unit 140 performs information extraction or extracts information based on user input received through a user input unit (e.g., touch screen, mouse, etc.) provided in or linked to the electronic device. Machine learning can be performed for this purpose.

Meanwhile, in the present invention, there is no particular limitation on the type of electronic device on which the work area is output, as long as the application according to the present invention can be driven. For example, electronic devices include smart phones, mobile phones, tablet PCs, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), smart mirrors, and smart devices. It can be at least one TV (smart TV).

In the present invention, separate reference numerals are not assigned to electronic devices or display units and user input units provided in electronic devices. However, it will be apparent to those skilled in the art that the work area described in the present invention is output on the display unit of the electronic device, and user input is received through a user input unit provided in the electronic device or linked to the electronic device.

Below, we will look in more detail at how to perform information extraction using the above-described information extraction system. In particular, below, we will first look at the information extraction method along with the flow chart.

Figure 2 is a flowchart for explaining the information extraction method according to the present invention, Figure 3 is a conceptual diagram for explaining the information extraction model according to the present invention, and Figure 4 is a tree (tree) of data output from the information extraction model according to the present invention. This is a conceptual diagram showing a tree-like appearance.

In the information extraction method according to the present invention, a step is performed to extract a plurality of words and location information for each of the plurality of words by performing OCR (Optical Character Recognition) on the target document (S110).

Here, the target document may be a paper document or an image of the paper document that is the target of information extraction. As previously discussed, the image 10 for a paper document may be received through various routes.

OCR data may include text included in the image and location information of the text.

The text included in the OCR data may be divided into a plurality of text segments according to preset standards included in the OCR data.

In one embodiment, a text fragment may be divided into word units, or a plurality of adjacent texts may be divided into one text fragment based on location information of the text.

In the process of classifying text fragments, multiple text fragments may be integrated into one, or one text fragment may be separated into multiple text fragments. For example, the text “105 h-street” extracted from image 10 in Figure 1 is divided into two text fragments (“105” and “h-street”) and then divided into one text fragment (“105 h -street”). For another example, “Happy store” extracted from image 10 in FIG. 1 can be divided into one text piece “Happy store” and then separated into multiple text pieces (“Happy” and “store”). there is.

The control unit 140 may integrate or separate text pieces based on the location information of the text. The model by which the control unit 140 integrates or separates text pieces may be created through machine learning, but is not limited to this.

Through the above-described process, multiple words can be extracted.

Meanwhile, location information may be matched for each of the plurality of extracted words. Here, location information corresponding to the extracted word may be location information on the target document. Specifically, the location information may use a point on the image of the target document as a reference point and include two-dimensional coordinate information about the reference point.

As described above, the location information may be relative coordinates, and the distance between a plurality of extracted words may be calculated using the location information.

Location information for each of a plurality of words can be used to determine the relationship between the plurality of words. This will be described later.

Next, a step of generating a plurality of tokens corresponding to each of the plurality of words is performed (S120).

The plurality of tokens may be generated as one of three types. First, the first and second types among the plurality of token types will be described.

The plurality of tokens may have either a first type token having a category attribute and a second type token having a data attribute corresponding to a value of the category.

Tokens corresponding to the first and second types may be generated based on the meaning of each of a plurality of words extracted from the target document.

In one embodiment, the plurality of words are, based on the meaning of each of the plurality of words and positional information for each of the plurality of words in the target document, the plurality of words having the attribute of the category. It is classified into at least one of a first type word corresponding to the type and a second type word having the properties of the data.

The classification process can be performed through an artificial intelligence model. The artificial intelligence model determines which category each of the plurality of words belongs to based on the plurality of predefined categories, and determines whether each of the plurality of words is an “attribute” (type 1) or a “value” (type 1). Type 2).

A first type token is generated corresponding to each of the first type words classified as the first type among the plurality of words, and a second type token corresponding to each of the second type words classified as the second type among the plurality of words is generated. A type token is created.

The first type token of the first type is a token that defines the attributes of a category and may include text indicating the category name (field name, category attribute). The text included in the first type token may be a category name included in information related to a previously stored category. Here, the text included in a specific first type token may not be the same as the first type word on which the specific first type token was created.

For example, referring to FIG. 3, the first type token generated based on the word “product name” classified as the first type does not include “product name” but includes the text “name”.

The second type of second type token may be the word itself extracted from the target document.

For example, referring to FIG. 3, the second type token corresponding to the word “Happy” classified as type 2 may be the text “Happy” itself.

Meanwhile, a specific first type token may be generated based on the meaning of a second type word. Specifically, a first type token may be generated for each category to which the second type word classified as the second type among the plurality of words belongs.

In one embodiment, if a specific second type word corresponding to a category value of a specific category exists among the plurality of words, and a word corresponding to the specific category does not exist, the word corresponding to the category value of the specific category exists. Based on a specific second type word, a specific first type token corresponding to the specific category may be generated.

For example, referring to FIG. 3, there is no word corresponding to the attribute of the “store name” category in the target document 10. The present invention specifies the “store name” category among a plurality of predefined categories based on the meaning of the second type words “Happy” and “Store” (11) extracted from the target document (10), and “store name” ” You can create a type 1 token corresponding to the category.

As described above, the present invention specifies the category included in the target document based on the actual meaning of the second type word even if there is no word indicating the attribute of the category in the target document.

In one embodiment, referring to Figure 3, in the target document 10, a plurality of words (“Happy”, “store”, “105 h-street”, “Order number:”, “20210520011”, “Product name ”, “Quantity”, “Amount”, “Carrot”, “3”, “2997”, “tomato”, “2”, “1000”) are extracted.

Meanwhile, some of the extracted words (“Product Name”, “Quantity”, “Amount”, “Order Number:”) are classified as Type 1 words, and some of the remaining words (“Happy”, “store”, “105 h -street”, “20210520011”, “Carrot”, “3”, “2997”, “tomato”, “2”, “1000”, “20210520011”) can be classified as type 2 words.

Afterwards, based on the meaning of each of the extracted words, some categories corresponding to each of the plurality of words among the plurality of predefined categories are specified. Specifically, based on the meaning of the extracted words, “store”, “name” (a subcategory of the “store” category), “address”, “item”, “list”, “menu”, “name” (“menu” ” category’s subcategories), “count”, and “price” categories can be specified.

Afterwards, the first type of token corresponding to each of the “address”, “item”, “list”, “menu”, “name” (subcategories of the “menu” category), “count”, and “price” categories is generated. And, second-type tokens corresponding to words classified as second-type words are generated.

Meanwhile, among the data to be extracted from the target document, data in categories that are unnecessary to the user can be set to be deleted when data is extracted. Users can set categories of data that are unnecessary to the user in advance and set tokens not to be created for data in those categories. Words that fall into pre-set categories (“Order number:”, “20210520011”) may be deleted.

In one embodiment, referring to the table shown in FIG. 3, a plurality of tokens (Output tokens, 320) may be generated based on a plurality of words classified as first and second type words. A plurality of tokens may be generated sequentially according to the flow of time (t, 310).

The plurality of tokens may include a third type of third type token. The third type of token will be described later.

Next, a step of arranging the plurality of tokens is performed so that the data structure between the plurality of words included in the target document is reflected (S130).

A plurality of words included in the target document may have a certain data structure. A data structure is a structure that represents the characteristics of a plurality of words and the relationships between words from a logical perspective. Types of data structures may include, but are not limited to, at least one of a list, array, tree, graph, queue, and stack.

For example, the target document 10 described in Figure 1 has a data structure that includes the “Store Name” category, “Store Address” category, “Order Product Name” category, “Order Quantity” category, and “Order Amount” category. . However, the data structure corresponding to the target document 10 is not absolute, and the data structure for the target document 10 may vary depending on the method of defining the data structure.

In this specification, the step of arranging a plurality of tokens is implemented to generate data with the same structure as the data structure between the plurality of words included in the target document, or the data structure between the plurality of words included in the target document is implemented according to a preset method. It can be implemented to generate data with a modified structure.

For example, the target document 10 described in Figure 1 has a data structure that includes the “Store Name” category, “Store Address” category, “Order Product Name” category, “Order Quantity” category, and “Order Amount” category. , The token arrangement step according to the present invention is the “Store” category, which is the upper category of the “Store Name” category and the “Store Address” category, and the “Store Name” category and “Store Address” category, which are lower concepts of the “Store” category. Data may be implemented to have a data structure including the associated “Order Product Name” category, “Order Quantity” category, and “Order Amount” category.

As described above, the present invention can newly define the category of a higher-level concept of the category included in the target document 10, or allow data of different categories to be related to each other.

In the step of arranging a plurality of tokens, the plurality of tokens are sequentially arranged using the meaning of the plurality of words and location information for each of the plurality of words.

The step of arranging the plurality of tokens does not necessarily need to be performed after all the plurality of tokens are generated. The present invention can also be performed by arranging the generated tokens while some of the plurality of tokens are generated, and then generating additional tokens.

In one embodiment, the present invention can be performed by arranging the generated tokens each time one token is generated.

Hereinafter, a method of arranging a plurality of tokens will be described in detail.

The first type token corresponding to the first category to which the specific second type word classified as the second type belongs and the second type token corresponding to the specific second type word may be arranged sequentially.

In one embodiment, first type tokens and second type tokens corresponding to the first category, which is the same category, may be arranged sequentially. In this specification, for convenience of explanation, an embodiment in which the first type token is arranged before the second type token is described, but the present invention is not limited to this, and the second type token may be arranged before the first type token. .

In another embodiment, when token arrangement is performed by arranging the generated tokens every time one token is generated, tokens arranged first may be generated before tokens arranged later.

Meanwhile, there may be a plurality of specific second type words belonging to the first category. In this case, a plurality of second type tokens corresponding to each of the plurality of specific second type words are sequentially arranged.

At this time, the arrangement order of the plurality of sequentially arranged second type tokens may be determined based on the meaning of each of the plurality of specific second type words and location information on the target document.

For example, referring to Figure 3, among the generated tokens 320, the second type tokens, “Happy” token and “store” token, are arranged sequentially following the “[name]” token, and the two Based on each word of the second type of token, the “Happy” token is arranged first, and the “store” token is arranged after the “Happy” token.

As described above, the present invention minimizes the size of words extracted from the target document, so that even when the relative position between texts becomes irregular as the shape of the target document is transformed or part of it is lost, a high Enables information to be extracted with accuracy.

Meanwhile, tokens of the first type belonging to different categories are arranged sequentially. Specifically, a first type token corresponding to the first category and a first type token corresponding to a second category different from the first category are sequentially arranged.

At this time, between the first type token corresponding to the first category and the second type token corresponding to the second category, there is a third type of token to distinguish the tokens corresponding to each of the first category and the second category. 3 type tokens can be arranged.

In one embodiment, when only type 1 tokens exist (when only words corresponding to attributes are extracted from the target document and words corresponding to values are not extracted), tokens belonging to the first category are type 3. The tokens are arranged consecutively to the first type token.

In another embodiment, when the tokens belonging to the first category are a first type token and one second type token, the first type token corresponding to the first category is arranged first, and the first type token The second type token may be arranged in succession. The third type tokens are arranged consecutively to the second type tokens.

In another embodiment, when the token belonging to the first category is a first type token and a plurality of second type tokens, the first type token corresponding to the first category is arranged first, and the first type token is arranged first, and the first type token corresponding to the first category is arranged first. A plurality of second type tokens corresponding to the specific second type word are sequentially arranged in succession with the corresponding first type token, and a specific second type token corresponding to the specific second type word is arranged last. The third type token may be arranged consecutively to the second type token.

As described above, the third type of token may be arranged in the last order among at least one token belonging to a specific category.

Based on the meaning of each of the plurality of words extracted from the target document 10, a category corresponding to each of the plurality of words among the plurality of predefined categories may be specified. If, among a plurality of predefined categories, there is a category of a higher concept than the concept defined by the specified category, the category of the higher concept may be specified as a category corresponding to each of a plurality of words.

The category of the above-mentioned superordinate concept can be understood as a supercategory of a specific category. In this specification, the category of a higher-level concept for a specific category is referred to as a higher-level category, and the category of a lower-level concept of the higher-level category is referred to as a lower level category. Since the above expression is a relative expression, it can be used based on a specific category.

For example, if there is a category corresponding to a higher-level concept of the first category, the category corresponding to the higher-level concept may be specified as a third category. In this case, the first category may be referred to as a lower category of the third category, and the third category may be referred to as a higher category of the first category.

When a category of a concept higher than the category of the higher-level concept has been previously defined, the category of the higher-level concept than the category of the higher-level concept may also be specified as a category corresponding to each of a plurality of words.

When the upper category of a specific category is specified, a first type token of the first type corresponding to the upper category may be generated. For example, a first type token of the first type corresponding to the specified third category is generated. A first type token corresponding to the third category, a first type token corresponding to the first category, and a second type token corresponding to the first category may be arranged sequentially.

Meanwhile, a plurality of subcategories of a specific category may be specified. For example, the first and second categories may be subcategories of the third category. In this case, first type tokens of the first type corresponding to the first to third categories are generated. A first type token corresponding to the third category, a first type token corresponding to the first category, a second type token corresponding to the first category, a third type token, a first type token corresponding to the second category The type token and the second type token corresponding to the second category may be arranged sequentially.

As described above, third-type tokens are arranged between first-type tokens corresponding to each of the two subcategories.

Referring to FIG. 3, a method of arranging tokens generated based on an actual receipt will be described in more detail.

In Figure 3, it is explained that the order in which tokens are generated and the order in which they are arranged are the same, but the order in which tokens are created and the order in which they are arranged do not necessarily match.

From the target document 10, based on the meaning of each extracted word, “[store]”, “[name]” (corresponding to a subcategory of the category corresponding to the “[store]” token), “[address] ]” (corresponds to a subcategory of the category corresponding to the “[store]” token), “[item]”, “[list]” (corresponds to a subcategory of the category corresponding to the “[item]” token), “ [menu]” (corresponds to a subcategory of the category corresponding to the “[list]” token), “[name]” (corresponds to a subcategory of the category corresponding to the “[menu]” token), “[count]” (corresponding to a subcategory of the category corresponding to the “[menu]” token) and “[price]” (corresponding to a subcategory of the category corresponding to the “[menu]” token) tokens are generated.

In addition, words classified as Type 2 (“Happy”, “store”, “105 h-street”, “20210520011”, “Carrot”, “3”, “2997”, “tomato”, “2”, A second type of token corresponding to “1000”) is created.

The [store] token corresponding to one of the two categories “store” and “item” that correspond to the highest concept among the plurality of categories is arranged as the first token. Afterwards, the [name] tokens corresponding to the “name” category, which correspond to the subcategories of the “store” category, are sequentially arranged in the [store] token, and the “Happy” corresponding to the second type word belonging to the “name” category. , “store” tokens are placed consecutively following the [name] token.

After all the tokens (321a) belonging to the “name” category are arranged, a third type token ([reduce], 321b) is sequentially added to the last token (“store” token”) among the tokens corresponding to the “name” category. are arranged. Afterwards, the [address] token corresponding to the “address” category, which corresponds to the subcategory of the “store” category, is sequentially arranged in the [reduce] token.

Although not shown, the arrangement method will be additionally described later. The “105 h-street” token corresponding to the second type word belonging to the “address” category is placed consecutively following the [address] token. After all tokens belonging to the “address” category are arranged, type 3 tokens are sequentially arranged at the very last token (“105 h-street” token) among the tokens corresponding to the “address” category.

Meanwhile, since all tokens corresponding to the “store” category have been arranged, the third type token is additionally arranged once more. In other words, two third-type tokens are sequentially arranged in the last token (“105 h-street” token) among the tokens corresponding to the “address” category.

Following the two third type tokens arranged consecutively, an [item] token corresponding to the “item” category is arranged. Following the [item] token, the [list] token corresponding to the “list” category, which is a subcategory of the “item” category, the [menu] token, corresponding to the “menu” category, which is a subcategory of the “list” category, “menu” [name] tokens corresponding to the “name” category, which is a subcategory of the category, are arranged sequentially. Following the [name] token, “carrot” tokens corresponding to values in the “name” category are arranged sequentially, and the “carrot” token is followed by a [reduce] token.

Afterwards, [count] tokens corresponding to the “count” category, which is a subcategory of the “menu” category, are sequentially arranged. Following the [count] token, “3” tokens corresponding to values in the “count” category are sequentially arranged, and the “3” token is followed by a [reduce] token.

Afterwards, [price] tokens corresponding to the “price” category, which is a subcategory of the “menu” category, are sequentially arranged. Following the [price] token, “2997” tokens corresponding to values in the “price” category are arranged sequentially, and the “2997” token is arranged.

All tokens corresponding to the “price” category, “menu” category, and “list” category are arranged, and following the “2997” token, three third-type tokens are arranged consecutively.

Afterwards, following the three third type tokens arranged consecutively, the [list] token corresponding to the “list” category, which is a subcategory of the “item” category, and the “menu” category, which is a subcategory of the “list” category. The [menu] token and the [name] token corresponding to the “name” category, which is a subcategory of the “menu” category, are arranged sequentially. Following the [name] token, “tomato” tokens corresponding to values in the “name” category are arranged sequentially, and the “tomato” token is followed by a [reduce] token.

Afterwards, [count] tokens corresponding to the “tomato” category, which is a subcategory of the “menu” category, are sequentially arranged. Following the [count] token, “2” tokens corresponding to values in the “count” category are sequentially arranged, and the “2” token is followed by a [reduce] token.

Afterwards, [price] tokens corresponding to the “price” category, which is a subcategory of the “menu” category, are sequentially arranged. Following the [price] token, “1000” tokens corresponding to values in the “price” category are arranged sequentially, and then “1000” tokens are arranged. The token arrangement ends with the “1000” token.

Meanwhile, data reflecting the data structure between the plurality of words included in the target document may be generated based on the generated tokens and the order in which the tokens are arranged.

Data reflecting the data structure between the plurality of words included in the target document is generated by performing actions corresponding to the tokens in the order in which the tokens are arranged.

In one embodiment, data reflecting the data structure between the plurality of words included in the target document may be generated in the form of a tree.

Tree data includes a plurality of nodes and a plurality of hierarchies, each node corresponds to one of a plurality of categories, and at least one of the properties and values of the corresponding category matches.

When creating tree data, the top node (S) is set as the standard node, and then, in the order in which the tokens are arranged, actions corresponding to the tokens are performed to create nodes or match data corresponding to the nodes.

The action for the first type of token is to create a child node of the standard node and change the created child node to the standard node, and the action for the second type token is to correspond to the standard node and the second type token. Matching words with values. The action for the third type of token is to change the node one level higher than the standard node to the standard node.

According to the above-mentioned criteria, tree data can be generated from tokens arranged in order.

Referring to FIGS. 3 and 4, an embodiment of generating tree data will be described. The description of how to generate and arrange tokens from the target document 10 is replaced with the content described above.

Referring to FIG. 3, the action 330 corresponding to the token is performed in the order in which the tokens are arranged (310). The numbers included in the token generation time column 311 of the table shown in FIG. 3 may be the order in which tokens are processed, and the actions included in the action column 331 are performed in the token processing order. Accordingly, parsing 340 is performed, and data included in the parsing string 341 is sequentially generated.

First, the top node (S) of the tree is set as the standard node, and the action (NT (store)) corresponding to the [store] token is performed at the standard node. Accordingly, the [store] node 410, which is a child node of the standard node (S), is created, and the standard node is changed to the [store] node 410.

Next, the action (NT(name)) corresponding to the [name] token is performed, and the [name] node 420, a child node of the [store] node 410, is created, and the reference node is the [name] node. Changed to (420).

Next, an action (GEN(Happy)) that matches the word “Happy” corresponding to the “Happy” token is performed on the [name] node 420. Next, an action (GEN(store)) that matches the word “store” corresponding to the “store” token is performed on the [name] node 420. Words matched to the [name] node 420 are serialized in the order in which the words are matched. Accordingly, “Happy store” (421) is matched to the [name] node (420).

As all actions included in area 331a of the table shown in FIG. 3 are performed, nodes 410 and 420 in FIG. 4 are created, and the text corresponding to node 420 is matched.

Next, the action corresponding to the [reduce] token included in area 331b of the table shown in FIG. 3 is performed. Accordingly, the standard node changes from the [name] node to the [store] node, which is the parent node of the [name] node.

Next, the action (NT(address)) corresponding to the [address] token is performed, the [address] node, which is a child node of the [store] node 410, is created, and the reference node is changed to the [address] node. .

Next, an action (GEN(105 h-street)) is performed that matches the word “105 h-street” corresponding to the “105 h-street” token to the [address] node. Accordingly, “105 h-street” is matched to the [address] node.

Next, the action corresponding to the two [reduce] tokens placed in succession is performed, and the reference node is changed to the top node (S).

Next, actions corresponding to the [item] token, [list] token, [menu] token, and [name] token are performed sequentially, with the [item] node 430, [list] node 440, and [menu] ] node 450 and [name] node 460 are created sequentially. Subsequently, as the action corresponding to the “carrot” token is performed, the word “carrot” (461) is matched to the [name] node (460).

According to the above-described rule, when the action on the generated token is repeated, the tree data shown in FIG. 4 can be generated.

The tree shown in FIG. 4 is data in which key-value pairs for each word included in the target document 10 are completely matched, and the categories of each data are completely defined.

As described above, the present invention includes the meaning of the plurality of words extracted from the target document and the location information for each of the plurality of words, using the generated token and the arrangement of the generated token to be included in the target document. Data that reflects the data structure between multiple words can be generated.

Meanwhile, the present invention determines the accuracy of the information extraction model based on the above-described tree data when performing machine learning on the information extraction model.

Figure 5 is a conceptual diagram showing the machine learning method of the information extraction model according to the present invention, and Figure 6 is a conceptual diagram showing calculating the accuracy of data output from the information extraction model according to the present invention.

Referring to FIG. 5, after performing OCR 511 on the target document 10 and estimating a plurality of words and location information 512 for each of the plurality of words, the plurality of words and location information are viewed. It is input into the information extraction model 513 according to the invention. Accordingly, sequentially arranged tokens are arranged sequentially.

Meanwhile, data corresponding to the target document 10 is processed into learning data. Processing for learning data is performed by the parse annotator 521, which outputs the previously stored data in the form of a plurality of tokens arranged sequentially. That is, the parse annotator 521 outputs data in the same format as the output data of the machine learning model according to the present invention. Through this, original data (learning data) for machine learning is created.

The error between tree data generated based on a plurality of tokens output from the information extraction model according to the present invention and the original data can be calculated, and training can be performed in a direction to minimize the error.

In one embodiment, referring to FIG. 6, the error of the information extraction model may be calculated through comparison between nodes included in tree data generated from the information extraction model and nodes constituting the original data. Specifically, the error of the information extraction model can be calculated by changing the tree data output from the information extraction model (top tree in FIG. 6) to be the same as the tree of the original data (bottom tree in FIG. 6).

Specifically, determine whether each node (610, 620, 630) of the tree data (top tree in FIG. 6) output from the information extraction model and the values (621, 631) matched to the nodes match the tree of the original data. and modify the non-matching nodes 630 and non-matching values 631 to be the same as the original data, so that the non-matching nodes 630 are modified to nodes 630' matching the original data, and The missing value (631) is modified to a value (631') that matches the original data. In this process, the score of the tree data output from the information extraction model is calculated. Since the score calculation method uses a previously known method, detailed description is omitted.

As described above, machine learning of the information extraction model according to the present invention is performed by calculating the accuracy of sequentially arranged tokens output from the information extraction model. Since the output tokens are not data in which “key-value” pairs are completely matched, it becomes easy to generate learning data to calculate the accuracy of the information extraction model.

Specifically, when generating learning data, tokens arranged in a certain order are generated according to predetermined rules (token generation rules), and then the accuracy of the information extraction model is determined by comparing them with the tokens output from the information extraction model. And, machine learning can be performed in the direction of improving the accuracy of the information extraction model. Accordingly, the time required to generate learning data for machine learning of the information extraction model can be greatly reduced.

Meanwhile, the present invention discussed above can be implemented as a program that is executed by one or more processes on a computer and can be stored in a medium (or recording medium) that can be read by such a computer.

Furthermore, the present invention discussed above can be implemented as computer-readable codes or instructions on a program-recorded medium. That is, the present invention may be provided in the form of a program.

Meanwhile, computer-readable media includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is.

Furthermore, the computer-readable medium may be a server or cloud storage that includes storage and can be accessed by electronic devices through communication. In this case, the computer can download the program according to the present invention from a server or cloud storage through wired or wireless communication.

Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, that is, a CPU (Central Processing Unit), and there is no particular limitation on its type.

Meanwhile, the above detailed description should not be construed as restrictive in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (16)

Performing optical character recognition (OCR) on a target document to extract a plurality of words and location information for each of the plurality of words;
generating a plurality of tokens corresponding to each of the plurality of words; and
Arranging the plurality of tokens so that the data structure between the plurality of words included in the target document is reflected,
The plurality of tokens have one of a first type having a category attribute and a second type having a data attribute corresponding to a value of the category,
In the step of arranging the plurality of tokens,
Using the meaning of the plurality of words and the location information for each of the plurality of words, first type first type tokens corresponding to a specific category among different categories are arranged,
Arranging a second type token corresponding to the specific category consecutively to a first type token corresponding to the specific category,
Arranging a third type token sequentially to the last arranged token among the second type tokens corresponding to the specific category,
The third type token is used to distinguish tokens corresponding to the specific category from tokens corresponding to other categories. delete According to paragraph 1,
The step of generating a plurality of tokens corresponding to each of the plurality of words,
From a semi-structured document, comprising the step of generating at least one of a first type token having the first type and a second type token having the second type, based on the meaning of each of the plurality of words. How to extract information. According to paragraph 1,
Based on the meaning of each of the plurality of words and the location information for each of the plurality of words in the target document, the plurality of words are divided into a first type word having the first type and a second type word having the second type. A method of extracting information from a semi-structured document, further comprising the step of classifying into at least one of the words. According to paragraph 4,
The step of generating a plurality of tokens corresponding to each of the plurality of words,
Generating at least one first type token to each correspond to the first type word,
A method of extracting information from a semi-structured document, characterized in that generating at least one second type token to each correspond to the second type word. According to clause 5,
The step of generating a plurality of tokens corresponding to each of the plurality of words,
A method for extracting information from a semi-structured document, characterized in that generating a first type token for each category to which the second type word belongs. According to clause 6,
In the step of generating a plurality of tokens corresponding to each of the plurality of words, a specific second type word corresponding to a category value of the specific category exists in the plurality of words, and a word corresponding to the specific category does not exist. If not,
A method for extracting information from a semi-structured document, further comprising generating a specific first type token corresponding to the specific category based on the specific second type word. delete According to paragraph 4,
In the step of arranging the plurality of tokens,
If the plurality of words include a plurality of specific second type words corresponding to the category value of the specific category,
sequentially arranging a plurality of second type tokens corresponding to each of the plurality of specific second type words,
Information from a semi-structured document, wherein the arrangement order of the plurality of sequentially arranged second type tokens is determined based on the meaning of each of the plurality of specific second type words and location information on the target document. How to extract it. According to paragraph 1,
In the step of arranging the plurality of tokens,
If a word corresponding to the data attribute is not extracted from the target document,
sequentially arranging a first type token corresponding to the specific category and a first type token corresponding to the other category,
A method for extracting information from a semi-structured document, characterized in that a third type token is arranged between the first type token corresponding to the specific category and the first type token corresponding to the other category. delete According to paragraph 1,
Further comprising generating data having a structure corresponding to the target document using the plurality of tokens,
In the step of generating the data,
When the third type token is detected among the plurality of tokens, the specific token detected just before the third type token is detected and the specific token consecutive to the third type token are recognized as tokens corresponding to different categories. A method of extracting information from a semi-structured document, characterized in that: According to paragraph 1,
The step of generating a plurality of tokens corresponding to each of the plurality of words,
A method for extracting information from a semi-structured document, characterized in that when a category corresponding to a high-level concept of the specific category exists, a first type token corresponding to the high-level concept category of the specific category is generated. According to clause 13,
Information from a semi-structured document, wherein a first type token corresponding to a higher concept category of the specific category, a first type token corresponding to the specific category, and a second type token corresponding to the specific category are arranged sequentially. How to extract . An OCR unit that performs optical character recognition (OCR) on the target document to extract a plurality of words and location information for each of the plurality of words;
Generating a plurality of tokens corresponding to each of the plurality of words,
A control unit that arranges the plurality of tokens so that the data structure between the plurality of words included in the target document is reflected,
The plurality of tokens have one of a first type having a category attribute and a second type having a data attribute corresponding to a value of the category,
The control unit,
Using the meaning of the plurality of words and the location information for each of the plurality of words, first type first type tokens corresponding to a specific category among different categories are arranged,
Arranging a second type token corresponding to the specific category consecutively to a first type token corresponding to the specific category,
Arranging a third type token sequentially to the last arranged token among the second type tokens corresponding to the specific category,
The third type token is a system for extracting information from a semi-structured document, characterized in that it is used to distinguish tokens corresponding to a specific category from tokens corresponding to other categories. A program that is executed by one or more processes in an electronic device and can be stored in a computer-readable recording medium,
The above program is,
Performing optical character recognition (OCR) on a target document to extract a plurality of words and location information for each of the plurality of words;
generating a plurality of tokens corresponding to each of the plurality of words; and
Includes instructions for performing the step of arranging the plurality of tokens so that the data structure between the plurality of words included in the target document is reflected,
The plurality of tokens have one of a first type having a category attribute and a second type having a data attribute corresponding to a value of the category,
In the step of arranging the plurality of tokens,
Using the meaning of the plurality of words and the location information for each of the plurality of words, the first type token corresponding to a specific category among the different categories is arranged first,
Arranging a second type token corresponding to the specific category consecutively to a first type token corresponding to the specific category,
Arranging a third type token sequentially to the last arranged token among the second type tokens corresponding to the specific category,
The third type is a program storable on a computer-readable recording medium, characterized in that it is for distinguishing tokens corresponding to the specific category from tokens corresponding to other categories.

Images