JP2022164593A - Method and system for extracting information from semi-structured document - Google Patents

Abstract

To provide a method and system for extracting information from a semi-structured document.SOLUTION: A method for extracting information from a semi-structured document includes the steps of: performing optical character recognition (OCR) on a target document to extract a plurality of words and position information on each of the words; generating a plurality of tokens corresponding to the words, respectively; and arranging the tokens so as to reflect a data structure between the words included in the target document. In the step of arranging the tokens, the tokens are arranged using meanings of the words and the position information on each of the words.SELECTED DRAWING: Figure 1

Description

There is an application for exception to loss of novelty

The present invention relates to methods and systems for retrieving information from documents.

The dictionary meaning of artificial intelligence can be said to be technology that realizes human learning ability, reasoning ability, perceptual ability, natural language understanding ability, etc. by computer programs. Such artificial intelligence has achieved dramatic progress through deep learning, which combines machine learning with a neural network that mimics the human brain.

Deep learning is a technology that allows computers to judge and learn just like humans, and through this, to cluster or sort things and data. Recently, it has become possible to analyze not only text data but also image data, and it is actively used in a wide variety of industrial fields.

Along with the development of such artificial intelligence, various types of automation are progressing in the field of office automation. In particular, in the field of business automation, great efforts are being made to convert content printed on paper into data based on image data analysis technology utilizing artificial intelligence. As part of this, in the field of office automation, the content contained in the document is analyzed through image analysis technology (or analysis technology for image data) that converts paper documents into images and analyzes the content contained in the image. It is digitized, and in this case, techniques are required to analyze the image to match the type of content contained in the document.

For example, when digitizing a document such as a receipt, it is necessary to check the receipt format, the contents of the text contained in the receipt, and the position of the text contained in the receipt. Accurate analysis of various factors is required.

Therefore, various techniques have been developed for processing information contained in an image into data in a form that can be processed by electronic equipment. For example, Korean Patent No. 10-1181209 discloses a method of constructing an OCR database.

However, the methods developed so far are limited to the level of sorting data according to rules determined empirically by humans. We need to continue to work on developing applications that can be reproduced accurately.

Korean Patent No. 10-1181209

The present invention relates to a method and system for retrieving information from a target document reflecting the inter-word data structure contained in the target document.

The present invention also provides a method and system for retrieving data from a target document in a form that facilitates training of a machine learning model for retrieving data from the target document.

Specifically, the present invention realizes the format of the learning data used to determine the accuracy of the machine learning model that extracts data from the target document and the data output from the machine learning model in a form that is easy for humans to process. To provide a data retrieval method and system that can reduce the time and cost of determining the accuracy of a machine learning model by doing so.

Further, the present invention seeks to provide a data retrieval method and system in which relationships between data contained in a target document are provided.

To solve the above-mentioned problems, the present invention provides a step of performing optical character recognition (OCR) on a target document to retrieve a plurality of words and location information for each of the plurality of words; generating a plurality of tokens corresponding to each of a plurality of terms; and arranging the plurality of tokens to reflect a data structure among the plurality of terms contained in the target document; wherein the step of arranging the plurality of tokens comprises arranging the plurality of tokens using the meanings of the plurality of terms and location information about each of the plurality of terms, can provide a method for retrieving the

Further, the present invention includes an OCR unit that performs optical character recognition (OCR) on a target document and extracts a plurality of words and position information about each of the plurality of words, and a control unit that generates a corresponding plurality of tokens and arranges the plurality of tokens so as to reflect a data structure between the plurality of words contained in the target document, wherein the control unit , using the meanings of the plurality of terms and location information for each of the plurality of terms to order the plurality of tokens, a system for extracting information from a semi-structured document.

Further, the present invention is a computer program comprising a plurality of instructions which, when executed, perform optical character recognition (OCR) on a target document to generate a plurality of words and the plurality of words generating a plurality of tokens corresponding to each of said plurality of terms; and reflecting a data structure between said plurality of terms contained in said target document. and arranging the plurality of tokens, wherein in the arranging the plurality of tokens, the plurality of tokens are arranged using the meanings of the plurality of words and location information about each of the plurality of words It is possible to provide a computer program running on a computer characterized by arranging.

As described above, the present invention uses the meanings of the plurality of terms and the location information for each of the plurality of terms retrieved from the target document, and uses the generated tokens and generated sequences of tokens. , it is possible to generate data that reflects the data structure between multiple words contained in the target document.

Further, the machine learning of the information retrieval model according to the present invention is performed by calculating the accuracy of a plurality of tokens output from the information retrieval model. Since the output tokens do not perfectly reflect the relationships between words, it is easier to generate learning data for calculating the accuracy of the information retrieval model.

Specifically, after generating tokens arranged in a certain order according to rules (token generation rules) that have already been established when generating learning data, the tokens output from the information retrieval model are compared. , the accuracy of an information retrieval model can be determined, and training can be performed on the information retrieval model to improve the accuracy of the information retrieval model. This greatly reduces the time it takes to generate learning data for machine learning of information retrieval models.

1 is a conceptual diagram for explaining an information retrieval system according to the present invention; FIG. 4 is a flow chart for explaining an information retrieval method according to the present invention; FIG. 4 is a conceptual diagram for explaining an information extraction model according to the present invention; FIG. 2 is a conceptual diagram showing how data output from an information retrieval model according to the present invention is treed; 1 is a conceptual diagram showing a machine learning method for an information retrieval model according to the present invention; FIG. FIG. 4 is a conceptual diagram showing how the accuracy of data output from the information extraction model according to the present invention is calculated; 1 is a conceptual diagram illustrating a conventional machine learning method for information retrieval; FIG.

Hereinafter, the embodiments disclosed in this specification will be described in detail based on the accompanying drawings. Redundant explanations are omitted. The suffixes "module" and "part" for the components used in the following description are added or mixed only in consideration of the ease of preparation of the specification, and are distinguished from each other by themselves. does not have any meaning or role. Moreover, in describing the embodiments disclosed herein, it is acknowledged that specific descriptions of related known technologies may obscure the gist of the embodiments disclosed herein. detailed description is omitted. It should be noted that the accompanying drawings are merely for facilitating understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the accompanying drawings. , to include any modifications, equivalents, or alternatives falling within the spirit and scope of the invention.

Phrases including ordinal numbers such as “first,” “second,” etc. can be used to describe various components, but said components are in no way limited by said terms. The above wording can only be used to distinguish one component from another.

When a component is referred to as being “coupled” or “connected” to another component, said component is directly linked to said other component. , or that there may be other components between them. In contrast, when a component is referred to as being “directly coupled” or “directly connected” to another component, there is no other structure between them. It should be understood that the element does not exist.

Singular expressions include pluralities unless the context clearly dictates otherwise.

In this application, terms such as "comprising," "including," or "having" refer to the presence of any feature, number, step, act, component, part, or combination thereof described in the specification. It should be understood that it is a specification only and does not preclude the presence or addition of one or more other features, figures, steps, acts, components, parts, or combinations thereof. is.

As mentioned above, with the development of artificial intelligence, various forms of office automation are progressing. There is a growing need to digitize (data) the format contained in the paper document as it is.

For example, a document 10 such as a receipt (receipt) shown in FIG. (Category), and for efficient data processing, the data contained in the paper document must be digitized by associating the same categories with each other.

As used herein, "category" means an item at any level within a defined taxonomy. The criteria for distinguishing data belonging to one category from data belonging to another category are not necessarily absolute and may vary depending on any rules governing the categories. Such rules can be applied differently to paper documents and data obtained by digitizing paper documents.

For example, a particular receipt consisting of a paper document contains five categories: "store name/department name", "department address", "order item name", "order item quantity", and "order item amount". However, when the data contained in the paper document is digitized, it may be reduced to two categories: "place of sale" and "item to order."

On the other hand, in this specification, words indicating attributes of a specific category are classified into the first type of words. For example, the words of the first type may include words (“store name/selling area name:”, “quantity:”, “selling area address”) that define the superordinate concept of the data belonging to the category. .

On the other hand, for purposes of this specification, words that indicate a particular category of values are grouped into a second type of words. For example, a second type of word may be data belonging to a category (“Happy store”, “105 h-street”, “Carrot”). In this specification, the words of the first type classified into the first type are called "field names", and the words of the second type classified into the second type are called "field values". good too.

On the other hand, from the point of view of data in machine-processable form, the first type of terms may be referred to as "keys" and the second type of terms may be referred to as "values." may be called

According to the definitions of terms and phrases provided above, a paper document, such as a receipt, may contain data in attribute-value pairs that fall within the same category. However, there is no limitation to this, and a paper document such as a paper receipt may contain only field values without field names for a particular category. In this case, the field name is simply omitted from the paper document, and the field value corresponding to the omitted field name contains the meaning associated with the field name.

A paper document, on the other hand, may contain only field names without field values. In this case, there is no value assigned to the particular item, and even if there is no value assigned to the particular item, the particular item may exist in the paper document.

The present invention will be described in detail below based on the definitions of the terms described above.

The paper document described above maintains relationships between data of the same category contained in the paper document and can be converted into machine-understandable data.

In one embodiment, an Optical Character Reader (OCR) can be used to retrieve words contained in a paper document. In the case of a document including data of multiple categories, such as a receipt, digitized words can be sorted into the same category independently through a parsing process for extracted words. This allows the formation of data containing multiple attribute-value pairs.

Through machine learning, efforts continue to be made to impose a predetermined data structure on unrelated digitized words using OCR. Referring to FIG. 7, the conventional information retrieval model extracts multiple words and location information from the target document 10 by performing OCR 711 on the target document 10, and based on this, a serialization model (Serializer) Through 712 , it serialized parts of the retrieved words, through Tagging model 713 categorized the serialized text, and through Tag2parse model 714 parsing.

It then passes through a parse normalizer 715 to generate JSON formatted data.

Conventional information extraction model training is performed by calculating the error between the data output from the Tag2parse model 714 and the original data and minimizing the error. At this time, in order to calculate the error, original data (hereinafter referred to as learning data) must be generated. The source data displays (721) annotations for each of the terms contained in the target document 10, serializes (722) the text segments contained in the target document 10, and passes through the tagging model 723 the inter-term is generated by parsing Since all of the processes described above are performed by human beings themselves, it takes a very long time to generate learning data for machine learning.

The present invention provides an information retrieval system and method that can reduce the time required to generate the learning data described above and minimize the cost required for machine learning of the information retrieval model.

The present invention provides a system for retrieving data from a target document that reflects the data structure between words contained in the target document.

A system for retrieving data from a target document is described in detail below in conjunction with the accompanying drawings.

FIG. 1 is a conceptual diagram for explaining an information retrieval system according to the present invention.

The information retrieval system 100 according to the present invention can be implemented in the form of an application or software. According to the software implementation of the information retrieval system 100 of the present invention, embodiments such as the procedures and functions described herein can be implemented by separate software modules. Each of the software modules can perform one or more functions and operations described herein.

Software implementation according to the present invention is implemented by the information retrieval system 100 shown in FIG. 1, and the configuration of the information retrieval system 100 will be described in more detail below.

The information retrieval system 100 according to the present invention can generate OCR data by performing OCR on an image. As used herein, OCR data may include text retrieved from an image and location information corresponding to the retrieved text. Here, the position information defines the position within the image (or paper document) of the retrieved text.

Here, the image may be an image obtained by scanning a paper document, or may be an image obtained by various other methods.

On the other hand, as shown in FIG. 1, the information retrieval system 100 according to the present invention may comprise at least one of the communication section 110, the storage section 120, the OCR section 130, and the control section 140. FIG.

The communication unit 110 is means for receiving an image (or document) 10 obtained by scanning a paper document, and may include at least one of a communication unit, a scanning unit, and an input unit. may be configured from means capable of receiving the

Information retrieval system 100 may retrieve data using image 10 received via communication unit 110 .

Storage unit 120 may then be configured to store various information according to the present invention. The types of the storage unit 120 are very diverse, and at least some of them may mean an external server 150 (at least one of a cloud server and a database (DB)). In other words, it may be understood that the storage unit 120 is not limited to a physical space as long as it is a space for storing information related to the present invention.

The storage unit 120 stores at least one of i) an image 10 obtained by scanning a paper document and data related thereto, ii) learning data used for machine learning of an information extraction model, and iii) extracted data. may be

The storage unit 120 may store information related to the categories described above. Specifically, as the information related to the category, conceptual information related to the category, text indicating the category name corresponding to the category, and information defining the value corresponding to the category may be stored.

Here, the category name may also be referred to as a field name or an attribute corresponding to the category.

On the other hand, a value corresponding to a category may contain at least one text belonging to the category.

On the other hand, the information associated with the categories may be updated as the information retrieval model of the present invention is trained. Specifically, as the information extraction model is trained, the number of categories defined in the storage unit 120 may be increased. At least one of the text indicating the first name and the information defining the value corresponding to the category may be updated.

Next, an OCR (Optical Character Reader) unit 130 is means for recognizing content included in the image 10, and recognizes the content included in the image 10 through at least one of various content recognition algorithms. can do. The OCR unit 130 can recognize content using algorithms based on artificial intelligence. Here, the content may include text (characters).

The OCR unit may retrieve text contained in the image and location information for the text. Here, the text position information may include information about where the text is positioned within the image 10 .

The controller 140 may then be configured to control the overall operation of the information retrieval system 100 according to the present invention. The control unit 140 may include a processor capable of processing artificial intelligence algorithms (or an artificial intelligence processor). Control unit 140 may recognize text from image 10 and perform information retrieval on the recognized text based on a deep learning algorithm.

In addition, the control unit 140 provides a work area for performing information retrieval, such a work area being a "user environment" or It may also be named "user interface".

Such a working area may be output (or provided) on the display portion of the electronic device. Further, the control unit 140 performs information retrieval based on user input received via a user input unit (e.g., touch screen, mouse, etc.) located on or associated with the electronic device, Alternatively, machine learning for information extraction can be performed.

On the other hand, in the present invention, there is no particular limitation on the type of electronic device to which the work area is output, as long as the application according to the present invention can be driven. For example, electronic devices include smart phones, mobile phones, tablet PCs, computers, notebook computers, digital broadcasting terminals, personal information terminals (PDA: Personal Digital Assistants), portable multimedia players (PMP: Portable Multimedia player, smart mirror, and/or smart TV.

In the present invention, an electronic device or a display unit and a user input unit installed in the electronic device are not denoted by separate reference numerals. However, the working area described in the present invention is output to the display unit of the electronic device, and user input is received via a user input unit located on or associated with the electronic device. is obvious to those skilled in the art.

The method of information retrieval using the information retrieval system described above will be described in more detail below. In particular, in the following, first, the information retrieval method will be described in connection with the flow chart.

FIG. 2 is a flow chart for explaining the information extraction method according to the present invention, FIG. 3 is a conceptual diagram for explaining an information extraction model according to the present invention, and FIG. FIG. 4 is a conceptual diagram showing how data output from an extraction model is treed;

In the information retrieval method according to the present invention, a step of performing OCR (Optical Character Recognition) on a target document to retrieve a plurality of words and position information about each of the plurality of words is performed (S110).

Here, the target document may be a paper document from which information is to be retrieved or an image of the paper document. As mentioned above, images 10 for paper documents can be received via various routes.

The OCR data may include text contained in the image and location information for the text.

The text contained in the OCR data may be segmented into a plurality of text segments according to pre-established criteria contained in the OCR data.

In one embodiment, text segments may be segmented into phrases, or multiple adjacent texts may be segmented into a single text segment based on text location information.

In the process of delimiting text segments, multiple text segments may be put together or one text segment may be separated into multiple text segments. For example, the text "105 h-street" taken from image 10 in FIG. -street”). As another example, "Happy store" taken from image 10 of FIG. may be separated into

The control unit 140 can group or separate text segments based on the positional information of the text. A model by which the control unit 140 assembles and separates text segments can be generated through machine learning, but is not limited to this.

Multiple words can be retrieved through the above process.

On the other hand, location information may be matched for each of the extracted words. Here, the positional information corresponding to the retrieved words may be positional information on the target document. Specifically, the position information may include two-dimensional coordinate information regarding the reference point, with a certain location on the image of the target document as a reference point.

As noted above, the location information may be relative coordinates, and distances between retrieved terms can be calculated using the location information.

Location information for each of the multiple terms can be used to determine relationships between the multiple terms. This will be described later.

A step of generating a plurality of tokens corresponding to each of the plurality of words is then performed (S120).

The plurality of tokens may be generated as any one of three types. First, among a plurality of token types, the first and second types will be explained.

The plurality of tokens may be of one of a first type of token having a categorical attribute and a second type of token having a data attribute corresponding to the categorical value.

Tokens corresponding to the first and second types can be generated based on the meaning of each of a plurality of terms retrieved from the target document.

In one embodiment, the plurality of terms are arranged in a first category having the category attribute based on the meaning of each of the terms and location information about each of the terms in the target document. A first type of term corresponding to the type and/or a second type of term having the data attribute.

The sorting process may be performed through an artificial intelligence model. The artificial intelligence model determines to which category each of the plurality of words belongs based on the plurality of categories already defined, and each of the plurality of words has an "attribute" (first type) or "value" (second type).

A first type token corresponding to each of the first type words classified into the first type among the plurality of words is generated, and a first type token corresponding to each of the plurality of words classified into the second type is generated. A second type of token is generated corresponding to each of the two types of words.

The first type of token is a token that defines an attribute of a category and may contain text indicating the category name (field name, category attribute). The text contained in the first type of token may be a category name contained in information relating to already stored categories. Here, the text contained in a particular first type token may not be the same as the first type words on which said particular first type token was generated.

For example, referring to FIG. 3, the first type token generated based on the word "name" of the first type does not contain "name" but the text "name". I'm in.

Said second type of tokens may be the words themselves extracted from the target document.

For example, referring to FIG. 3, the second type token corresponding to the second typed word "Happy" may be the text "Happy" itself.

On the other hand, certain first type tokens may be generated based on the semantics of second type words. Specifically, a first type token may be generated for each category to which the second type words classified as the second type belong among the plurality of words.

In one embodiment, if there is a specific second type word corresponding to the category value of a specific category among the plurality of words, and there is no word corresponding to the specific category, the specific category A particular first type token corresponding to the particular category may be generated based on the particular second type word corresponding to the category value of .

For example, referring to FIG. 3, the target document 10 does not have words corresponding to the attribute of the category of "store name/department name". According to the present invention, based on the meanings of the second type words "Happy" and "Store" 11 extracted from the target document 10, among the plurality of categories already defined, the category of "store name/department name" is defined. A first type of token can be generated that identifies and corresponds to the "store name/department name" category.

As noted above, the present invention identifies categories contained in a target document based on the substantive meaning of terms of the second type, even if there are no terms in the target document that indicate attributes of the category. Identify.

In one embodiment, referring to FIG. 3, multiple words (“Happy”, “store”, “105 h-street”, “order number:”, “20210520011”, “item name”, “quantity , "amount", "Carrot", "3", "2997", "tomato", "2", "1000") are retrieved.

On the other hand, some of the extracted words ("product name", "quantity", "price", "order number:") are sorted into words of the first type, and the remaining parts ("Happy", " store", "105 h-street", "20210520011", "Carrot", "3", "2997", "tomato", "2", "1000", "20210520011") are words of the second type may be sorted into

Subsequently, based on the respective meanings of the retrieved terms, some of the previously defined categories corresponding to each of the terms are identified. Specifically, based on the meaning of the words retrieved, "store", "name" (a subcategory of the category of "store"), "address", "item", "list", "menu", "name (a subcategory of the "menu" category), "count" and "price" categories may be specified.

Then, a first type corresponding to each of the categories "address", "item", "list", "menu", "name" (a subcategory of the category of "menu"), "count", and "price" are generated, and second type tokens are generated corresponding to the words sorted into the second type of words.

On the other hand, among the data to be retrieved from the target document, data in a category unnecessary for the user may be set to be deleted when the data is retrieved. The user may preset a category of data that is unnecessary for the user, and set such that a token for data in that category is not generated. Words belonging to the already set category (“order number:”, “20210520011”) may be deleted.

In one embodiment, referring to the table shown in FIG. 3, a plurality of tokens (output tokens) 320 may be generated based on a plurality of terms sorted into first and second types of terms. good. A plurality of tokens may be generated sequentially according to the flow of time (t).

The plurality of tokens may include tokens of a third type. A third type of token is described below.

A step of arranging the plurality of tokens to reflect the data structure between the plurality of words contained in the target document is then performed (S130).

A plurality of terms contained in the target document can have a predetermined data structure. A data structure is a structure that shows the characteristics of multiple words and the relationships between words from a logical point of view, and the types of data structures are lists, rows, trees, graphs, queues, and stacks. Although it may contain at least one of, it is not limited to this at all.

For example, the target document 10 described with reference to FIG. It has a data structure containing categories. However, the data structure corresponding to the target document 10 is not necessarily absolute, and the data structure regarding the target document 10 may vary depending on the method of defining the data structure.

In this specification, the step of arranging a plurality of tokens may be implemented so as to generate data of the same structure as the data structure between the plurality of words included in the target document. It may be implemented to generate data of a structure in which the data structure between multiple words is transformed according to a pre-established method.

For example, the target document 10 described with reference to FIG. Having a data structure containing categories, the step of arranging tokens according to the present invention includes the category of "store name/selling area name" and the category of "selling area", which is a higher category of the category of "selling area address". The category of "store name / sales area name" and "sales area address" category, which are subordinate concepts of the category, the "order product name" category, the "order quantity" category, and the "order amount" category that are related to each other Data having a data structure containing it may be implemented.

As described above, the present invention can define new superordinate categories of the categories included in the target document 10, or associate different categories of data with each other.

In the step of arranging the plurality of tokens, the plurality of tokens are arranged in order using the meanings of the plurality of terms and location information for each of the plurality of terms.

The step of arranging the plurality of tokens does not necessarily have to be performed after all of the plurality of tokens have been generated. The present invention may be carried out by a method in which a further token is generated after arranging the generated tokens in a state in which some of the plurality of tokens are generated.

In one embodiment, the present invention may be performed in such a manner that the generated tokens are ordered each time a token is generated.

The method for arranging multiple tokens will be described in detail below.

A first type token corresponding to a first category to which a specific second type word classified as a second type belongs, and a second type token corresponding to the specific second type word Tokens may be ordered.

In one embodiment, tokens of the first type and tokens of the second type corresponding to a first category of the same category may be ordered. In this specification, for ease of explanation, an embodiment in which tokens of the first type are ordered before tokens of the second type will be described, but is not limited thereto. type tokens may be ordered before the first type tokens.

In another embodiment, when tokens are arranged in a manner such that the generated tokens are arranged each time one token is generated, the tokens that are arranged earlier are arranged later. may be generated earlier than

On the other hand, there may be a plurality of specific second type words belonging to the first category. In this case, a plurality of second type tokens corresponding to each of the plurality of specific second type words are arranged in order.

At this time, the order in which the plurality of second type tokens arranged in order is determined based on the respective meanings of the plurality of specific second type words and positional information on the target document. may

For example, referring to FIG. 3, among the generated tokens 320, tokens of the second type, “Happy” tokens and “store” tokens, are ordered following the “[name]” token. and the "Happy" token is ordered first and the "store" token is ordered following the "Happy" token, based on the respective words of the two second type tokens.

As described above, the present invention minimizes the size of words extracted from a target document, so that even if the shape of the target document is distorted or partly lost, there will be a large gap between texts. To extract information with high accuracy based on the meaning of words even when relative positions are irregular.

On the other hand, tokens of the first type belonging to different categories are ordered. Specifically, a first type token corresponding to the first category and a first type token corresponding to a second category different from the first category are arranged in order.

At this time, between the first type token corresponding to the first category and the second type token corresponding to the second category, the first category and the second category A third type of token may be arranged for distinguishing tokens corresponding to each of .

In one embodiment, if there are only tokens of the first type among tokens belonging to the first category (when only terms corresponding to attributes are extracted from the target document and no terms corresponding to values are extracted), Tokens of the third type are ordered following the tokens of the first type.

In another embodiment, if the tokens belonging to the first category are a first type token and one second type token, the first type token corresponding to the first category is first Tokens of the first type may be ordered followed by tokens of the second type. Tokens of the third type are ordered following tokens of the second type.

In another embodiment, if the tokens belonging to a first category are a first type token and a plurality of second type tokens, the first type tokens corresponding to said first category are first , followed by a plurality of second type tokens corresponding to the specific second type words, and the specific Of the second type tokens corresponding to the second type words of , the third type token may be arranged following the specific second type token arranged last.

As described above, the third type token may be ordered last among at least one token belonging to a particular category.

Based on the meaning of each of the plurality of terms retrieved from the target document 10, a category corresponding to each of the plurality of terms may be identified among the plurality of previously defined categories. Among the already defined categories, if there is a category of a higher concept than the concept defined by the specified category, even if the category of the higher concept is specified as a category corresponding to each of the plurality of words good.

The broader categories described above may be understood to be supercategories of a particular category. In this specification, categories that are superordinate to a particular category are referred to as supercategories, and categories that are subordinate to the supercategory are referred to as subcategories. Since the phrases are relative phrases, a particular category may be used as a reference.

For example, if there is a category corresponding to the superordinate concept of the first category, the category corresponding to the superordinate concept may be identified as the third category. In this case, the first category may be referred to as a subcategory of the third category, and the third category may be referred to as a supercategory of the first category.

When the category of the broader concept than the category of the broader concept is already defined, the category of the broader concept than the category of the broader concept may also be specified as the category corresponding to each of the plurality of words.

If a supercategory of a particular category is identified, a first type of token corresponding to the supercategory may be generated. For example, tokens of the first type corresponding to the identified third category are generated. A first type token corresponding to the third category, a first type token corresponding to the first category and a second type token corresponding to the first category are ordered. may

On the other hand, multiple subcategories of a particular category may be specified. For example, the first and second categories may be subcategories of the third category. In this case, tokens of the first type corresponding to the first to third categories are generated. a first type token corresponding to said third category; a first type token corresponding to said first category; a second type token corresponding to said first category; Tokens, tokens of a first type corresponding to said second category and tokens of a second type corresponding to said second category may be ordered.

As mentioned above, between tokens of the first type corresponding to each of the two subcategories, tokens of the third type are juxtaposed.

Based on FIG. 3, the method of arranging tokens generated based on actual receipts will be described in more detail.

In FIG. 3, the explanation is given on the assumption that the token generation order and the token arrangement order are the same, but the token generation order and the token arrangement order do not necessarily match.

Based on the respective meanings of the words retrieved from the target document 10, "[store]", "[name]" (corresponding to subcategories of the category corresponding to the token of "[store]"), "[address] ” (corresponding to the subcategories of the category corresponding to the token of “[store]”), “[item]”, “[list]” (corresponding to the subcategories of the category corresponding to the token of “[item]”), "[menu]" (corresponding to the subcategory of the category corresponding to the token of "[list]"), "[name]" (corresponding to the subcategory of the category corresponding to the token of "[menu]"), "[ count]” (corresponding to a subcategory of the category corresponding to the token of “[menu]”) and “[price]” (corresponding to a subcategory of the category corresponding to the token of “[menu]”) are generated. be.

In addition, words classified into the second type ("Happy", "store", "105 h-street", "20210520011", "Carrot", "3", "2997", "tomato", "2" , “1000”) are generated.

Among the plurality of categories, the [store] token corresponding to one of the two categories "store" and "item" corresponding to the highest concept is arranged as the first token. Next, the [name] token corresponding to the "name" category corresponding to the subcategory of the "store" category is listed following the [store] token, and the second token belonging to the "name" category is listed. Tokens of "Happy" and "store" corresponding to words of the type are successively arranged after the token of [name].

After all the tokens 321a belonging to the category of "name" are arranged, the last token (token of "store") among the tokens corresponding to the category of "name" is followed by the third type. Tokens ([reduce]) 321b are arranged. Next, the [address] token corresponding to the "address" category corresponding to the subcategory of the "store" category is arranged following the [reduce] token.

Although not shown, the arrangement method will be described in detail below. Tokens of "105 h-street" corresponding to words of the second type belonging to the category of "address" are listed following the token of [address]. After all the tokens belonging to the category of 'address' are arranged, the last token (the token of '105 h-street') among the tokens corresponding to the category of 'address' is followed by the third type tokens are arranged.

On the other hand, since all tokens corresponding to the "store" category have been ordered, tokens of the third type are ordered one more time. That is, following the last token (“105 h-street” token) among the tokens corresponding to the “address” category, two third type tokens are consecutively arranged.

Following the two third-type tokens listed in succession, the [item] token corresponding to the "item" category is listed. Following the [item] token, the [list] token corresponding to the "list" category, which is a subcategory of the "item" category, corresponds to the "menu" category, which is a subcategory of the "list" category. Tokens of [menu] corresponding to the category of "menu" and tokens of [name] corresponding to the category of "name" which is a subcategory of the category of "menu" are successively arranged. The [name] token is followed by the "carrot" token corresponding to the value belonging to the "name" category, and the "carrot" token is followed by the [reduce] token. .

Next, the [count] tokens corresponding to the "count" category, which is a subcategory of the "menu" category, are successively arranged. The [count] token is followed by the "3" token corresponding to the value belonging to the "count" category, and the "3" token is followed by the [reduce] token. .

Next, the [price] tokens corresponding to the "price" category, which is a subcategory of the "menu" category, are successively arranged. Following the [price] token, "2997" tokens corresponding to values belonging to the "price" category are arranged, and "2997" tokens are arranged.

Since all the tokens corresponding to the "price" category, the "menu" category, and the "list" category are listed, following the "2997" token, three tokens of the third type are listed in succession. .

Then, three tokens of the third type listed in succession are followed by tokens of [list] corresponding to categories of "list" which are subcategories of the category of "item", subcategories of the category of "list". The [menu] token corresponding to the category "menu" and the [name] token corresponding to the subcategory "name" of the "menu" category are successively arranged. The [name] token is followed by the "tomato" token corresponding to the value belonging to the "name" category, and the "tomato" token is followed by the [reduce] token. .

Next, the [count] tokens corresponding to the "tomato" category, which is a subcategory of the "menu" category, are consecutively arranged. The [count] token is followed by the "2" token corresponding to the value belonging to the "count" category, and the "2" token is followed by the [reduce] token. .

Subsequently, tokens of [price] corresponding to the category of "price" which is a subcategory of the category of "menu" are continuously arranged. Following the [price] token, "1000" tokens corresponding to values belonging to the "price" category are arranged, and "1000" tokens are arranged. The arrangement of tokens ends with the "1000" token.

On the other hand, data reflecting the data structure between the plurality of words included in the target document may be generated based on the generated tokens and the order in which the tokens are arranged.

Data reflecting the data structure between the plurality of words included in the target document is generated by performing actions corresponding to the tokens in the order in which the tokens are arranged.

In one embodiment, data reflecting a data structure among the plurality of words included in the target document may be generated in a tree shape.

The tree data includes multiple nodes and multiple layers, each node corresponds to one of multiple categories, and at least one of attributes and values of the corresponding category is matched.

After setting the top node S as a reference node when generating tree data, the nodes are generated by performing actions corresponding to the tokens in the order in which the tokens are arranged, or the data corresponding to the nodes are generated. match.

The action for the first type token is to create a subnode of the base node and change the generated subnode to the base node, and the action for the second type token is to convert the base node to the second node. It matches words corresponding to tokens of the type as values. The action for a third type of token is to change the node one step above the reference node to the reference node.

Tree data may be generated from the ordered tokens according to the criteria described above.

An embodiment for generating tree data will be described with reference to FIGS. 3 and 4. FIG. The description of the method of generating and arranging tokens from the target document 10 shall be replaced with the above description.

Referring to FIG. 3, the tokens perform actions 330 corresponding to the tokens in order. The numbers included in the token generation time column 311 in the table shown in FIG. 3 may be the order of processing the tokens, and the actions included in the action column 331 are performed according to the token processing order. As a result, the parse 340 is performed, and the data included in the parse string 341 are generated in order.

First, the top node S of the tree is set as a reference node, and the action NT (store) corresponding to the [store] token is performed at the reference node. As a result, a node 410 of [store], which is a lower node of the reference node S, is generated, and the reference node is changed to the node 410 of [store].

Next, the action NT (name) corresponding to the token of [name] is performed, the node 420 of [name], which is a lower node of the node 410 of [store], is generated, and the reference node is the node 420 of [name]. Be changed.

Next, an action GEN(Happy) is performed to match the word "Happy" corresponding to the token "Happy" to the [name] node 420 . Subsequently, the action GEN(store) is performed to match the word "store" corresponding to the token "store" to the node 420 of [name]. The terms matched to the [name] node 420 are serialized in the order in which the terms were matched. As a result, "Happy store" 421 is matched to the [name] node 420 .

As all the actions contained in the area 331a of the table shown in FIG. 3 are performed, nodes 410 and 420 in FIG. 4 are generated and the text corresponding to the 420 node is matched.

Next, the action corresponding to the [reduce] token included in the area 331b of the table shown in FIG. 3 is performed. As a result, the reference node is changed from the [name] node to the [store] node, which is the upper node of the [name] node.

Next, the action NT (address) corresponding to the token of [address] is performed, the node of [address] which is a lower node of the node 410 of [store] is generated, and the reference node is changed to the node of [address]. be.

Next, the action GEN (105 h-street) is performed to match the word "105 h-street" corresponding to the token of "105 h-street" to the node of [address]. As a result, "105 h-street" is matched to the [address] node.

Then, the action corresponding to the two [reduce] tokens arranged in succession is performed, and the reference node is changed to the top node S.

Next, the actions corresponding to the [item] token, the [list] token, the [menu] token, and the [name] token are performed in order, and the [item] node 430, [list] node 440, [ menu] node 450 and [name] node 460 are generated in order. Subsequently, the word "carrot" 461 is matched to the [name] node 460 by performing an action corresponding to the "carrot" token.

When actions are repeatedly performed on tokens generated according to the above rules, the tree data shown in FIG. 4 can be generated.

The tree shown in FIG. 4 is data in which the key-value pairs for each of the words contained in the target document 10 are completely matched, and the categories of each data are completely defined.

As described above, the present invention uses the meanings of the plurality of terms and the location information for each of the plurality of terms extracted from the target document, and uses the generated tokens and the sequence of generated tokens. , it is possible to generate data that reflects the data structure between multiple words contained in the target document.

On the other hand, the present invention judges the accuracy of the information retrieval model based on the above tree data when performing machine learning on the information retrieval model.

FIG. 5 is a conceptual diagram showing the machine learning method of the information retrieval model according to the present invention, and FIG. 6 is a conceptual diagram showing how the accuracy of data output from the information retrieval model according to the present invention is calculated. be.

Referring to FIG. 5, after performing OCR 511 on the target document 10 to estimate a plurality of terms and location information 512 for each of the plurality of terms, the terms and location information are processed by the information retrieval model according to the present invention. Enter 513. This causes the ordered tokens to be ordered.

On the other hand, data corresponding to the target document 10 is processed into learning data. Processing for learning data is performed by a parse annotator 521, which outputs the already stored data in the form of a plurality of tokens arranged in order. That is, the parse annotator 521 outputs data in the same format as the output data of the machine learning model according to the present invention. Through this, original data (learning data) for machine learning is generated.

An error between tree data generated based on a plurality of tokens output from the information extraction model according to the present invention and the original data may be calculated, and training may be performed to minimize the error.

In one embodiment, referring to FIG. 6, the error of the information retrieval model may be calculated by comparing the nodes included in the tree data generated from the information retrieval model with the nodes that make up the original data. good. Specifically, the error of the information retrieval model is calculated while changing the tree data output from the information retrieval model (the topmost tree in FIG. 6) in the same manner as the original data tree (the bottommost tree in FIG. 6). may

Specifically, whether or not each node 610, 620, 630 of the tree data output from the information extraction model (the topmost tree in FIG. 6) and the values 621, 631 matched to the node match the tree of the original data. unmatched nodes 630 and unmatched values 631 are modified in the same manner as the original data so that unmatched nodes 630 are modified to nodes 630′ that match the original data, and unmatched values 631 are modified to the original data. Let it be modified to a value 631' that matches the data. In this process, the score of the tree data output from the information retrieval model is calculated. Since the score calculation method utilizes an already known method, a detailed description thereof will be omitted.

As described above, the machine learning of the information retrieval model according to the present invention is performed by calculating the accuracy of the ordered tokens output from the information retrieval model. Since the output tokens are not data in which the "key-value" pairs are perfectly matched, learning data for calculating the accuracy of the information retrieval model is likely to be generated.

Specifically, after generating tokens arranged in a certain order according to rules (token generation rules) that have already been established when generating learning data, the tokens output from the information retrieval model are compared. , the accuracy of an information retrieval model can be determined, and machine learning can be performed to improve the accuracy of the information retrieval model. This greatly reduces the time it takes to generate learning data for machine learning of information retrieval models.

On the other hand, the present invention described above can be implemented as a program that is executed by one or more processes in a computer and can be stored in such a computer-readable medium (or recording medium).

Furthermore, the present invention described above can be implemented as computer-readable codes or commands on a program-recorded medium. That is, the present invention may be provided in the form of a program.

A computer-readable medium, on the other hand, encompasses any kind of recording device on which data that is read by a computer system is stored. Examples of computer readable media include hard disk drives (HDD), solid state disks (SSD), silicon disk drives (SDD), read-only memory (ROM), Random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

Furthermore, the computer-readable medium may be a server or cloud storage that includes the storage and that the electronic device can access through communication. In this case, the computer can download the program according to the present invention from a server or cloud storage via wired or wireless communication.

In addition to these, in the present invention, the computer described above is an electronic device equipped with a processor, that is, a central processing unit (CPU: Central Processing Unit). do not do.

However, the above detailed description should not be construed as restrictive in any respect, but rather should be considered illustrative. The scope of the invention should be determined by reasonable interpretation of the claims, and all changes that come within the equivalent scope of the invention are included in the scope of the invention.

Claims (16)

A method of retrieving information from a semi-structured document, comprising:
performing optical character recognition (OCR) on a target document to retrieve a plurality of terms and location information for each of the plurality of terms;
generating a plurality of tokens corresponding to each of the plurality of terms;
arranging the plurality of tokens to reflect a data structure between the plurality of terms contained in the target document;
including
In the step of arranging the plurality of tokens,
aligning the plurality of tokens using the meanings of the plurality of words and location information for each of the plurality of words;
A method characterized by: the plurality of tokens are of one of a first type of token having a category attribute and a second type of token having a data attribute corresponding to the category value;
A method according to claim 1, characterized in that: generating a plurality of tokens corresponding to each of the plurality of terms,
generating at least one of a first type of token having the categorical attribute and a second type of token having the data attribute based on the meaning of each of the plurality of terms;
3. A method according to claim 2, characterized in that: The method further comprises:
based on the meaning of each of the plurality of terms and the positional information about each of the plurality of terms in the target document, the plurality of terms to a first type of term corresponding to the first type having the category attribute and sorting into at least one of a second type of terms having said data attribute;
4. A method according to claim 2 or 3, characterized in that: In generating a plurality of tokens corresponding to each of the plurality of terms,
generating at least one first type token corresponding to each of the first type words sorted into the first type among the plurality of words;
generating at least one second type token to respectively correspond to the second type words sorted into the second type;
5. A method according to claim 4, characterized in that: In generating a plurality of tokens corresponding to each of the plurality of terms,
generating first type tokens for each of the categories to which the second type words classified into the second type belong;
6. A method according to claim 5, characterized in that: generating a plurality of tokens corresponding to each of the plurality of terms,
if there is a specific second type word corresponding to a category value of a specific category in the plurality of words and there is no word corresponding to the specific category,
generating tokens of a particular first type corresponding to the particular category based on words of the particular second type corresponding to category values of the particular category;
further comprising
7. A method according to claim 6, characterized in that: In the step of arranging the plurality of tokens,
A first type token corresponding to a first category to which the specific second type words classified into the second type belong, and a second type corresponding to the specific second type words. , and the tokens of
8. A method according to claim 6 or 7, characterized in that In the step of arranging the plurality of tokens,
when there is a plurality of words of the specific second type,
sequencing a plurality of second type tokens corresponding to each of the plurality of specific second type words;
The order of the plurality of second type tokens arranged in order is determined based on the meaning of each of the plurality of specific second type words and position information on the target document;
9. A method according to claim 8, characterized in that: In the step of arranging the plurality of tokens,
Sequence a first type token corresponding to the first category and a first type token corresponding to a second category different from the first category;
Between the tokens of the first type corresponding to the first category and the tokens of the second type corresponding to the second category, each of the first category and the second category has a third type of token is ordered to distinguish the corresponding tokens;
9. A method according to claim 8, characterized in that: In the step of arranging the plurality of tokens,
a first type of token corresponding to said first category among a first type of token corresponding to said first category and a second type of token corresponding to said specific second type of words; first, and
sequencing a first type token corresponding to the first category followed by a second type token corresponding to the specific second type word;
Of the second type tokens corresponding to the specific second type words, arranging the third type tokens following the specific second type token arranged last;
11. A method according to claim 10, characterized in that: The method further comprises:
generating data having a structure corresponding to the target document using the plurality of tokens;
In the step of generating the data,
when the third type of token is detected among the plurality of tokens,
A specific token detected immediately before the third type token is detected and a specific token following the third type token are regarded as tokens corresponding to different categories from each other,
12. A method according to claim 11, characterized in that: In generating a plurality of tokens corresponding to each of the plurality of terms,
When there is a category corresponding to the superordinate concept of the first category,
identifying a category corresponding to the broader concept as a third category, and generating the first type token corresponding to the identified third category;
13. A method according to any one of claims 8 to 12, characterized in that: A first type token corresponding to the third category, a first type token corresponding to the first category, and a second type token corresponding to the first category are arranged in order. to be
14. A method according to claim 13, characterized in that: A system for retrieving information from a semi-structured document, comprising:
an OCR unit that performs optical character recognition (OCR) on a target document to retrieve a plurality of terms and location information for each of the plurality of terms;
a control unit that generates a plurality of tokens corresponding to each of the plurality of terms and arranges the plurality of tokens so as to reflect a data structure between the plurality of terms included in the target document;
with
The control unit
aligning the plurality of tokens using the meanings of the plurality of words and location information for each of the plurality of words;
A system characterized by: A computer program running on a computer and comprising a plurality of instructions,
When said instructions are executed by said computer,
performing optical character recognition (OCR) on a target document to retrieve a plurality of terms and location information for each of the plurality of terms;
generating a plurality of tokens corresponding to each of the plurality of terms;
arranging the plurality of tokens to reflect a data structure between the plurality of terms contained in the target document;
and
In the step of arranging the plurality of tokens,
aligning the plurality of tokens using the meanings of the plurality of words and location information for each of the plurality of words;
A computer program characterized by:

Images