JP7367139B2 - Data search method and system - Google Patents

Description

The present invention relates to a data search method and system for searching data including field values of different categories.

The dictionary meaning of artificial intelligence is a technology that realizes human learning, reasoning, perceptual, and natural language understanding abilities through computer programs. This kind of artificial intelligence has developed dramatically through deep learning, which combines machine learning with neural networks that imitate the human brain.

Deep learning is a technology that enables computers to judge and learn like humans, thereby grouping or classifying objects and data.In recent years, it has become possible to analyze not only text data but also image data. It is now actively used in a wide variety of industrial fields.

With the development of artificial intelligence, various types of automation are being carried out in the field of office automation. In particular, in the field of office automation, much effort is being made to convert content printed on paper into data based on image data analysis technology that utilizes artificial intelligence. As part of this, in the office automation field, the content contained in documents is converted into data using image analysis technology (or image data analysis technology) that converts paper documents into images and analyzes the content contained in the images. , techniques are needed to analyze images according to the type of content contained in the document.

For example, when converting a document containing a receipt into data, the accuracy of various elements related to the receipt, such as the format of the receipt, the content of the text included in the receipt, and the position of the text included in the receipt, is required. A thorough analysis is required.

Therefore, various techniques have been developed for processing information contained in images into data that can be processed by electronic devices. For example, Patent Document 1 discloses a method for constructing an OCR (Optical Character Reader) database, but the methods developed so far are at the level of classifying data according to rules determined empirically by humans. Therefore, if there is an error in the OCR data, not only will an inaccurate database be constructed, but searches using the database may not be performed smoothly.

On the other hand, various search services have been provided in recent years. For example, a service is provided that uses a receipt to search for a place where the receipt was used. Therefore, there is an increasing need for technology that searches for data corresponding to content in various formats such as images, audio, and text.

Conventionally, in order to correct OCR recognition errors, preprocessing techniques using regular expressions and the like have been relied upon. Such a method requires a lot of time and cost, and has problems in that the correction performance is not high.

Additionally, in the case of content that includes multiple field values, multiple field information must be used in the search to obtain the desired result, which typically relies on a heuristic, rule-based model. In particular, there has been a problem in that search performance varies greatly depending on which field is selected, or desired search results cannot be obtained.

Therefore, there is a need for a technique that allows desired search results to be obtained even when text is erroneously recognized due to an OCR recognition error or the like.

Korean Registered Patent No. 10-1181209

The present invention provides a method and system for embedding data including field values belonging to different categories into data in a form that can be used in an electronic device, and searching the data based on the embedding results.

Specifically, the present invention provides a method and system for embedding data in a form that can be utilized in electronic devices while maintaining characteristics of different categories contained in the data.

The present invention also provides a method and system that allows data corresponding to data containing different field values to be easily retrieved from a database.

Furthermore, the present invention provides a method and system that allows desired results to be retrieved from a database with high accuracy even when retrieving data corresponding to data containing error values due to text recognition errors, voice recognition errors, etc. It provides:

In order to solve the above problems, the present invention includes the steps of receiving content including a plurality of field values, arranging the field values included in the content, and providing a plurality of classifiers for classifying categories to which the field values belong. generating a model input value by adding a model input value; generating a vector of the content using the model input value and the learned deep learning model; and searching for data corresponding to the search target data from the plurality of already stored data based on the degree of similarity between vectors corresponding to each of the vectors.

Further, the present invention includes a communication unit that receives content including a plurality of field values, and a model that arranges the field values included in the content, but adds a plurality of classifiers that classify the categories to which the field values belong. Generate an input value, generate a vector of the content using the model input value and the learned deep learning model, and determine the similarity between the generated vector and the vector corresponding to each of the plurality of data already stored. and a control unit that searches for data corresponding to the search target data from the plurality of already stored data based on the search target data.

As described above, the present invention embeds data by classifying the categories to which multiple field values included in the data belong, so it is possible to generate vectors that maintain the characteristics of different categories included in the data. . By utilizing generated vectors that maintain characteristics of different categories for data retrieval, the present invention is not limited to retrieving only the same data as already stored data, but can search multiple data contained in a target document. Data searches can be performed taking into account the similarity of values belonging to categories.

Further, according to the present invention, data is searched based on the degree of similarity between vectors generated as a result of embedding, so there is no need to perform a search depending on a search rule defined by a person when searching for data.

Furthermore, according to the present invention, it is possible to perform a search that takes into account the degree of similarity for each category of data, so it is possible to search data that frequently contains noise or errors (for example, character recognition data (OCR data), voice recognition data). Data retrieval can be performed with high accuracy even when using this method.

1 is a conceptual diagram for explaining a data search system according to the present invention. 1 is a conceptual diagram showing a data search method according to the present invention. 3 is a flowchart for explaining a data search method according to the present invention. FIG. 2 is a conceptual diagram for explaining a data embedding model according to the present invention. FIG. 3 is a conceptual diagram showing a state in which vectors generated as a result of data embedding according to the present invention are floating in a vector space. FIG. 2 is a conceptual diagram illustrating an embodiment of searching data using OCR data. FIG. 2 is a conceptual diagram illustrating an embodiment of searching data using OCR data. FIG. 2 is a conceptual diagram illustrating an embodiment in which data is searched using a vector generated as a result of data embedding.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components will be denoted by the same reference numerals regardless of the drawing numbers, and redundant explanation thereof will be omitted. The suffixes "module" and "part" used in the following explanations are given or mixed to facilitate the preparation of specifications, and they themselves have significance and usefulness. It's not something you have. Furthermore, when describing the embodiments of the present invention, if it is determined that detailed explanation of related known techniques would obscure the gist of the embodiments of the present invention, the detailed explanation will be omitted. Furthermore, the attached drawings are only for helping understanding of the embodiments of the present invention, and the technical idea of the present invention is not limited by the attached drawings, and any changes that fall within the idea and technical scope of the present invention may be made. , should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as "first", "second", etc. are used to describe various components, but the components are not limited to the above terms. These terms are only used to distinguish one component from another.

When a component is referred to as being "coupled" or "connected" to another component, it may be directly coupled or connected to the other component, and there may be other components in between. It should be understood that it is permissible to do so. In contrast, when an element is referred to as being "directly coupled" or "directly connected" to another element, there are no intermediate elements present.

Singular expressions include plural expressions unless otherwise specified.

As used herein, terms such as "comprising" and "having" are intended to specify the presence of features, numbers, steps, acts, components, parts, or combinations thereof described herein. It should be understood that this does not exclude in advance the presence or possibility of adding one or more other features, figures, steps, acts, components, parts or combinations thereof.

In this specification, content refers to various types of information and contents that can be processed by a computer, and is in various forms such as text, images, sounds, files, etc., and is not limited to a specific form.

A category refers to an item at any level within a defined classification. The criteria for distinguishing data belonging to a particular category from data belonging to other categories is not absolute, but depends on arbitrary rules that define the categories. These rules are applied differently to original content (for example, a paper document, an image of a paper document, audio data) and structured data thereof.

For example, documents such as various types of receipts contain multiple categories related to sellers and consumers, such as store name, business number, store phone number, store address, order product name, order product quantity, etc. For efficient processing, it is necessary to convert the data contained in paper documents into data by associating the same categories.

For example, a specific receipt, which is a paper document, includes three categories: "store name," "ordered product name," and "ordered product quantity," but when digitizing the data contained in the paper document, It may be reduced to two categories: "store name" and "ordered product".

On the other hand, in this specification, data indicating attributes of a specific category is defined as a field name (for example, "Department name:", "Quantity:", "Telephone number", etc.), and data indicating the value of a specific category is defined as Define data as field values (eg, "NLP CAFE", "S City", "01-234-568", etc.).

On the other hand, each piece of content may include a field name and a field value, regardless of the type of content. For example, each of a paper document, an image of a paper document, and audio data may include a field name and a field value. Therefore, field names and field values have various forms such as text, images, and audio data.

On the other hand, from the perspective of data in a machine-processable format, the field name is called an "attribute" and the field value is called a "value."

According to the definition of the term above, content may include data of "attribute-value" pairs included in the same category. However, the present invention is not limited to this, and the content may not include field names but only field values for a specific category. In this case, the field name is simply omitted, and the field value corresponding to the omitted field name includes a meaning related to the field name.

On the other hand, the content may include only field names without field values. In this case, there is no value assigned to a specific item, and even if there is no value assigned to a specific item, the specific item may exist.

In the above example, a paper document (or an image of a paper document) is converted into text that can be processed by an electronic device through character recognition, and the converted text is classified into different categories. Therefore, data containing field values belonging to different categories is generated.

On the other hand, data including field values belonging to different categories can be generated based on text collected not only by OCR but also by other methods. For example, a speech recognition product can generate data that includes field values belonging to different categories. Specifically, the user's voice recognized by voice recognition is converted into text, and the converted text is classified into different categories and computerized.

As mentioned above, data containing field values belonging to different categories can be generated in a variety of ways. The present invention provides a method and system for retrieving data corresponding to the generated data from an already stored database.

Meanwhile, data search according to the present invention is performed based on a vector generated as a result of embedding data in the content.

The present invention provides a method for efficiently searching for data containing field values belonging to different categories. Specifically, the present invention improves the accuracy of data retrieval through efficient embedding that converts data containing field values belonging to different categories into information in a machine-understandable form.

The present invention provides a data retrieval method and system that improves the accuracy of data retrieval using vectors generated by a new method of data embedding.

The new method of data embedding will be specifically described below with reference to the accompanying drawings.

FIG. 1 is a conceptual diagram for explaining a data retrieval system according to the present invention, and FIG. 2 is a conceptual diagram for explaining a data retrieval method according to the present invention.

The data retrieval system 100 according to the present invention may be implemented in the form of an application or software. According to the software implementation of the data retrieval system 100 according to the present invention, embodiments such as processes and functions described herein may be implemented in separate software modules. Each of the software modules may perform one or more functions and operations described herein.

A software implementation of the present invention is realized by a data retrieval system 100 shown in FIG. The configuration of the data search system 100 will be described in more detail below.

Data retrieval system 100 according to the present invention can receive content that includes multiple field values. The received content is converted into data in a format required by the data search system 100.

For example, the data retrieval system 100 according to the present invention can receive an image of a paper document and generate OCR data by performing text recognition on the image. As used herein, OCR data may include text extracted from an image and location information corresponding to the extracted text. Here, the position information defines the position of the extracted text within the image (or paper document). The data retrieval system 100 according to the present invention can classify the extracted text into different categories.

As another example, the data retrieval system 100 according to the present invention can receive audio data, convert the audio data to text, and then classify the converted text into different categories.

Meanwhile, the data search system 100 according to the present invention includes at least one of a communication unit 110, a storage unit 120, an OCR unit 130, a control unit 140, and a voice recognition unit 150, as shown in FIG. However, the data retrieval system 100 according to the present invention may include more or fewer components than the above-mentioned components, and at least some of the above-mentioned components are located at physically separated locations. may be placed.

First, the communication unit 110 is a means for receiving an image 10 obtained by scanning a paper document or audio data, and may include at least one of a communication unit, a scanning unit, and an input unit, and receives other images 10. It may also consist of a means.

The data search system 100 can receive content such as the image 10 or audio data received via the communication unit 110, and can embed data in the content.

Next, the storage unit 120 may store various information according to the present invention. The storage unit 120 has a wide variety of types, and at least a portion thereof may include a DB (database) 160. The DB 160 may be an external server or a cloud server physically separated from the data search system 100, and the data search system 100 can utilize the DB 160 like the storage unit 120 by communicating with the DB 160.

That is, the storage unit 120 may be any space in which information related to the present invention is stored, and there is no physical space restriction. In this specification, the storage unit 120 and the DB 160 are not distinguished, and data stored in the DB 160 will also be described as data stored in the storage unit 120.

The storage unit 120 stores i) data used for content generation (image 10 obtained by scanning a paper document or audio data) and related data, ii) learning data used for machine learning of the data embedding model, and iii. ) at least one of the embedded data is saved.

Next, the OCR unit 130 is a means for recognizing the text included in the image 10, and is capable of recognizing the text included in the image 10 using at least one of various text recognition algorithms. The OCR unit 130 can recognize text using an algorithm based on artificial intelligence.

The OCR unit 130 can extract text included in the image 10 and location information of the text. Here, the text position information includes information regarding the position of the text within the image 10.

Next, the control unit 140 controls the overall operation of the data search system 100 related to the present invention. The control unit 140 may include a processor (or an artificial intelligence processor) that processes an artificial intelligence algorithm.

Further, the control unit 140 provides a work area for data embedding, and such a work area is a "user environment" for performing data embedding or machine learning for data embedding. It is also referred to as a "user interface."

Such a work area may be output (or provided) on a display unit of an electronic device. Furthermore, the control unit 140 performs data embedding based on a user input received via a user input unit (for example, a touch screen, a mouse, etc.) included in the electronic device or in conjunction with the electronic device; Or machine learning for data embedding can be performed. Further, the control unit 140 can receive the content 10 and search data corresponding to the received content 10 from the data 240a, 240b stored in the storage unit 120 and other data stored in the storage unit 120. .

Note that in the present invention, there is no particular restriction on the type of electronic device to which the work area is output, as long as it can start the application according to the present invention. For example, electronic devices include smartphones, mobile phones, tablet PCs, computers, notebook computers, digital broadcast terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), smart mirrors, and smart TVs ( smart TV).

In the present invention, no reference numerals are given to the electronic device or the display unit and user input unit provided in the electronic device. However, it is obvious to those skilled in the art that the work area in the present invention is output to a display unit of an electronic device, and that user input is received via a user input unit provided in the electronic device or interlocked with the electronic device. be.

Meanwhile, the data search system according to the present invention can search for data corresponding to content from already stored data.

Hereinafter, as an example of using the data search system according to the present invention, a process of receiving an image 10 of a paper document and searching data corresponding to the image from already stored data will be described. As mentioned above, the received content is not limited to images.

As shown in FIG. 2, the data retrieval system according to the present invention receives an image 10 of a paper document and extracts 220 text and text location information from the image 10 through text recognition. Here, characters in the text may not be accurately recognized due to damage 210 to the paper document, and such a case is referred to as noise 221. Thereafter, the data retrieval system classifies the extracted text 220 into each of a plurality of different categories and generates structured data 230. Here, structured data is expressed in a predetermined format (eg, JSON, XML).

The data retrieval system 100 uses this to retrieve data 240a corresponding to the image 10 from a plurality of previously stored data 240a, 240b and other data stored in the storage unit 120.

Here, the present invention embeds content including a plurality of field values belonging to different categories and utilizes it for data search. The method of embedding data into content will be explained in more detail below.

Hereinafter, a method for embedding data using the data search system described above will be explained in more detail. In particular, below, a data embedding method will first be described with reference to a flowchart.

FIG. 3 is a flowchart for explaining the data retrieval method according to the present invention, FIG. 4 is a conceptual diagram for explaining the data embedding model according to the present invention, and FIG. 5 is a flow chart for explaining the data embedding model according to the present invention. FIG. 3 is a conceptual diagram showing a state in which the vectors obtained by the calculation are floated in a vector space.

In the data search method according to the present invention, a step of receiving content is performed (S110).

The content may include a plurality of field values, and the field values may each correspond to a plurality of different categories. As described above, the content is data in which field value categories are classified, or is processed into such a format. For example, a plurality of field values and their categories are expressed in a predetermined format (eg, JSON, XML, etc.).

That is, the data retrieval system according to the present invention receives data in a form in which field value categories are classified from the outside, or receives original data (image or audio data of a paper document), and then performs field search based on the original data. Data divided by name and field value can be generated and used for searching.

Next, a step of arranging the field values included in the content and adding a plurality of classifiers for dividing categories of the field values based on the category to which the field values belong to generate model input values is performed. (S120).

The data retrieval system 100 according to the present invention sequentially arranges a plurality of field values to generate input values for a deep learning model for data embedding. Here, the data retrieval system 100 can generate model input values such that field values belonging to different categories are classified by using a classifier that classifies categories to which each of the plurality of field values belongs. In addition to the category delimiter for classifying categories, a delimiter indicating the start or end of data input, a delimiter indicating the absence of a corresponding field value, etc. may also be used.

For example, the data search system 100 generates one piece of data by concatenating field values included in content in a predetermined order, and arranges delimiters at the front or rear of the field values. Therefore, the model input value may be data in which a plurality of field values and a plurality of delimiters are arranged in a line in a predetermined order.

On the other hand, the field values correspond to a plurality of categories, and the plurality of categorization elements added to the model input value correspond to the plurality of categories, respectively. That is, the model input values include field values and classifiers corresponding to each of the plurality of categories.

Here, field values and classifiers belonging to the same category are arranged adjacent to each other. That is, a specific category element and a specific field value corresponding to a specific category among the plurality of categories are arranged adjacent to each other. In this specification, when referring to either a field value or a division molecule belonging to the same category, the field value and the division molecule corresponding to the field value will be referred to as a field value corresponding to the division molecule.

A particular section molecule corresponding to a particular field value may be arranged before or after the particular field value. Therefore, some partition elements included in the model input value may be arranged before or after the model input value, or may be arranged between different field values.

Referring to FIG. 2, for the input of image 10, model input values are generated as "[CLS]NLP COFFEE[SEP_Name]S City[SEP_Address]". Here, "[]" (including the text inside "[]") is a delimiter that plays a specific role or a delimiter that separates field values belonging to different categories, and is not delimited by "[]". The text indicates field values included in the content. In the example, [CLS] is a class molecule representing the entire data, [SEP_Name] is a category molecule indicating the end of the name field value, and [SEP_Address] is a category molecule indicating the end of the address field value. .

On the other hand, if the content includes a field name that defines a specific category and does not include a field value corresponding to the specific category, the model input value is a special classifier ( mask). The mask may be arranged adjacent to a section molecule corresponding to the particular category.

If there is no field value, the model input value generated from the specific content may be constructed by arranging a mask instead at the position where the field value corresponding to the corresponding category should be arranged. For example, certain content contains a field name that defines "Business Registration Number", but the field value is represented by a corresponding mask ([MASK_biz]), and the model input value is "[CLS]NLP CAFE". [SEP_name] S city [MASK_biz] [SEP_biz]".

Next, a step of generating a vector of the content using the model input value and the learned deep learning model is performed (S130).

Here, the step of converting each of the field values included in the model input value into at least one first type token, and the step of converting each of the plurality of delimiters into a second type token may be performed. good.

The data retrieval system 100 converts first and second type tokens from field values and delimiters corresponding to a specific category so that the first type tokens corresponding to each of the plurality of categories are classified from each other. Arrange adjacent to each other.

Here, one or more first type tokens corresponding to one field value may be generated.

In one embodiment, multiple first type tokens may be generated from a single field value. A field value consisting of one word or a plurality of words may be divided into a plurality of texts in the token conversion process, and an already set text may be combined with at least a portion of the divided text. For example, the field value "NLP COFFEE" included in the model input value is converted into a plurality of first type tokens ("NL", "#P", "COFF", "#EE"). Here, the text "#" included in the first type token is text that defines that there is no blank space between it and the previous token, and is to be combined with a part of the text divided from the field value. It's okay.

The first type tokens corresponding to a particular field value among the field values may be composed of one or more, and the plurality of first type tokens corresponding to the particular field value are adjacent to each other. May be arranged. For example, the plurality of first type tokens ("NL", "#P", "COFF", "#EE") generated from the field value "NLP COFFEE" may be arranged in sequence.

On the other hand, the second type token may be one converted from each of a plurality of delimiters included in the model input value.

In one embodiment, the second type token is adapted to distinguish field values belonging to different categories, but the second type token itself may be of a form that does not include a specific meaning. For example, the second type token may be in the form of [SEP1], [SEP2], and [SEP3].

In another embodiment, each of the second type tokens may include a value indicating an attribute of the category corresponding to the second type token. Specifically, the plurality of second type tokens each include text indicating the plurality of categories, and the text included in any one of the second type tokens is different from the text included in the other one. They may be different. For example, the second type token may include field names of a particular category, such as [SEP_Name], [SEP_Address].

On the other hand, when there are multiple first type tokens converted from field values belonging to a specific category, the second type token corresponding to the specific category is arranged first among the plurality of first type tokens. The first type tokens may be arranged at the front of the first type token or at the rear of the first type token arranged last among the plurality of first type tokens.

For example, the first and second type tokens converted from the model input value "[CLS]NLP_COFFEE[SEP_Name]S City[SEP_Address]" are "[CLS]/NL/#P/COFF/#EE/[SEP1]". /S/Ci/#ty/[SEP2]". (The "/" is simply a display for distinguishing tokens.) Note that when the model input value includes a mask, the mask token is arranged adjacent to the second type token corresponding to the mask token.

The arranged first and second type tokens and mask tokens are input to a trained deep learning model to generate a vector corresponding to the content.

As a deep learning model for embedding data, a model that can be used when embedding a sequence, specifically, an RNN or a Transformer type model (for example, BERT) can be used.

The trained deep learning model produces a vector of structured data containing field values belonging to different categories. Specifically, the trained deep learning model generates vectors for each of the data stored in the storage unit 120 and generates a vector for the received content. That is, the content and already stored data to be searched using the content are vectorized.

An embodiment of generating content vectors with reference to FIG. 5 will be described. For example, the control unit 140 performs OCR 520 on an image 510 of a target document to generate OCR data, and separates the content into different categories from the OCR data. Structured data 530 including a plurality of field values belonging to each other is generated, and a vector of content 530 is generated by data embedding 540.

Meanwhile, already stored data is also vectorized using the learned deep learning model. Vectors 551a to 553a, 551b to 554b, and 551c to 554c corresponding to the previously saved data are shown on the vector plane. When generating a vector of already stored data, the steps of converting already stored data into structured data (eg, 510 and 520) are omitted.

In FIG. 5, for convenience of explanation, vectors generated by data embedding are shown two-dimensionally, but vectors generated by data embedding may have dimensions larger than two dimensions.

On the other hand, in FIG. 5, for convenience of explanation, the data to which data will be embedded will be explained as containing only two types of categories (name, tel), but the data to which data will be embedded will include a larger number of categories. May contain fields.

As shown in the figure, some of the floating vectors 551a to 553a are arranged adjacent to each other in the first region 550a. Note that some of the other vectors 551b to 554b are arranged adjacent to each other in the second region 550b. Still other vectors 551c to 554c are arranged adjacent to each other within the third region 550c.

Deep learning models are trained to vary the distance between vectors depending on the degree of similarity between the data. Specifically, the deep learning model is trained such that the higher the similarity between data, the closer the data is placed, and the lower the similarity between the data, the farther the data is placed.

A vector generated as a result of embedding data in the content 530 generated based on the OCR data is floated on the first area 550a.

As described above, the present invention embeds data by classifying the categories to which multiple field values included in the data belong, so it is possible to generate vectors that maintain the characteristics of different categories included in the data. .

Furthermore, by utilizing the generated vectors in which characteristics of different categories are maintained for data retrieval, the present invention is not limited to retrieving only the same data as already stored data, but also the data included in the target document. It is possible to perform data searches that take into account the similarity of values belonging to multiple categories. Hereinafter, a data search using the vectors will be specifically explained.

6a and 6b are conceptual diagrams showing an embodiment of searching data using OCR data, and FIG. 7 shows an embodiment of searching data using a vector generated as a result of data embedding. It is a conceptual diagram.

A vector of the content is generated using deep learning, and then a vector is generated from the plurality of already stored data based on the similarity between the generated vector and the vector corresponding to each of the plurality of already stored data. A step of searching for data corresponding to the search target data is performed (S140).

In the aforementioned work area, an interface screen for searching data using images can be displayed.

Referring to FIGS. 6a and 6b, search target content, such as an image 600 of a receipt, is output in the work area. The image 600 is either an already stored image, an image taken by a camera built into the electronic device displaying the work area, or an image from another device other than the electronic device displaying the work area. It may also be a received image.

Meanwhile, text extracted as a result of OCR on the image 600 can be displayed in the work area. For data classified into field names among the text extracted as a result of OCR, the extracted text may not be displayed as is, but already saved text may be displayed. It may be displayed on the work area even if it does not exist in the extracted text.

For example, referring to FIG. 6a, image 600 includes a category related to store names, but does not include field names related to the category. The control unit 140 determines that a category related to the store name exists in the image 600 based on the meaning of the field value "HLP Coffee", and displays the field name ("name" 611) already saved in the work area. can.

On the other hand, the control unit 140 determines that a category related to the product name exists in the image 600 based on the second type of data "cafe latte (hot)." Here, the control unit 140 can display the already saved field name "item1" on the work area, although there is a field name "product name" corresponding to a category related to the product name in the extracted text. .

As described above, in the work area, the text extracted as a result of OCR can be divided into field names 611 to 615 and field values 621 to 625 and displayed. Here, the control unit 140 can match field names and field values belonging to the same category, and display data based on the matching result. For example, a field name "name" and a field value "HLP Coffee" belonging to the same category are matched and displayed adjacent to each other on the work area.

The control unit 140 generates content by matching data belonging to the same category based on the meaning of the extracted text, and generates a vector using the generated content and a deep learning model. Thereafter, the distance between the generated vector and the plurality of already stored data vectors is compared, and based on the comparison result, at least one is selected from the plurality of already stored data vectors.

The control unit 140 converts the "attribute-value" format data already stored in the database into the above-mentioned vector using the learned deep learning model. Data already stored in the database can be represented in a vector space as explained in FIG.

The control unit 140 may calculate the distance between the vector of the content and other already stored vectors, and select at least one vector from the other already stored vectors in order of decreasing distance between the vectors.

Thereafter, the control unit 140 may output data corresponding to the selected vector. Data corresponding to the selected vector can be displayed on the aforementioned work area.

To this end, the data embedding model according to the invention may include a decoder that converts the aforementioned vector into data in the form of "attribute-value" pairs. The decoder is machine-trained to convert data into the same form of data that was used as input data when generating a particular vector. Therefore, the decoder outputs data (eg, JSON, XML) in which categories to which a plurality of field values belong are classified.

For example, referring to FIG. 6b, the work area includes a first vector and a second vector that are the closest in distance from the vector corresponding to the content generated from the image described in FIG. 6a, respectively. Data 631 and 632 consisting of "attribute-value" pairs are displayed. Among the data, data 631 corresponding to the image 600 described in FIG. 6a is included.

More specifically, referring to FIG. 7, the control unit 140 calculates the distance between vectors 551a to 553c corresponding to a plurality of previously stored data and a vector 560 corresponding to the content. As a result, distances d1 to d3 are calculated for each of the vectors 551a to 553c corresponding to a plurality of already saved data. The control unit 140 selects the vector 552a having the closest distance from the vector 560 corresponding to the content from the vectors 551a to 553c corresponding to already saved data, and converts the vector 552a into "attribute-value" pair data (Name: NLP COFFEE, Tel: 01-234-567) and output.

As described above, according to the present invention, data is searched based on the similarity between vectors generated as a result of embedding, so there is no need to rely on search rules determined by a person when searching for data. It disappears.

Furthermore, according to the present invention, since it is possible to perform a search that takes into account the degree of similarity for each category of data, it is possible to perform a search using data that frequently contains noise or errors (for example, OCR data, voice recognition data). Data searches can be performed with high accuracy.

On the other hand, the present invention described above can be implemented as a program that is executed by one or more processes on a computer and can be stored on a computer-readable medium (or recording medium).

Further, the present invention described above can be implemented as computer readable codes or commands on a program recording medium. That is, the present invention can be provided in the form of a program.

On the other hand, the computer-readable medium includes any type of recording device on which data that can be read by a computer system is recorded. Examples of computer readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage. Examples include equipment.

The computer-readable medium also includes storage and can be a server or cloud storage that the electronic device can access via communication. In this case, the computer can download the program according to the invention from the server or cloud storage by wired or wireless communication.

Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, that is, a central processing unit (Central Processing Unit, CPU), and its type is not particularly limited.

On the other hand, the detailed description of the present invention is illustrative and should not be construed as limiting in any respect. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all modifications within the scope of equivalents of the present invention are included within the scope of the present invention.

10 Image 100 Data Search System 110 Communication Department 120 Storage Department 130 OCR Department 140 Control Department 150 Voice Recognition Department 160 DB (Database)

Claims (15)

receiving content including a plurality of field values;
arranging the field values included in the content, and adding a plurality of division molecules for classifying categories to which the field values belong to generate model input values;
generating a vector of the content using the model input value and the trained deep learning model;
searching for data corresponding to the search target data from the plurality of already stored data based on the degree of similarity between the generated vector and vectors corresponding to each of the plurality of already stored data. ,Data retrieval methods. The field values each correspond to a plurality of categories, and
2. The data search method according to claim 1, wherein the plurality of division molecules correspond to the plurality of categories, respectively. 3. The data search method according to claim 2, wherein a specific segment element and a specific field value corresponding to a specific category among the plurality of categories are arranged adjacent to each other. 4. The data retrieval method according to claim 3, wherein the specific segment is arranged before or after the specific field value. converting each of the field values included in the model input value into at least one first type token;
and converting each of the plurality of delimiters into a second type token,
The first and second type tokens converted from the field value and the delimiter corresponding to the specific category such that the first type tokens corresponding to each of the plurality of categories are classified from each other, The data retrieval method according to claim 4, wherein the data retrieval method is arranged adjacent to each other. The second type token is
each including text indicating the plurality of categories;
6. The data search method according to claim 5, wherein the text included in one of the second type tokens is different from the text included in the other one. The first type tokens corresponding to the specific field value among the field values are plural, and the plurality of first type tokens corresponding to the specific field value are arranged adjacent to each other. The data retrieval method according to claim 5, characterized in that: The second type token corresponding to the category to which the specific field value belongs is located at the front of the first type token arranged first among the plurality of first type tokens or among the plurality of first type tokens. 8. The data retrieval method according to claim 7, wherein the data retrieval method is arranged after the first type token arranged last. If the content includes a field name defining a particular category and does not include the field value corresponding to the particular category, the model input value includes a mask token corresponding to the particular category;
2. The data search method according to claim 1, wherein the mask token is arranged adjacent to a section molecule corresponding to the specific category. The step of searching for data corresponding to the search target data from the plurality of already saved data,
Comparing the distance between the vector of the content and the vector corresponding to each of the plurality of already stored data;
selecting at least one vector from the vectors corresponding to each of the plurality of already stored data based on the comparison result;
The data search method according to claim 1, further comprising the step of outputting data corresponding to the selected vector. The data corresponding to the selected vector is
Contains multiple field values belonging to different categories,
11. The data retrieval method according to claim 10, wherein the plurality of field values are classified into categories. The content is
2. The data retrieval method according to claim 1, wherein the data is in a form in which categories to which the plurality of field values belong are classified. 13. The data search method according to claim 12, wherein the content is generated from either OCR data or audio data related to images. a communication unit that receives content including a plurality of field values;
arranging field values included in the content, adding a plurality of division molecules for classifying categories to which the field values belong, and generating model input values;
generating a vector of the content using the model input value and the learned deep learning model;
a control unit that searches data corresponding to the search target data from the plurality of already stored data based on the similarity between the generated vector and the vector corresponding to each of the plurality of already stored data; Including, data retrieval system. A computer program including a plurality of instructions,
When the instruction is executed,
receiving content including a plurality of field values;
arranging the field values included in the content, adding a plurality of division molecules for classifying categories to which the field values belong, and generating model input values;
generating a vector of the content using the model input value and the trained deep learning model;
searching for data corresponding to the search target data from the plurality of already stored data based on the similarity between the generated vector and the vector corresponding to each of the plurality of already stored data;
A computer program that runs on a computer.