KR102435849B1 - Method for providing search result for non-text based object in documents - Google Patents

Abstract

A method for searching non-text-based objects such as tables, images, and graphs included in documents using an artificial intelligence-based search model is provided.
A document is given a description of an object that is not text-based, such as tables, images, and graphs, and a specific paragraph within the document contains a description of the object. Accordingly, a passage for each entity can be generated by collecting these text-based information. The title or abstract of the document may also be included in the passage. Using the text message generated in this way, non-text-based objects such as tables, images, and graphs included in a document can be searched for.
When an AI-based search model is used, search results with improved performance can be obtained compared to using only a search model based on unsupervised learning.

Description

Method for providing search result for non-text based object in documents}

The present invention relates to a method for providing a search result for a non-text-based entity included in a document, and more specifically, to a table and image included in a document using an information retrieval model learned based on the Wick Supervision methodology. It is about a method of providing search results for non-text-based objects such as , graphs, etc.

Search technology has been continuously developed since Google introduced the PageRank search technology based on graph theory. This search technology is based on unsupervised learning, and it was possible to search only if a bundle of documents was given. A representative example of a search model based on unsupervised learning is BM25, and it shows very improved performance when used together with the query expansion technique called RM3. As an open source, Anserini is widely used in academic fields and fields.

On the other hand, various search models have been proposed according to research in the academic field to apply artificial intelligence techniques in the field of natural language processing. For example, deep learning-based search models such as DRMM, KNRM, and PACRR have been proposed. BERT, released by Google in 2018, showed good performance in various natural language processing fields, and research to use it as a search model based on a transformer or language model has continued.

In the Ad-Hoc Information Retrieval section of Paper With Code, a website that introduces open source artificial intelligence models for each field, the current time of AI-based search models, including Anserini, a search model based on unsupervised learning SOTA (State-of-the-Art), that is, the search model that shows the best performance can be identified. According to a researcher at the University of Waterloo, Canada named Jimmy Lin, the search models of deep learning prior to BERT, that is, search models such as DRMM, KNRM, and PACRR, are the search models based on the unsupervised learning methodology, Anserini and Although the performance was comparable or rather inferior, it is said that the models proposed after BERT performed better than Anserini (see Lin, Jimmy. "The Neural Hype, Justified! A Recantation."). These details can also be confirmed on the leader board of the Ad-Hoc Information Retrieval item of Paper With Code. From these academic research results, it can be seen that the accuracy of the search results can be improved by the AI-based search model.

However, the AI-based search model has several limitations.

In order to use an AI-based search model for inference, it must first be trained, and this training requires a large amount of labeled data. Labeled data must be processed and provided by humans, but considering the amount of data required for learning, the cost of labeling is too high, which is uneconomical.

As another problem, search models based on unsupervised learning generally do not have a problem even if the length of a document is long, but most of the AI-based search models have a limitation in the length of documents that can be processed. For example, in the case of BERT, the maximum number of tokens that can be processed is limited to 512. Therefore, there is no problem when a corpus composed of short texts is used as a search target, but it is difficult to apply when searching for long documents such as patents and papers.

On the other hand, if a method for searching objects such as tables, images, and graphs included in a document using text-based information retrieval technology is provided, the user's convenience will be increased.

[1] https://paperswithcode.com/task/ad-hoc-information-retrieval [2] MacAvaney, Sean, et al. "CEDR: Contextualized embeddings for document ranking." Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019. [3] Dai, Zhuyun, and Jamie Callan. "Deeper text understanding for IR with contextual neural language modeling." Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019.

An object of the present invention is to solve the above problem, and to provide a method for providing search results by using a text-type query for objects such as tables, images, and graphs included in a document.

According to an aspect of the present invention, there is provided a computer-implemented method for searching for an entity included in a document using an artificial intelligence-based search model, wherein (a) a description of a specific entity included in the document and the entity are mentioned generating a passage corresponding to the object, including a paragraph; (b) forming an entity-related corpus formed by collecting passages for the entity formed by step (a) from a plurality of documents; (c) retrieving N passages corresponding to the input query from the corpus formed in step (b) by a search model based on an unsupervised methodology; (d) re-ranking the N passages retrieved in step (c) based on the input query by an AI-based search model; and, (e) outputting objects corresponding to the N passages reranked in step (d) as search results.

According to another aspect of the present invention, there is provided an apparatus for searching an object included in a document using an artificial intelligence-based search model, comprising: at least one processor; and at least one memory storing computer-executable instructions, wherein the computer-executable instructions stored in the at least one memory are transmitted by the at least one processor to (a) a specific object included in the document. generating a passage corresponding to the object, including a description of the object and a paragraph in which the object is mentioned; (b) forming an entity-related corpus formed by collecting passages for the entity formed by step (a) from a plurality of documents; (c) retrieving N passages corresponding to the input query from the corpus formed in step (b) by a search model based on an unsupervised methodology; (d) re-ranking the N passages retrieved in step (c) based on the input query by an AI-based search model; and, (e) outputting objects corresponding to the N passages reranked in step (d) as search results.

According to another aspect of the present invention, as a computer program for searching for an object included in a document using an artificial intelligence-based search model, it is stored in a non-transitory storage medium, and by the processor, (a) generating a passage corresponding to the object, including a description of the object and a paragraph in which the object is mentioned; (b) forming an entity-related corpus formed by collecting passages for the entity formed by step (a) from a plurality of documents; (c) retrieving N passages corresponding to the input query from the corpus formed in step (b) by a search model based on an unsupervised methodology; (d) re-ranking the N passages retrieved in step (c) based on the input query by an AI-based search model; and (e) outputting objects corresponding to the N passages reranked in step (d) as search results. A computer program is provided.

According to the present invention, there is provided a method for providing a search result using a text-type query for an object included in a document.

1 is a flowchart illustrating a method for learning an artificial intelligence-based information retrieval model according to the present invention.
2 is a flowchart illustrating an information retrieval method using an artificial intelligence-based information retrieval model according to the present invention.
3 is a diagram schematically illustrating an information retrieval model according to an example of the present invention.
4 is a diagram schematically illustrating an apparatus for performing an information retrieval method using an artificial intelligence-based information retrieval model according to the present invention.
5 is a diagram for comparing a search result according to an embodiment of the present invention with a search result of another model;
6 is a diagram for comparing a search result according to an embodiment of the present invention with a search result of another model;

Hereinafter, an embodiment according to the present invention will be described in detail with reference to the accompanying drawings. The same or similar reference numerals are assigned to the same or similar components, and overlapping descriptions thereof are omitted. In describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. The accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed herein is not limited by the accompanying drawings, and all changes and equivalents included in the spirit and scope of the present invention It should be understood to include what is or what is replaced.

Terms including an ordinal number such as 1st, 2nd, etc. may be used to describe various components, but these terms are used only for the purpose of distinguishing one component from other components, and the corresponding components are defined by these terms. not limited by The singular expression includes the plural expression unless the context clearly dictates otherwise.

As used herein, terms such as “comprises”, “comprises” or “have” are to be understood as limiting the existence of a feature, step, element, or combination thereof described in the specification, and one or more other It is not intended to exclude the possibility that features, steps, components, or combinations thereof may be present or added.

1 shows a method for learning an artificial intelligence-based information retrieval model according to the present invention, and FIG. 2 shows a step of performing inference using the artificial intelligence-based information retrieval model according to the present invention, 3 schematically shows an example of an information retrieval model according to the present invention. Hereinafter, the present invention will be described with reference to these drawings.

[Learning stage of information retrieval model based on artificial intelligence]

information retrieval model

In this specification, the IR model is divided into a 'search model based on an unsupervised methodology' and an 'artificial intelligence-based search model'. The former means information retrieval models based on statistical or other unsupervised learning methodologies such as BM25 and QL (Query Likelihood). It means an information retrieval model prepared by learning, including

In the present invention, the former search model based on the unsupervised learning methodology is used to prepare pseudo-labels in the learning stage, and is used to retrieve passages from the corpus in the inference stage, and the latter is used to It is used to re-rank the passage retrieved in the reasoning step.

Information retrieval models for which open sources exist are disclosed in the Ad-Hoc Information Retrieval section of Paper With Code.

Search model based on unsupervised learning methodology

As search models based on unsupervised methodologies, BM25 and Query Likelihood (QL) are known, and Anserini is widely used as an open source. Anserini is a search model that uses RM3, a query expansion methodology, along with BM25. According to the research results, the performance of Anserini is similar to that of deep learning-based search models such as DRMM, KNRM, and PACRR, but it is known that the performance of Anserini is inferior to that of language model-type search models such as BERT.

The search model based on the unsupervised learning methodology does not require learning and determines the similarity between the query and the document based on statistical theories.

AI-based search model

As AI-based search models, deep learning-based search models such as DRMM, KNRM, and PACRR, and transformer or language model-based search models such as BERT are known. According to Paper With Code, the best performing SOTA (State-of-the-Art) search model as of the filing date of this patent is CEDR, which has a structure that combines the language model-type BERT and the deep-learning-type search model. is understood to be

The AI-based search model is completed by learning the similarity between the query and the document based on the training data. Academically, AI-based search models are trained using datasets that provide query-document relationships, such as TREC, SQuAD, and MS Marco.

Language model search engines such as BERT use a self-supervision learning methodology in the pre-training stage, but deep learning in the fine-tuning stage for information retrieval. Like the search model of , supervised learning based on a dataset on the query-document relationship is required.

Weaksupervision Methodology

Query-document relation datasets such as TREC, SQuAD, and MS Marco that are used academically correspond to ground truth extracted based on real users' search activity, or some use the title of a document as a query. However, in the actual field, only a document corpus exists, and there is no query input by the user to search for it, or even if there is, it is very insufficient to train an AI-based search model. In the present invention, when only a document bundle exists, a pseudo-query is generated from each document, a pseudo-query-document relationship is formed using the generated pseudo-query, and a pseudo-label is generated. , a methodology for learning an AI-based search model using this is provided. The Wick Supervision methodology for the AI-based search model according to the present invention broadly includes the following steps: 1) generating a pseudo-query from each document in the document bundle, 2) the generated pseudo-query Creating a pseudo-query-document relationship using . Hereinafter, each of the above-described steps will be described by dividing the items.

1) Steps to create a pseudo-query

One or more keywords are extracted from each document in the document bundle and determined by pseudo-query. A method of extracting a keyword from a document uses a known keyword extraction technique. Keyword extraction algorithms can also be largely divided into techniques based on unsupervised learning and techniques based on supervised learning, and open sources for a number of techniques exist.

Although most techniques are methods of extracting multiple keywords or phrases from a document, recently, as a method based on supervised learning, a technique called Doc2Query, which uses BERT to generate a query in the form of a natural language sentence, has also been released as an open source. there is a bar

2) Create a pseudo-label

M documents are retrieved from the document bundle by using the pseudo-query extracted from each document as an input to a search model based on unsupervised learning methodologies such as BM25. In this case, the document from which the corresponding pseudo-query is extracted is highly likely to be included in the top among the M documents, but may not be included.

Among the retrieved M documents, the top m (<M) documents are labeled as positive training data, and at least some of the remaining documents among the M documents are labeled as negative training data. According to the AI-based search model, it is possible to additionally label other parts as neutral training data. In general, positive data and negative data are necessary, but neutral data is not necessarily required.

Here, the number of retrieved documents M, the number of positive learning data m, the number of negative learning data, and the number of neutral learning data are integers, and may be determined differently depending on the characteristics of the document bundle as a kind of hyperparameter. Developers can increase the accuracy of AI-based search models by adjusting these hyperparameters in an experimental or theoretical approach according to the characteristics of the document bundle.

3) Learning the AI-based search model

Since a query-document relationship is required to train an AI-based search model, the training data is provided in the form of a relationship between a query used for retrieval and a document retrieved by the query. For example, (pseudo-query, positive document) may be provided in the form of a pair of two data, or (pseudo-query, positive document, negative document) may be provided in a form in which three pieces of data are related. Of course, up to a neutral document can also be associated with four data.

The amount of data required for learning is not known exactly, and it is mainly confirmed by an experimental method. However, the amount of data required for training increases according to the number of parameters included in the model. For search models based on the BERT series of language models, pre-trained models are generally published based on Wikipedia, etc. Therefore, a small amount of data is required.

In academic research, the amount of learning data is determined according to the available dataset, but in the actual field, the amount of learning data may be determined according to the amount of documents included in the document bundle. When the amount of documents included in the document bundle is insufficient for learning, other documents in a similar field may be added to the document bundle, or the amount of data may be increased by using a data augmentation technique.

When training an AI-based model, it is generally desirable to verify performance by dividing the data into training data and verification data. For example, 80% of the total data may be used for learning, and the remaining 20% of the data may be used for verification.

[Inference stage]

In the present invention, the reasoning step refers to a process of providing a search result in response to a query input by a user. The reasoning step can be roughly divided into the following steps: a retrieving step, a re-ranking step, and an output step. Hereinafter, each step will be described in detail.

retrieval stage

It is a step of extracting N documents from a document bundle which is a search target by a search model based on an unsupervised learning methodology. In general, the AI-based search model takes considerably longer for inference than the search model based on the unsupervised learning methodology. Accordingly, when the AI-based search model is applied to all documents included in the document bundle, it takes too long, and the user's convenience deteriorates. Therefore, after retrieving a large number of documents by the search model based on the unsupervised learning methodology, which is relatively fast in the first place, it is common to apply the AI-based search model only to the secondly retrieved documents. Research to omit the retrieval step by improving the inference speed of the AI-based search model is continuously being conducted, but it is understood that the inference speed has not yet reached a sufficiently fast inference speed considering the user convenience.

At this stage, accuracy is important, but recall is more important. On the other hand, in the second re-ranking stage, accuracy is more important than recall. One of the methods to increase the recall is to retrieve a number of documents several times larger than the number of documents to be provided in the search results. Another method is to increase the recall rate of the search model based on the unsupervised learning methodology. By using a technique called DeepCT Index using BERT, it is possible to increase recall while using a search model based on the same unsupervised learning methodology. In order to retrieve a large amount of documents, the time required for retrieval increases in proportion to the number of documents. In the case of using the DeepCT Index technique, it is possible to reduce the search time by reducing the number of documents to be retrieved, while achieving the same level of recall as when retrieving a number of documents several times larger than that.

As a search model based on the unsupervised learning methodology, the same search model used in the learning step may be used, or a different search model may be used. It is important to select a search model with high recall in the relevant field. In academic research, Anserini (BM25+RM3) is widely used as an open source-based search model.

re-ranking stage

The relevance between the retrieved N documents and the query entered by the user is evaluated by a search model based on the unsupervised learning methodology and sorted according to the order of relevance. In this step, the relation between the retrieved N documents and the query entered by the user is re-evaluated by the AI-based search model and then rearranged, that is, re-ranked. This step is performed by the AI-based search model learned in the above-described learning step.

Search result output

The N documents reranked by the AI-based search model are sorted according to the order of relevance and output as a search result.

[Handling in special cases]

Matters related to the length of the document

In general, search models based on unsupervised learning methodologies have no problem in execution even if the length of documents is long, but AI-based search models often have a limit on the maximum length of documents that can be processed. In particular, AI-based search models introduced after BERT have a limited number of maximum tokens that can be processed. For example, in the case of BERT, the maximum number of tokens that can be processed is limited to 512.

When there is a limit on the maximum length of a document that can be processed by the AI-based search model, each document in a document corpus can be divided into passages with a length less than the limited length and can be searched. In the present specification, a bundle of passages composed of short texts is referred to as a 'corpus' in order to distinguish it from a document bundle composed of long documents.

In the learning stage, the passages included in the corpus are the same as described above except that the passages are replaced with documents. That is, after extracting a pseudo-query from a passage and generating a pseudo-label using a search model based on an unsupervised learning methodology, an AI-based search model is trained. The AI-based search model learns based on the relationship between the pseudo-query and the passage, not the relationship between the pseudo-query and the entire document.

In the reasoning step, it is necessary to divide each document in the document bundle into passages to make a corpus, and to perform preprocessing of storing the correspondence between passages and documents in a form that can be referred to. In the retrieval step, N passages are retrieved from the corpus based on the query input by the user, and in the reranking step, the retrieved N passages are reranked.

Documents that are not passages in the search results should be provided in order of relevance. To this end, with reference to the correspondence between the passages and documents provided in the pre-processing step, the documents are arranged so as to correspond to the re-ranked passage's relevance sort order and provided as a search result. Since several passages are separated from one document, a plurality of passages extracted from one document may be included in the reranked result. In this case, the sort order of documents may correspond to the order of passages with the highest degree of relevance.

document expansion

In the case of a long document, it contains multiple paragraphs, and the content of each paragraph does not necessarily match the topic. Accordingly, when each paragraph is divided into passages and stored as a corpus, there is a possibility that the relevance is evaluated low even though the paragraphs are separated from related documents.

In order to prevent such a problem, the title of the document or a phrase equivalent thereto may be added to each passage separated from the document. That is, the same phrase is added to each passage separated from one document. In this specification, the extension by adding a phrase having relevance to the passage is expressed as a key word 'document extension'. A phrase corresponding to the title of the document may be generated using, for example, a text summarization technique. The document summarization technique is largely divided into extractive summarization and abstract summarization, both of which can be applied. The technique for generating additional phrases for document extension is not necessarily limited to the document summary technique, and any technique that can compressively express the subject of the document may be used.

For example, a plurality of keywords are included in a query input by a user. Some keywords may be included in the original message, but other keywords may be included in a phrase added by document expansion. Since the phrase added by the document extension includes the subject of the entire document, there is a possibility that the corresponding passage contains the content intended by the user to search. If the corresponding phrase was not added by document extension, a passage that would not have been retrieved may be retrieved by document extension. This may increase the recall of the retrieval step. As described above, it is important that the recall is high in the retrieval step.

If the length of the phrase added to the passage due to the document extension is too long, the length of the passage increased by the document extension may exceed the length that the AI-based search model can handle. In order to minimize the side effects caused by this, it is preferable that the text added by the document extension be placed at the front of the passage. Even if the rear end of a passage whose length is increased due to document expansion is cut off, if the keyword included in the query entered by the user is not included in the rear end, the passage will still be extracted in the retrieval stage.

domain adaptiveness

When an AI-based search model is trained as a document of a specific domain, it can be used as a search model specialized for that domain. For example, when fine-tuning a BERT that has been pre-trained with a general document bundle such as Wikipedia as a document of a specific domain, in the pre-learning stage, general relationships between vocabularies are learned, and in fine-tuning, a vocabulary specific to that domain is learned. will learn

A search model specialized for a specific domain may have relatively poor performance for other domains, but it improves performance in that domain.

Ensemble search model

In the above method, the results reranked by the artificial intelligence-based search model were used as the final search results. may be In this case, the final evaluation is expressed as Equation 1.

[Equation 1]

(Final evaluation) = a*(Evaluation of search model based on unsupervised learning methodology)

+ (1-a)*(Evaluation of AI-based search model)

In Equation 1, a is a value between 0 and 1, and may be adjusted to provide the best search result as a hyper parameter.

The ensemble model does not necessarily guarantee better performance compared to the single model, and it can be reviewed for adoption depending on the domain.

4 shows a computer device for performing a search method according to the present invention.

Since the search method and the learning method according to the present invention have been described in detail with reference to FIGS. 1 to 3 , an apparatus 100 for performing such a search method will be briefly described with reference to FIG. 4 .

Referring to FIG. 4 , the computer device 100 includes a processor 110 , a non-volatile storage unit 120 for storing programs and data, a volatile memory 130 for storing programs being executed, and information between a user and a user. It consists of an input/output unit 140 for inputting and outputting , and a bus, which is an internal communication path between these devices. The running program may include an operating system and various applications. Although not shown, it includes a power supply unit.

In the learning step, an AI-based search model is learned in the memory 130 using the learning data stored in the storage unit 120 . In the reasoning step, the memory 130 executes the unsupervised learning-based search model 210 and the artificial intelligence-based search model 220 stored in the storage unit 120 . The corpus is stored in the storage unit 120 , and a search method is performed based on a query input through the input/output unit 140 .

[Example]

Hereinafter, an embodiment in which the search result providing method according to the present invention is applied to a patent document owned by the Korea Institute of Industrial Technology will be described.

AI-based search model

As an AI-based search model, the CEDR-KNRM model, which is open source, was used. The CEDR-KNRM model is a structure that processes BERT and KNRM in parallel when described briefly, and details can be checked from related papers and published source codes. According to Paper With Code, it was the model with the best performance at the time of implementing the embodiment, and it is confirmed that there is no change as of the filing date.

corpus

Patent documents to be searched, that is, the document corpus was approximately 3000, and since the length of all documents exceeds 512 tokens that BERT can process, the corpus was formed by dividing it into paragraphs. The corpus contains approximately 120,000 passages.

The relationship between each passage and the extracted patent document is stored so that it can be referenced later in the reasoning step.

Pseudo-query generation

One or more keywords or phrases were extracted as pseudo-queries by applying the keyword extraction technique to the passage formed by each paragraph of the patent document. The technique used here is RAKE, which has been published in papers and open sources.

Deduplication of the same pseudo-query resulted in approximately 40,000 pseudo-queries.

Forming Query-Document Relationships

With each pseudo-query as input, 300 passages were extracted from the corpus using BM25, a search model based on the previously known unsupervised learning methodology. Since there were approximately 40,000 pseudo-queries, the total number of extracted passages is 12,000,000. This is a sufficient amount of data to train the CEDR-KNRM model.

pseudo-labeling

Among the 300 passages extracted by each pseudo-query, the upper m passages were classified as positive learning data, and the lower p passages were classified as negative learning data. Since the CEDR-KNRM model also uses neutral learning data, the remaining passages except for the upper m passages and the lower p passages are classified as neutral learning data.

Hyperparameters such as m and p were adjusted while validating the performance of the search model after learning by the inventor reviewing the search results. Since it is a dataset without ground truth, a part of the prepared training data can be used as data for verification, but since the inventor is a field that can be manually verified, the performance was verified through manual work.

inference stage

When a user inputs a query composed of a plurality of keywords, 300 passages are retrieved from the corpus using BM25. The retrieved 300 passages are then provided as a trained CEDR-KNRM model along with the query entered by the user. The CEDR-KNRM model reranks 300 passages. Search the patent documents corresponding to the re-ranked 300 passages and arrange the order of the patent documents to correspond to the order of the passages. If there is one belonging to the same patent document among the retrieved packages, the corresponding patent document is sorted according to the order of the highest-rated passage, and the lower-order passage is ignored when sorting the patent document. Therefore, the search results are less than or equal to 300.

Compare search results

5 and 6 illustrate a comparison between a result searched by the search method developed according to the present embodiment and a result searched by another search method.

5 is a search result among patents owned by the Korea Institute of Industrial Technology by inputting a query including two keywords "wheelchair" and "driving assistance". is the search result from KIPRIS. As shown, in the search method developed according to the present embodiment, documents containing words exactly matching the keywords "wheelchair" and "driving report" as well as other devices available to persons with reduced mobility are searched for. While patents are also searched, only documents containing words that exactly match the keywords "wheelchair" and "drive report" are searched in KIPRIS.

6 shows the results of searching among patents held by the Korea Institute of Industrial Technology by entering a query including six keywords: “scanning sonar”, “safety monitoring”, “marine operation equipment”, “marine monitoring”, and “aquaculture monitoring” As , the upper side is the result searched by the search method developed according to this embodiment, and the lower side is the search result from the Google patent search. However, in the case of the Google patent search, the search was performed except for the keyword "farm monitoring" because the search was not performed when the number of keywords included in the query exceeded 5. As shown, in the search method developed according to the present embodiment, documents including semantically similar words even if they do not exactly match the keywords included in the query are searched for, whereas there are no search results in the Google patent. These results are the same for KIPRIS (not shown).

As can be seen from FIGS. 5 and 6 , the search method according to the present invention using an artificial intelligence-based search model may additionally search documents that do not include keywords included in the input query but are semantically related. have.

[Application to retrieval of tables, images, graphs, etc. included in documents]

For non-text-based objects such as tables, images, and graphs included in a document, a text-based description corresponding to the document exists. For example, in the case of a thesis, reference signs and brief descriptions of the objects are placed above or below for tables, images, graphs, etc., and text-based descriptions of the objects exist within the document. As another example, in the case of a patent publication, there is a brief description of the drawings in the detailed description of the specification, and there is a paragraph describing each drawing. Depending on the type of object, reference signs are given in Table 1, Figure 1, Figure 1, and the like.

Based on such text-based information, a passage corresponding to each non-text-based object such as a table, image, or graph may be generated. For example, a passage corresponding to Table 1 of Document 1 may be generated by combining a description of the corresponding table placed above or below the corresponding table and a paragraph including the description of the corresponding Table 1 in Document 1. . Furthermore, if the document extension technique described above is applied, the title, abstract, etc. of document 1 can be added to the passage.

These passages form a corpus of non-text-based entities included in the document. An AI-based search model can be trained using only the corpus for objects, not text-based. Alternatively, a corpus of non-text based objects may be processed only as a search target, and a search model learned based on a corpus composed of paragraphs of a document may be used.

If the relationship between documents, non-text-based objects and passages is tabulated or indexed, not only passages corresponding to the search results but also non-text-based objects (tables, images, graphs, etc.) corresponding to the passage are searched together. results can be provided.

The foregoing detailed description should not be construed as restrictive in any way but as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (9)

A computer-implemented method for searching a non-text-based object included in a document using an artificial intelligence-based search model, comprising:
(a) given a document containing a non-text-based entity, generate a passage corresponding to that entity, including a description of the particular non-text-based entity contained in the document, and a paragraph to which the entity is referenced; and creating a relationship between the document, entity and passage;
(b) forming an entity-related corpus formed by collecting passages for the entity formed by step (a) from a plurality of documents and a relation table between the documents, entities, and passages;
(c) retrieving N passages corresponding to the input query from the corpus formed in step (b) by a search model based on an unsupervised methodology;
(d) re-ranking the N passages retrieved in step (c) based on the input query by an AI-based search model; and,
(e) outputting the N passages reranked in step (d), an entity corresponding thereto, and a document including the passage and the entity as a search result
A computer-implemented method for retrieving non-text-based objects embedded in documents that contain
The method according to claim 1, wherein in step (a),
Each passage is generated to further include the title of the document including the specific object
A computer-implemented method for retrieving non-text-based objects contained in a document, characterized in that
The method according to claim 1, wherein in step (a),
Each passage is generated to further include an abstract of a document containing the specific entity.
A computer-implemented method for retrieving non-text-based objects contained in a document, characterized in that
The method according to claim 1,
The AI-based search model is a corpus that is learned based on a corpus created by collecting paragraphs of each document, which is generated separately from the entity-related corpus of step (b).
A computer-implemented method for retrieving non-text-based objects contained in a document, characterized in that
A device for searching non-text-based objects included in documents using an artificial intelligence-based search model, comprising:
at least one processor; and
at least one memory for storing computer-executable instructions;
The computer-executable instructions stored in the at least one memory are executed by the at least one processor,
(a) given a document containing a non-text-based entity, generate a passage corresponding to that entity, including a description of the particular non-text-based entity contained in the document, and a paragraph to which the entity is referenced; and creating a relationship between the document, entity and passage;
(b) forming an entity-related corpus formed by collecting passages for the entity formed by step (a) from a plurality of documents and a relation table between the documents, entities, and passages;
(c) retrieving N passages corresponding to the input query from the corpus formed in step (b) by a search model based on an unsupervised methodology;
(d) re-ranking the N passages retrieved in step (c) based on the input query by an AI-based search model; and,
(e) outputting the N passages reranked in step (d), an entity corresponding thereto, and a document including the passage and the entity as a search result
A device for retrieving non-text-based objects embedded in a document that causes it to run.
6. The method of claim 5,
Each passage is generated to further include the title of the document including the specific object
A device for retrieving non-text-based objects contained in a document, characterized in that
6. The method of claim 5,
Each passage is generated to further include an abstract of a document containing the specific entity.
A device for retrieving non-text-based objects contained in a document, characterized in that
6. The method of claim 5,
The AI-based search model is a corpus that is learned based on a corpus created by collecting paragraphs of each document, which is generated separately from the entity-related corpus.
A device for retrieving a non-text object included in a document, characterized in that it.
A computer program for searching non-text-based objects included in documents using an artificial intelligence-based search model,
stored on a non-transitory storage medium, coupled to the processor,
(a) given a document containing a non-text-based entity, generate a passage corresponding to that entity, including a description of the particular non-text-based entity contained in the document, and a paragraph to which the entity is referenced; and creating a relationship between the document, entity and passage;
(b) forming an entity-related corpus formed by collecting passages for the entity formed by step (a) from a plurality of documents and a relation table between the documents, entities, and passages;
(c) retrieving N passages corresponding to the input query from the corpus formed in step (b) by a search model based on an unsupervised methodology;
(d) re-ranking the N passages retrieved in step (c) based on the input query by an AI-based search model; and,
(e) outputting the N passages reranked in step (d), an entity corresponding thereto, and a document including the passage and the entity as a search result
A computer program stored in a non-transitory storage medium to execute a method comprising a.

Images