KR20230101214A - System and Method of paragraph exploration and paragraph selection for Machine Reading Comprehension - Google Patents

Abstract

The present invention is performed by a computing device using a database in which documents to be searched are stored and a control server having an arithmetic function, and a system for searching and selecting paragraphs to perform machine reading comprehension, comprising: a question input unit into which a preset question is input; a paragraph search unit that searches for and selects a paragraph by using a similarity between the input question and a paragraph belonging to a search target document; and a paragraph transmitter for transmitting the target paragraphs selected by the paragraph search unit to the machine reading comprehension model according to transmission order, wherein the paragraph search unit calculates a similarity score between a question and a paragraph and selects an effective paragraph. paragraph; a paragraph grouping unit for grouping valid paragraphs and calculating group scores; a machine reading comprehension target section for selecting an analysis target document using a group score and additionally selecting a machine reading target paragraph within the analysis target document; and a transmission order calculation unit for calculating the transmission order of the machine-reading target paragraph.

Description

Paragraph search and paragraph selection system and method for machine reading comprehension {System and Method of paragraph exploration and paragraph selection for Machine Reading Comprehension}

The present invention relates to a paragraph search and paragraph selection system and method. Specifically, it relates to a paragraph search and paragraph selection system and method for machine reading comprehension.

Conventionally, in order to obtain specialized knowledge in a specific field online, a user inputs a word or sentence that he/she wants to know through an electronic library or Internet portal search. In addition, it was necessary to go through the process of checking whether or not the information the user needed was correct for the results found by the search engine.

As such, knowledge retrieval for a specific field tends to have a high level of difficulty in processing a question in natural language because there are not only few books on the field but also many technical terms. For this reason, there are many systems that collect books and related data on a specific field, store them in a database, and search them. Recently, there has been an attempt to develop a question-answering system by introducing machine reading comprehension technology.

Machine Reading Comprehension is an artificial intelligence algorithm method that learns the ability of a machine to read and understand documents through questions, and infers and outputs an appropriate response that exists in the document when a new question is given.

A lot of progress has been made using BERT (Pre-training of deep Bidirectional Transformers for Language Understanding), a language model that has been pre-trained as a technology recently released in the field of natural language processing. By using BERT alone, it shows the best performance in the field of question answering by combining additional models with insufficient information.

When operating a machine reading comprehension service that finds an answer to a question, in order to find a document that is likely to have the correct answer to the question, which is the input data of the model, all documents in the document database are entered into the model to find the correct answer, or if the correct answer is A method to find the correct answer by selecting documents with similarity criteria is proposed.

In order to operate such a machine reading comprehension service, pre-processing of selection to find documents that are candidates for correct answers to questions is usually performed through vector-based cosine similarity or keyword-based similarity. The correct answer was inferred by finding a document or paragraph through the similarity and entering it into the machine reading comprehension model.

In order to operate a machine reading comprehension service with conventional technology, a high level of natural language preprocessing technology is required for databases, and cosine similarity and keyword similarity technologies with excellent performance must be used to quickly find answers to questions.

In addition, a program that infers an answer through a machine reading comprehension model consumes a lot of system resources, so it is inefficient to scan numerous documents to find an answer to a single question. Even if natural language preprocessing or similarity calculation technology is not at a high level, for resource efficiency, it is necessary to find paragraphs and documents with correct answers.

(Document 1) Korea Patent Registration No. 10-2292040 (2021.08.13)

The paragraph search and paragraph selection system and method for machine reading comprehension according to the present invention have the following problems.

First, we try to extract the machine reading comprehension target paragraph, which is a paragraph similar to the input question, and pass it to the machine reading comprehension model.

Second, it is intended to extract effective paragraphs while reducing the time to select the target paragraphs to be delivered to the machine reading comprehension model.

Third, it is intended to reduce the time to find the correct answer through the machine reading comprehension target paragraph.

The problems of the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

The present invention is performed by a computing device using a database in which documents to be searched are stored and a control server having an arithmetic function, and a system for searching and selecting paragraphs to perform machine reading comprehension, comprising: a question input unit into which a preset question is input; a paragraph search unit that searches for and selects a paragraph by using a similarity between the input question and a paragraph belonging to a search target document; and a paragraph transmitter for transmitting the target paragraphs selected by the paragraph search unit to the machine reading comprehension model according to transmission order, wherein the paragraph search unit calculates a similarity score between a question and a paragraph and selects an effective paragraph. paragraph; a paragraph grouping unit for grouping valid paragraphs and calculating group scores; a machine reading comprehension target section for selecting an analysis target document using a group score and additionally selecting a machine reading target paragraph within the analysis target document; and a transmission order calculation unit that calculates a transmission order of a paragraph to be read by the machine.

In the present invention, the effective paragraph unit may include a similarity calculation unit that extracts all paragraphs included in the search target document and calculates a similarity score between each extracted paragraph and an input search question.

In the present invention, the similarity score can be calculated by the similarity calculation unit through vector-based cosine similarity or keyword-centered similarity.

In the present invention, the effective paragraph unit sorts each paragraph in the order of similarity score calculated by the similarity calculation unit, and selects a paragraph having a similarity score equal to or higher than a preset value or a preset number of paragraphs in order of similarity score. A selector may be provided.

In the present invention, the paragraph grouping unit may include a paragraph grouping unit for grouping the paragraphs selected from the effective paragraph unit according to documents to which each paragraph belongs.

In the present invention, the paragraph grouping unit may include a group score calculating unit that calculates a group score by calculating an average value of similarity scores of each paragraph grouped by the paragraph grouping unit.

In the present invention, the machine reading target paragraph unit may include a document selection unit for arranging documents in the order of group scores calculated in the paragraph group unit and selecting a document having a group score equal to or higher than a preset value as an analysis target document. .

In the present invention, when the number of documents having a group score equal to or higher than a preset value exceeds the preset maximum number of documents, the document selector may select only up to the preset maximum number of documents.

In the present invention, the machine reading comprehension unit may include a paragraph adding unit for adding a predetermined number of paragraphs as machine reading target paragraphs by arranging similarity scores of remaining paragraphs other than valid paragraphs in the document selected by the document selection unit. there is.

In the present invention, the transfer priority calculation unit may calculate a transfer priority score according to the following relational expression 1 for each paragraph selected from the target paragraphs for machine reading comprehension.

[Relationship 1]

Transmission priority value = (paragraph rank value within group × 100) + group priority value

The present invention is performed by a computing device using a database in which search target documents are stored and a control server having an arithmetic function, and searching for and selecting a paragraph to perform machine reading comprehension, wherein the computing device inputs a predetermined question into a question input unit. Step S100 being entered; Step S200 of searching for and selecting a paragraph by using the similarity between the input question and the paragraph belonging to the search target document by the paragraph search unit; and a step S300 of transmitting, by the paragraph transfer unit, the target paragraphs selected by the paragraph search unit to the machine reading comprehension model according to the transmission order, wherein the effective paragraph calculates a similarity score between the question and the paragraph , step S210 of selecting a valid paragraph; Step S220 of grouping valid paragraphs by the paragraph grouping unit and calculating group scores; Step S230 of selecting an analysis target document by using a group score by the machine reading target paragraph and additionally selecting a machine reading target paragraph within the analysis target document; and a step S240 in which the transmission order calculation unit calculates the transmission order of the paragraph to be read by the machine.

In the present invention, the step S210 may include a step S211 in which the similarity calculation unit extracts all paragraphs included in the search target document and calculates a similarity score between each extracted paragraph and the input search question.

In the present invention, the similarity score calculation in step S211 can be calculated through vector-based cosine similarity or keyword-centered similarity.

In the present invention, in the step S210, the effective paragraph selector arranges each paragraph in the order of similarity score calculated by the similarity calculation unit, and selects a paragraph having a similarity score equal to or higher than a preset value or a preset number of paragraphs in order of similarity score. A selection step S212 may be provided.

In the present invention, the step S220 may include a step S221 in which a paragraph grouping unit groups the paragraphs selected from the valid paragraphs by document to which each paragraph belongs.

In the present invention, the step S220 may include a step S222 in which the group score calculator calculates a group score by calculating an average value of similarity scores of each paragraph grouped by the paragraph grouping unit.

In the present invention, the step S230 may include a step S231 in which the document selection unit sorts documents in the order of group scores calculated in the paragraph group unit and selects documents having a group score equal to or higher than a preset value as the analysis target document. there is.

In the present invention, in step S231, if the number of documents having group scores equal to or greater than the preset value exceeds the preset maximum number of documents, only up to the preset maximum number of documents can be selected.

In the present invention, the step S230 may include a step S232 in which the paragraph addition unit sorts the similarity scores of the remaining paragraphs other than valid paragraphs in the document selected by the document selection unit and adds a predetermined number of paragraphs as target paragraphs for machine reading. can

In the present invention, in the step S240, for each paragraph to be machine-read selected in the step S230, a transmission priority score may be calculated according to the relational expression 1 below.

[Relationship 1]

Transmission priority value = (paragraph rank value within group × 100) + group priority value

The present invention can be combined with hardware and implemented as a computer program stored in a computer-readable recording medium to execute the paragraph search and paragraph selection method for machine reading comprehension according to the present invention by a computer.

The paragraph search and paragraph selection system and method for machine reading comprehension according to the present invention have the following effects.

First, there is an effect of extracting a paragraph similar to the input question, the machine reading comprehension target paragraph, and passing it to the machine reading comprehension model.

Second, it has the effect of extracting effective paragraphs while shortening the time to select the target paragraphs to be delivered to the machine reading comprehension model.

Third, it has the effect of shortening the time to find the correct answer through the machine reading comprehension target paragraph.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a block diagram of a paragraph search and paragraph selection system for machine reading comprehension according to the present invention.
2 is a detailed configuration diagram of a paragraph search unit according to the present invention.
3 is a flowchart of a paragraph search and paragraph selection method for machine reading comprehension according to the present invention.
4 is a detailed flowchart of a paragraph search step according to the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described so that those skilled in the art can easily practice it. As can be easily understood by those skilled in the art to which the present invention pertains, the embodiments described below may be modified in various forms without departing from the concept and scope of the present invention. Where possible, identical or similar parts are indicated using the same reference numerals in the drawings.

The terminology used in this specification is only for referring to specific embodiments and is not intended to limit the present invention. As used herein, the singular forms also include the plural forms unless the phrases clearly indicate the opposite.

As used herein, the meaning of "comprising" specifies particular characteristics, regions, integers, steps, operations, elements, and/or components, and other specific characteristics, regions, integers, steps, operations, elements, components, and/or components. It does not exclude the presence or addition of groups.

All terms including technical terms and scientific terms used in this specification have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. The terms defined in the dictionary are additionally interpreted as having a meaning consistent with the related technical literature and the currently disclosed content, and are not interpreted in an ideal or very formal meaning unless defined.

Expressions related to directions used in this specification, for example, expressions of front/back/left/right, top/bottom, and longitudinal/lateral directions may be interpreted with reference to directions disclosed in the drawings.

The present invention relates to a process of preprocessing data to be input to an artificial intelligence model for machine reading comprehension. If valid data is entered into the machine reading comprehension model, the result of machine reading comprehension will be more meaningful.

Accordingly, the technical feature of the present invention is to select highly effective paragraphs while minimizing the time to extract the target paragraphs to be input to the machine reading comprehension model.

Key terms in the present invention are defined as follows.

'Documents to be searched' refer to all related documents acquired through various methods such as web crawling and stored in the database.

The 'analysis target document' refers to a document that is analyzed to select a target paragraph for machine reading among search target documents.

'Valid paragraph' is a paragraph with high similarity to the question, and is used to select the analysis target document.

'Additional paragraph' refers to a paragraph included in the document to be analyzed, which is not a valid paragraph, but is additionally selected.

'Excluded paragraph' means a valid paragraph included in the analysis target document or a paragraph excluded from the analysis target.

'Group' refers to a grouping of valid paragraphs among the paragraphs included in the document to be analyzed.

'Group score' means the average value of similarity scores of valid paragraphs included in the group.

'Machine comprehension target paragraph' is a paragraph that is transmitted as input data to the machine reading comprehension model, and means a valid paragraph and an additional paragraph, and excluded paragraphs are reflected when necessary.

The 'machine reading comprehension model' includes known machine reading comprehension models such as BERT.

Hereinafter, the present invention will be described with reference to the drawings. For reference, the drawings may be partially exaggerated in order to explain the features of the present invention. In this case, it is preferable to interpret in light of the whole purpose of this specification.

1 is a block diagram of a paragraph search and paragraph selection system for machine reading comprehension according to the present invention. 2 is a detailed configuration diagram of a paragraph search unit according to the present invention.

The present invention relates to a system for searching for and selecting paragraphs to perform machine reading comprehension, performed by a computing device using a database in which documents to be searched are stored and a control server having an arithmetic function.

A paragraph search and paragraph selection system for machine reading comprehension according to the present invention includes a question input unit 100 into which a preset question is input; a paragraph search unit 200 that searches for and selects a paragraph by using a similarity between an input question and a paragraph belonging to a search target document; and a paragraph transmission unit 300 that transmits the machine reading comprehension target paragraph selected by the paragraph search unit 200 to the machine reading comprehension model according to the transmission order.

The paragraph search unit 200 according to the present invention includes a valid paragraph unit 210 that calculates a similarity score between a question and a paragraph and selects a valid paragraph; a paragraph grouping unit 220 that groups valid paragraphs and calculates group scores; a machine reading comprehension target section 230 for selecting an analysis target document using a group score and additionally selecting a machine reading target paragraph within the analysis target document; and a transmission order calculation unit 240 that calculates the transmission order of the machine-reading target paragraph.

Hereinafter, the effective paragraph unit 210 according to the present invention and its sub-components, the similarity calculation unit 211 and the effective paragraph selection unit 212, will be described.

The effective paragraph unit 210 according to the present invention may calculate a similarity score between a question and a paragraph and select a valid paragraph.

The effective paragraph unit 210 may include a similarity calculation unit 211 that extracts all paragraphs included in the search target document and calculates a similarity score between each extracted paragraph and an input search question.

In the similarity calculation unit 211 according to the present invention, the similarity score can be calculated through vector-based cosine similarity or keyword-centered similarity. It is also possible to use other scoring algorithms.

Table 1 below shows an example of calculating a similarity score.

The effective paragraph unit 210 sorts each paragraph in the order of the similarity score calculated by the similarity calculation unit 211, and selects a paragraph having a similarity score higher than a preset value or a preset number (A) of paragraphs in order of similarity score. A valid paragraph selector 212 for selection may be provided.

Table 2 below shows an example of extracting the top 600 paragraphs by arranging similarity scores in descending order.

As an example, a first embodiment in which all paragraphs having a similarity score of 90 or more are selected as valid paragraphs is possible. However, in the first embodiment, the number of valid paragraphs selected may not be constant.

As another example, a second embodiment in which 600 paragraphs are selected by arranging them in order of similarity score is also possible. In this case, the lowest similarity score of 600 paragraphs may be, for example, less than 90 points, or the lowest similarity score may exceed 95 points, for example.

As another example, a third embodiment in which the first embodiment and the second embodiment are merged and implemented may also be possible. For example, an embodiment in which a predetermined number of paragraphs are selected with a predetermined similarity score or higher may be possible.

Hereinafter, the paragraph grouping unit 220 and the paragraph grouping unit 221 and the group score calculation unit 222, which are sub-components, according to the present invention will be described.

The paragraph grouping unit 220 according to the present invention may group valid paragraphs and calculate group scores.

The paragraph grouping unit 220 may include a paragraph grouping unit 221 that groups the paragraphs selected from the effective paragraph unit 210 according to documents to which each paragraph belongs.

The paragraph grouping unit 220 may include a group score calculation unit 222 that calculates a group score by calculating an average value of similarity scores of each paragraph grouped by the paragraph grouping unit 221 .

Table 3 below calculates the group score, that is, the average value (average similarity score) of the similarity scores of valid paragraphs included in the group after grouping by document ID, and ranks the top 50 groups with the highest average similarity score (document ranking) 1 Shows an example of selecting ~50th place.

In the present invention, 'grouping' means an operation of gathering paragraphs having the same document information (document ID).

After grouping the paragraphs, the similarity scores of the paragraphs in the group are averaged to calculate the group score, and the group scores can be used to rank the groups.

Hereinafter, the paragraph unit 230 subject to machine reading according to the present invention and its subordinate components, the document selection unit 231 and the paragraph addition unit 232, will be described.

The machine reading comprehension target paragraph unit 230 according to the present invention selects an analysis target document using a group score, and can additionally select a machine reading target paragraph within the analysis target document.

The machine reading target paragraph unit 230 includes a document selection unit 231 that sorts documents in the order of group scores calculated in the paragraph group unit 220 and selects a document having a group score equal to or higher than a preset value as an analysis target document. can be provided

The document selector 231 may select only up to the preset maximum number of documents when the number of documents having group scores equal to or greater than the preset value exceeds the preset maximum number of documents (B).

The document selection unit 231 sorts in descending order based on group scores. If it is greater than the preset maximum number of documents (B), only B groups with high group scores are extracted and the remaining groups are deleted.

In a typical conventional method, a paragraph or document is searched for, and the list is transmitted to the correct answer reasoning unit to find the correct answer. In this step, documents are grouped and their importance is compared.

A high average similarity score of a group or document means that there are many paragraphs similar to the correct answer in the document to which the group belongs. Conversely, documents with a low average similarity score are less likely to have correct answers, so it would be appropriate to exclude them from the target for the efficiency of correct answer inference.

In the present invention, when the maximum number of documents (B) is not set, there is a possibility of extracting all documents existing in the database. Accordingly, in the present invention, time can be reduced by setting the maximum number of documents and removing documents with a low probability of correct answers.

The machine reading target paragraph unit 230 sorts the similarity scores of the remaining paragraphs other than valid paragraphs in the document selected by the document selection unit 231, and adds a predetermined number of paragraphs as machine reading target paragraphs (232). ) can be provided.

Table 4 below shows an example in which additional paragraphs are expanded in the analysis target document to hold 40 machine reading target paragraphs for each analysis target document.

In each document, the number of paragraphs similar to the user's question may be expanded, and paragraphs may be added using the similarity score calculated by the similarity calculation unit 211, so that each document has a preset number of paragraphs.

What is selected in the valid paragraph selection unit 212 is called a 'valid paragraph', and what is added in the present paragraph addition unit 232 is called an 'additional paragraph'. Additional paragraphs may be added in order of similarity score. For example, if there are 30 valid paragraphs, 10 additional paragraphs can be added in order of similarity score, so that a total of 40 paragraphs can be set.

This process is a configuration for sorting each document (group) in the order of the similarity score with the question. In the previous step, documents that may have answers to the query have been found, and paragraphs for those documents that may have answers are added again. To this end, rankings are given to paragraphs for each document through similarity scores.

If a paragraph containing an actual correct answer is excluded due to a low similarity score in the overall paragraph score, the possibility of finding the correct answer is increased by expanding and adding additional paragraphs for each document in the paragraph addition section.

Through this configuration, the time to access the paragraph containing the correct answer can be shortened.

Meanwhile, if the number of 'valid paragraphs' exceeds a preset number (eg, 40), the excess is excluded from the analysis target, and the paragraphs excluded in this way are referred to as 'excluded paragraphs'. In the present invention, 'excluded paragraphs' may be determined in order of similarity scores.

Since there is no guarantee that the correct answer will always be derived from a paragraph with a high similarity score, it can be excluded from the machine reading comprehension target paragraph even though it is higher than the similarity score of a valid paragraph or additional paragraph belonging to another document to be analyzed. Also, even if the similarity score is relatively low, it can be included in the machine reading comprehension target paragraph.

Hereinafter, the transmission priority calculator 240 according to the present invention will be described.

The transmission order calculation unit 240 according to the present invention may calculate the transmission order (final score) of the paragraph to be read by machine.

The transmission order calculation unit 240 may calculate the transmission order (final score) according to the relational expression 1 below for each paragraph selected by the machine reading target paragraph section 230 .

[Relationship 1]

Transmission priority value = (paragraph rank value within group × 100) + group priority value

Transmission priority is a value that determines the priority order of transmission in machine reading comprehension. When transmission order scores are calculated and sorted in descending order, values with lower transmission order scores are transmitted to the machine reading comprehension model in order.

Table 5 below shows an embodiment before calculating the transmission order (final score) and sorting in ascending order.

Looking at Table 5, it can be seen that 40 paragraphs (paragraphs subject to machine reading) were selected for each of the document (CCC) ranked first in the group ranking (document ranking) to the document (YYY) ranked 50th. Here, within each document, 40 machine reading target paragraphs belonging to each document are sorted by transmission level (final score) according to the similarity score.

Table 6 below shows an example after calculating the transmission order (final score) and sorting in ascending order.

The similarity score rank within the 40 machine reading comprehension target paragraphs belonging to each document has the same status with respect to other documents.

For example, the first-ranked paragraph of each document has a transmission priority (final score) higher than that of the second-ranked paragraph of other documents, regardless of the group ranking (document ranking), according to relational expression 1. However, the transmission order (final score) between the same paragraph ranks in each document follows the group rank (document ranking).

Therefore, 40 paragraphs, which are the first in paragraph ranking of each document, are transmitted in the order of group ranking (document ranking). Next, 40 paragraphs, which rank second in paragraph ranking of each document, are also transmitted in order of group ranking (document ranking).

On the other hand, in the above transmission order, the machine reading comprehension target paragraphs may be input as input data to the machine reading comprehension model. The machine reading comprehension model finds paragraphs at a level that can be treated as correct answers. A level treated as a correct answer means that it belongs to a range of correct answers equal to or higher than a preset level value.

Here, if a correct answer belonging to the range of correct answers is derived from among the target paragraphs input to the machine reading comprehension model, an embodiment in which the remaining target paragraphs for machine reading not yet transmitted is no longer transmitted is possible.

In addition, even if an answer within the range of correct answers is derived, the machine reading comprehension model derives the final correct answer after transmitting the number of paragraphs corresponding to at least 50% of the selected machine reading comprehension target paragraph, e.g. An embodiment is also possible.

On the other hand, the present invention can be implemented as a paragraph search and paragraph selection method invention. Specifically, it can be implemented as a paragraph search and paragraph selection method for machine reading comprehension.

This method invention is substantially the same invention as the system invention described above, and the category of the invention is different. Therefore, the configuration common to the system invention will be replaced with the above description, and the following will focus on the gist of the method invention.

3 is a flowchart of a paragraph search and paragraph selection method for machine reading comprehension according to the present invention. 4 is a detailed flowchart of a paragraph search step according to the present invention.

The present invention is performed by a computing device using a database storing documents to be searched for and a control server having an arithmetic function, and searching for and selecting paragraphs to perform machine reading comprehension, the computing device using a question input unit (100) Step S100 in which a set question is input; Step S200 of the paragraph search unit 200 searching for and selecting a paragraph by using the similarity between the input question and the paragraph belonging to the search target document; and a step S300 in which the paragraph transmission unit 300 transmits the machine reading target paragraph selected by the paragraph search unit 200 to the machine reading comprehension model according to the transmission order.

Step S200 according to the present invention includes a step S210 in which the valid paragraph unit 210 calculates a similarity score between a question and a paragraph and selects a valid paragraph; Step S220 of grouping valid paragraphs by the paragraph grouping unit 220 and calculating group scores; Step S230 in which the machine reading comprehension section unit 230 selects the analysis target document using the group score and additionally selects the machine reading comprehension target paragraph within the analysis target document; and a step S240 in which the transmission order calculation unit 240 calculates the transmission order of the machine-reading target paragraph.

In the present invention, step S210 may include step S211 in which the similarity calculating unit 211 extracts all paragraphs included in the search target document and calculates a similarity score between each extracted paragraph and the input search question. .

In the present invention, the similarity score calculation in step S211 can be calculated through vector-based cosine similarity or keyword-centered similarity.

In the present invention, in step S210, the effective paragraph selection unit 212 arranges each paragraph in the order of similarity score calculated by the similarity calculation unit 211, and arranges paragraphs having a similarity score equal to or higher than a preset value or in order of similarity score. A step S212 of selecting a preset number of paragraphs may be provided.

In the present invention, step S220 may include a step S221 in which the paragraph grouping unit 221 groups the paragraphs selected from the effective paragraph unit 210 by document to which each paragraph belongs.

In the present invention, step S220 may include step S222 in which the group score calculator 222 calculates a group score by calculating an average value of similarity scores of each paragraph grouped by the paragraph grouping unit 221.

In the present invention, in step S230, the document selection unit 231 sorts the documents in the order of group scores calculated by the paragraph grouping unit 220 and selects documents having a group score equal to or higher than a preset value as the analysis target document. Step S231 may be provided.

In the present invention, in step S231, if the number of documents having group scores equal to or greater than the preset value exceeds the preset maximum number of documents, only up to the preset maximum number of documents can be selected.

In the present invention, in step S230, the paragraph button addition unit 232 sorts the similarity scores of the remaining paragraphs other than valid paragraphs in the document selected by the document selection unit 231, and selects a predetermined number of paragraphs as target paragraphs for machine reading. An additional step S232 may be provided.

In the present invention, in step S240, a transmission priority score may be calculated according to the relational expression 1 below for each paragraph to be read by machine selected in step S230.

[Relationship 1]

Transmission priority value = (paragraph rank value within group × 100) + group priority value

Meanwhile, the present invention may be implemented as a computer program. Specifically, the present invention may be combined with hardware and implemented as a computer program stored in a computer-readable recording medium to execute the paragraph search and paragraph selection method for machine reading comprehension according to the present invention by a computer.

Methods according to embodiments of the present invention may be implemented in a program form readable by various computer means and recorded on a computer readable recording medium. Here, the recording medium may include program commands, data files, data structures, etc. alone or in combination. Program instructions recorded on the recording medium may be those specially designed and configured for the present invention, or those known and usable to those skilled in computer software. For example, recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CDROMs and DVDs, and magneto-optical media such as floptical disks. optical media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of the program command may include a high-level language that can be executed by a computer using an interpreter, as well as a machine language generated by a compiler. These hardware devices may be configured to act as one or more software modules to perform the operations of the present invention, and vice versa.

The embodiments described in this specification and the accompanying drawings merely illustrate some of the technical ideas included in the present invention by way of example. Therefore, since the embodiments disclosed in this specification are intended to explain rather than limit the technical spirit of the present invention, it is obvious that the scope of the technical spirit of the present invention is not limited by these embodiments. All modified examples and specific examples that can be easily inferred by those skilled in the art within the scope of the technical idea included in the specification and drawings of the present invention should be construed as being included in the scope of the present invention.

100: question input unit
200: paragraph search unit 210: valid paragraph
211: similarity calculation unit 212: valid paragraph selection unit
220: paragraph grouping unit 221: paragraph grouping unit
222: group score calculation unit 230: machine reading comprehension section
231: document selection unit 232: paragraph addition unit
240: transmission priority calculation unit
300: paragraph transmission unit

Claims (21)

A system that is performed by a computing device using a database in which search target documents are stored and a control server having an arithmetic function, and searches and selects paragraphs to perform machine reading comprehension,
a question input unit into which a preset question is input;
a paragraph search unit for searching and selecting a paragraph by using a similarity between the input question and a paragraph belonging to a search target document; and
and a paragraph transmitter for transmitting the machine reading comprehension target paragraph selected in the paragraph search unit to the machine reading comprehension model according to transmission order,
The paragraph search section
a valid paragraph unit for calculating a similarity score between a question and a paragraph and selecting a valid paragraph;
a paragraph grouping unit for grouping valid paragraphs and calculating group scores;
a machine reading comprehension target section for selecting an analysis target document using a group score and additionally selecting a machine reading target paragraph within the analysis target document; and
A paragraph search and paragraph selection system for machine reading comprehension, comprising a transmission order calculation unit for calculating the transmission order of a paragraph targeted for machine reading comprehension. The method of claim 1,
The valid paragraph is
Extract all paragraphs included in the search target document;
A paragraph search and paragraph selection system for machine reading comprehension, comprising a similarity calculation unit for calculating a similarity score between each extracted paragraph and an input search question. The method of claim 2,
The paragraph search and paragraph selection system for machine reading comprehension, characterized in that the calculation of the similarity score in the similarity calculation unit can be calculated through vector-based cosine similarity or keyword-based similarity. The method of claim 2,
The valid paragraph is
Sort each paragraph in the order of the similarity score calculated by the similarity calculator,
A paragraph search and paragraph selection system for machine reading comprehension, characterized by comprising an effective paragraph selection unit for selecting a paragraph having a similarity score higher than a preset value or a preset number of paragraphs in order of similarity scores. The method of claim 1,
The paragraph grouping part
A paragraph search and paragraph selection system for machine reading comprehension, characterized in that a paragraph grouping unit grouping the paragraphs selected from the effective paragraph unit by document to which each paragraph belongs. The method of claim 5,
The paragraph grouping part
The paragraph search and paragraph selection system for machine reading comprehension, characterized by comprising a group score calculation unit for calculating a group score by calculating an average value of similarity scores of each paragraph grouped in the paragraph grouping unit. The method of claim 1,
The machine reading comprehension target section
Sort the documents in the order of group scores calculated in the paragraph grouping unit;
A paragraph search and paragraph selection system for machine reading comprehension, comprising a document selection unit for selecting a document having a group score equal to or higher than a pre-set value as an analysis target document. The method of claim 7,
The document selection section
If the number of documents with group scores higher than the preset value exceeds the preset maximum number of documents,
A paragraph search and paragraph selection system for machine reading comprehension, characterized in that only up to a preset maximum number of documents are selected. According to claim 7 or claim 8,
The machine reading comprehension target section
In the document selected in the document selection unit,
A paragraph search and paragraph selection system for machine reading comprehension, comprising a paragraph adding unit for sorting similarity scores of remaining paragraphs other than valid paragraphs and adding a preset number of paragraphs as machine reading target paragraphs. The method of claim 1,
The transfer priority calculator
A paragraph search and paragraph selection system for machine reading comprehension, characterized in that for each paragraph selected from the machine reading comprehension target paragraph, a transmission priority score is calculated according to the following relational expression 1.
[Relationship 1]
Transmission priority value = (paragraph rank value within group × 100) + group priority value A method of searching for and selecting a paragraph to perform machine reading, which is performed by a computing device using a database in which search target documents are stored and a control server having an arithmetic function, the computing device comprising:
Step S100 of inputting a preset question to a question input unit;
Step S200 of searching for and selecting a paragraph by using the similarity between the input question and the paragraph belonging to the search target document by the paragraph search unit; and
A step S300 in which the paragraph transmission unit transmits the machine reading comprehension target paragraph selected in the paragraph search unit to the machine reading comprehension model according to the transmission order;
The step S200 is
Step S210 of calculating a similarity score between the question and the paragraph by the valid paragraph and selecting a valid paragraph;
Step S220 of grouping valid paragraphs by the paragraph grouping unit and calculating group scores;
Step S230 of selecting an analysis target document by using a group score by the machine reading target paragraph and additionally selecting a machine reading target paragraph within the analysis target document; and
A paragraph search and paragraph selection method for machine reading comprehension, characterized in that it comprises a step S240 in which the transmission order calculation unit calculates the transmission order of the target paragraph for machine reading comprehension. The method of claim 11,
The step S210 is
The similarity calculation unit extracts all paragraphs included in the search target document,
A paragraph search and paragraph selection method for machine reading comprehension, comprising a step S211 of calculating a similarity score between each extracted paragraph and an input search question. The method of claim 12,
The paragraph search and paragraph selection method for machine reading comprehension, characterized in that the calculation of the similarity score in step S211 can be calculated through vector-based cosine similarity or keyword-based similarity. The method of claim 12,
The step S210 is
The effective paragraph selector includes a step S212 of arranging each paragraph in the order of the similarity score calculated by the similarity calculation unit and selecting a paragraph having a similarity score equal to or higher than a preset value or a preset number of paragraphs in order of similarity score. Paragraph search and paragraph selection method for machine reading comprehension. The method of claim 11,
The step S220 is
A paragraph search and paragraph selection method for machine reading comprehension, characterized by comprising a step S221 of grouping the paragraphs selected from the valid paragraphs by a paragraph grouping unit according to documents to which each paragraph belongs. The method of claim 15
The step S220 is
The paragraph search and paragraph selection method for machine reading comprehension, characterized by comprising a step S222 of calculating a group score by calculating an average value of the similarity scores of each paragraph grouped by the group score calculation unit in the paragraph grouping unit. The method of claim 11,
The step S230 is
A paragraph search for machine reading comprehension comprising a step S231 of a document selection unit sorting documents in the order of group scores calculated in the paragraph group unit and selecting a document having a group score equal to or higher than a preset value as an analysis target document. and paragraph selection methods. The method of claim 17
The step S231 is
A paragraph search and paragraph selection method for machine reading comprehension, characterized in that when the number of documents having group scores equal to or greater than the preset value exceeds the preset maximum number of documents, only the preset maximum number of documents is selected. According to claim 17 or claim 18,
The step S230 is
A paragraph for machine reading comprehension comprising a step S232 of adding a predetermined number of paragraphs as target paragraphs by arranging similarity scores of remaining paragraphs other than valid paragraphs in the document selected by the document selection unit by the paragraph addition unit. Navigation and paragraph selection methods. The method of claim 11,
The step S240 is
A paragraph search and paragraph selection method for machine reading comprehension, characterized in that for each paragraph to be machine reading comprehension selected in step S230, a transmission priority score is calculated according to the following relational expression 1.
[Relationship 1]
Transmission priority value = (paragraph rank value within group × 100) + group priority value A computer program stored in a computer-readable recording medium combined with hardware to execute the paragraph search and paragraph selection method for machine reading comprehension according to claim 11 by a computer.

Images