KR20240052394A - Device and method for generating korean commonsense reasoning dataset - Google Patents

Abstract

A data set generating device is disclosed. The dataset creation device includes a data collection unit that collects text data, a preprocessor that performs a preprocessing operation on the text data, and a dataset extraction unit that builds a dataset from the preprocessed text data, and extracts the dataset. The dataset is created as a result of preprocessing, and the dataset is constructed by extracting concept information set sentence pairs that satisfy predetermined conditions from concept information set sentence pairs, each of which consists of a concept information set and a sentence corresponding to the concept information set.

Description

Korean common sense reasoning ability data generation device and method {DEVICE AND METHOD FOR GENERATING KOREAN COMMONSENSE REASONING DATASET}

The present invention relates to natural language generation in the field of Natural Language Process (NLP) research, and to a method of automatically building a dataset to evaluate the common sense reasoning ability of a deep learning-based language model.

Common sense reasoning is the ability to infer general knowledge related to concepts or given conditions that did not appear during the learning stage. CommonsenseQA constructed a dataset with questions close to common sense based on the concept of ConceptNet, and classified and analyzed the common sense reasoning skills required for each question into categories. VQA proposed a task that tests the model's ability to answer questions when given an image and a question. It was verified through human verification that the task requires common sense reasoning that requires not only the given information but also concepts that did not appear in learning.

In recent Natural Language Process (NLP) research, various common sense inference datasets have been released. The Cosmos QA dataset was constructed based on facts that are not externally revealed in the context. The CoS-E dataset tried to strengthen the model's common sense inference when learning the dataset by adding human explanations of common sense based on CommonsenseQA. The XCOPA dataset attempted to construct a dataset related to common sense and at the same time alleviate differences in common sense that may arise due to linguistic and cultural differences. There is no dataset for generation-based common sense inference in the Korean language, and there is no improvement research based on it. Therefore, the present invention aims to build a system that can create a new text generation dataset and suggest the future direction of Korean natural language processing research.

Republic of Korea Patent No. 10-2409667 (announced on June 16, 2022) Republic of Korea Patent Publication No. 2022-0077311 (published on 2022.06.09.) Republic of Korea Patent Publication No. 2022-0064966 (published on May 19, 2022)

Korean natural language processing research still has two problems: (1) lack of natural language generation research and (2) lack of sentence generation ability based on general knowledge. First, since most of the Korean language is focused on natural language understanding tasks, there are no datasets or evaluation standards to quantitatively evaluate the performance of language models for natural language generation results. Therefore, it is difficult to conduct improvement research compared to natural language understanding due to the lack of clear guidelines or verification of the language model's creation ability. Second, the Korean language model (not only lacks datasets and evaluation standards to verify performance) also has significant difficulties in generating sentences based on simple common sense knowledge.

A dataset generation device according to an embodiment of the present invention includes a data collection unit that collects text data, a preprocessor that performs a preprocessing operation on the text data, and a dataset extractor that builds a dataset from the preprocessed text data. The dataset extractor extracts concept information set sentence pairs that satisfy predetermined conditions from concept information set sentence pairs that are generated as preprocessed results and each consists of a concept information set and a sentence corresponding to the concept information set. Build the dataset.

The present invention presents a method to derive and evaluate the common sense inference ability of a Korean language generation model through the Korean common sense inference dataset. Through the dataset, the generative model can generate sentences containing common sense inference results based on the Korean language. In addition, through an evaluation method based on natural language understanding, it is possible to evaluate and comparatively analyze the common sense reasoning ability and sentence reconstruction ability for the generative model, rather than comparatively analyzing the Korean generative model. Furthermore, it can suggest a new direction and contribute to the development of natural language generation research by improving the natural sentence generation and common sense reasoning abilities of the Korean language generation model.

In order to more fully understand the drawings cited in the detailed description of the present invention, a detailed description of each drawing is provided.
Figure 1 is a diagram for explaining the common sense inference results of the generation model through the dataset proposed in the present invention.
Figure 2 shows an example of generating a sentence by combining the concept information set of the proposed evaluation dataset.
Figure 3 is a functional block diagram of a dataset generation device for a Korean language model, according to an embodiment of the present invention.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in this specification are merely illustrative for the purpose of explaining the embodiments according to the concept of the present invention. They may be implemented in various forms and are not limited to the embodiments described herein.

Since the embodiments according to the concept of the present invention can make various changes and have various forms, the embodiments will be illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes all changes, equivalents, or substitutes included in the spirit and technical scope of the present invention.

Terms such as first or second may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component, for example, without departing from the scope of rights according to the concept of the present invention, a first component may be named a second component and similarly a second component The component may also be named a first component.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to that other component, but that other components may also exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between. Other expressions that describe the relationship between components, such as "between" and "immediately between" or "neighboring" and "directly adjacent to" should be interpreted similarly.

The terms used in this specification are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in this specification, but are not intended to indicate the presence of one or more other features. It should be understood that it does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms as defined in commonly used dictionaries should be interpreted as having meanings consistent with the meanings they have in the context of the related technology, and unless clearly defined in this specification, should not be interpreted in an idealized or overly formal sense. No.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached to this specification. However, the scope of the patent application is not limited or limited by these examples. The same reference numerals in each drawing indicate the same members.

In the present invention, we propose a dataset (named KommonGen) in which a (Korean) language generation model generates sentences based on common sense reasoning ability.

Through the proposed dataset (KommonGen), the performance of the model can be evaluated for Korean sentences created by combining conceptual information consisting of objects and actions commonly seen in daily life. In order to generate a correct sentence, the model must have the ability to create a single sentence by combining each conceptual information and prior knowledge of the general relationship between each conceptual information. Additionally, the sentences generated by the model must include all given conceptual information and must be natural sentences that comply with Korean grammar rules.

For example, if conceptual information such as {“beach”, “surging”, “waves”, “seen”} in Figure 1 is given in a random order, based on the conceptual information, a person may say, “The beach where the waves are washing up is It is possible to create a short sentence that can be accepted by common sense by considering the relationship between each conceptual information, such as “I see it.” Likewise, a Korean-based generative model must reflect general knowledge and generate sentences that combine conceptual information.

In addition, the dataset (KommonGen) presented in the present invention allows quantitative performance evaluation of the sentence composition ability and common sense reasoning ability of the Korean language generation model. Furthermore, based on the evaluation criteria, the Korean language generation ability of arbitrary language models (e.g., KoGPT21 and KoBART2, which are public large-scale Korean pre-trained language models, and mBART-50, which is a multilingual language model) can be compared and analyzed. The dataset and evaluation method proposed in this invention are expected to contribute to the development of Korean-based natural language generation research.

Representative language models that have undergone pre-training based on a large Korean corpus include KoGPT2 and KoBART. KoGPT2 is a model released by SKT-AI by learning the GPT2 model, which consists of only the Transformer decoder, with over 40GB of Korean text. The model consists of 12 layers, a head, and 125M model parameters. KoBART was also released by SKT-AI, and the BART model using Transformer's encoder-decoder structure was learned with over 40GB of Korean text. KoBART configured training for Korean by applying the Text Infilling method used in pre-training of the BART model. The model consists of 6 layers, 16 heads, and 124M model parameters. In the present invention, in order to compare and analyze the performance of Korean-based generation models, the mBART-50 model, which learned 50 multilingual corpora including Korean, was added to the test subject. mBART-50 is a model that adds bidirectional fine-tuning training for each language pair to mBART, which was pre-trained for 25 multiple languages, and expanded the range to 50 multiple languages. The model learned more than 54GB of Korean text and consists of 24 layers, 16 heads, and 610M model parameters.

However, the scope of the present invention is not limited to the type of (Korean) production model described above, and various language production models not described may be used depending on the embodiment.

In the present invention, the dataset was constructed in a way that was not overly limited to a specific field or used technical terms to generate sentences based on general knowledge. Therefore, in the cognitive technology-visual intelligence field of AI-HUB, MS-COCO (T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick , “Microsoft coco: Common objects in context,” European conference on computer vision, pp. 740-755, 2014.) The caption information in the dataset was first translated into Korean and error correction was performed.

As shown in Table 1, in order to extract descriptive sentences in the form of sentences rather than phrases from a total of 576,704 image caption data translated into Korean, 163,629 sentences ending with the final ending “-da.” were selected. The selected sentences were analyzed using Mecab( T. KUDO, “Mecab: Yet another part-of-speech and morphological analyzer,” http://mecab. sourceforge.net/, 2005.) morpheme segmentation and part-of-speech tagging were performed.

For the parts of speech used, nouns and verbs (only) were used to organize conceptual information with as objective content as possible. In the case of nouns, common nouns, proper nouns, dependent nouns, unit nouns, numerals, and pronouns are included. In the case of verbs, they include verbs, adjectives, auxiliary verbs, and affirmative/infinitive verbs. However, in the case of Korean, as endings must be considered when extracting verbs used in sentences, conceptual information was organized in a form including prefinal endings, final endings, conjunctive endings, and prepositional endings. As a result of part-of-speech tagging, the set of concept information extracted from each sentence was designed to include at least one verb and two nouns, considering the typical Korean descriptive sentence types of “subject,” “object,” and “predicate.”

The present invention used the concept information set and sentence pair that satisfied the minimum inclusion conditions as a dataset when it had a total of 3 to 6 concept information. The training dataset is 4,354/27,448/56,249/56 concept set-sentence pairs containing between 3 and 6 concept information, and the validation dataset is 201/309 containing between 3 and 6 concept information. /488/It consists of two concept sets and sentence pairs. The evaluation dataset contains conceptual information that was not learned in the training data, and the common sense reasoning ability and sentence construction ability of the generating model were evaluated by increasing the proportion of more complex composition methods. For this purpose, each concept information set in the evaluation dataset included at least one concept information that did not exist in the training dataset, and consisted of up to six concept set-sentence pairs with a 5.1% increase in the concept information rate. However, in addition, sentences containing conceptual information that does not appear in the training dataset and are used very less frequently have awkward expressions and grammatical structures that are not used relatively often. Therefore, binary classification of awkward sentences and natural sentences can be additionally performed using three judges whose native language is Korean and who have higher education or higher. As a result, among the 2,000 concept information-sentence pairs previously constructed for evaluation, 1,432 concept information-sentence pairs from which 568 awkward sentences were removed with majority agreement were used as the final evaluation dataset.

[Table 1]

Below, the evaluation indicators are explained.

There are two ways to evaluate the generation model, using coverage scores based on n-gram overlap of sentences and whether or not concept information is included.

First, to evaluate the structural integrity of the generated sentences of the model, an n-gram-based evaluation index, BLEU (K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” Proceedings of the 40th annual meeting on association for computational linguistics, pp 311-318, 2002), METEOR (S. Banerjee and A. Lavie, “Meteor: An automatic metric for mt evaluation. with improved correlation with human judgments,” Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summary, pp. 65-72, 2005.), and ROUGE (C.-Y. LIN, “Rouge: A package for automatic evaluation of summaries,” Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, Barcelona, Spain, pp. 74-81, 2004. This evaluation method reflects some of the general word order and grammatical features of Korean descriptive sentences through n-gram overlap and calculates the degree of agreement with the correct sentence written by a person.

Second, the generation result is segmented into morpheme units and expressed as a coverage percentage to indicate how much each sentence contains a given concept information. Coverage is an indicator that indicates whether the conceptual information that the generation model must include is reorganized into sentences according to intent.

Additionally, in order to standardize the values calculated from each performance indicator, the Mecab-based morpheme segmentation method is used for the generated results excluding ROUGE. In the case of ROUGE, considering that the evaluation of Korean segmentation results is not complete, each model calculates the score by mapping index values from the dictionary built through the tokenizer.

Below, the experiment and its results are described.

Experiments on the generative model were conducted on the KoGPT2 model, which consists of only a transformer decoder, and the KoBART and mBART-50 models, which have an encoder and decoder as a sequence-to-sequence model. The deep learning framework for the experiment was Pytorch 1.9 (A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems, Vol. 32, pp. 8026-8037, 2019. and Huggingface Transformers (https://github. com/huggingface/transformers) was used. Experimental hyperparameters for KoGPT2 are batch 4 with gradient accumulation, initial learning rate 5 × 10^(-5), warmup steps 400, AdamW optimizer (β ₁ = 0.9, β ₂ = 0.999, ε = 1e - 8), block I set size to 128, seed to 42, and epochs to 5. In the case of KoBART and mBART-50, batch 16 with gradient accumulation, initial learning rate 5 × 10^(-5), warmup steps 400, AdamW optimizer (β ₁ = 0.9, β ₂ = 0.999, ε = 1e - 8), We set epochs 5, max source length 64, max target length 256, and src/tgt language ko_KR (mBART-50 only). Additionally, for the experiment, the present invention used computer resources equivalent to 128GB of RAM, 18-core Intel Xeon Gold 6230 CPU, and NVIDIA A6000 (48GB) GPU.

To generate sentences based on common sense reasoning for given conceptual information, the model is trained to reconstruct a set of input concepts x ∈ X and sentences y ∈ Y composed of those concepts. Equation 1 below reflects the training process of the generative model.

[Equation 1]

of It means the maximum generated length for y sentences, and the equation is calculated as a continuous conditional probability for the input sequence (Y. Bengio, R. Ducharme, P. Vincent, and C. Janvin, “A neural probabilistic language model, “The journal of machine learning research, Vol. 3, pp. 1137-1155, 2003. refers to the hidden vector returned by the model, and e(·) represents the embedding value of the input token.

[Equation 2]

Afterwards, as shown in Equation 2, the parameter θ of the model for KommonGen dataset D is learned to maximize the log likelihood of the prediction probability for the next token in autoregressive sentence generation.

The learned model generates sentences using beam size 10, max sequence length 30, min sequence length 10, and n-gram size 3 as decoding hyperparameters. The 10 sentences generated as large as the beam size are ranked according to the number of segments they contain. The ranking of candidate sentences is rearranged in descending order according to the number of concept information, and the top one sentence is used as the final result.

[Table 2]

Table 2 shows the results of a comparative analysis of the performance of the generation model using the KommonGen dataset using the presented evaluation criteria. In all evaluation indicators, the sequence-to-sequence models KoBART and mBART-50 show higher performance than KoGPT2, which consists of only a decoder. This result is because the sequence-to-sequence model has an encoder that can understand the context of the input, so it has an advantage in common sense reasoning in the process of reconstructing concept information. On the other hand, KoGPT2, which consists of only a decoder, shows low performance as the prompt-based generation method does not sufficiently narrow the gap between the pre-training stage of unidirectionally learning a large corpus and fine-tuning training. Coverage also shows that KoGPT2 and KoBART generate sentences with about 23% and 10% of the given conceptual information lost, respectively. These performance indicators indicate that the two models lack common sense reasoning ability and have difficulty constructing sentences for given conditions.

In particular, mBART-50, which was pre-trained on 50 multilingual corpora including Korean, shows the highest performance in evaluations based on n-gram base and degree of conceptual information matching. These results appear to have been influenced by the fact that more than 14GB more Korean corpus was used for pre-training than the model learned with a single Korean corpus and that the model parameters were approximately 5 times larger. And this is because the two-way training method introduced by mBART-50 in the process of doubling the existing mBART's 25 language resources greatly improved the model's understanding of Korean, which is a medium/low resource. Furthermore, the back translation method used to augment multilingual data appears to have had a positive impact on reconstructing the KommonGen evaluation dataset, which consists of sentences from the MS-COCO dataset translated into Korean.

Figure 2 shows an example of generating a sentence by combining a set of concept information from the KommonGen evaluation dataset. First, KoGPT2 generated sentences that omitted the given conceptual information “cafe,” and used somewhat awkward expressions and Korean grammar. Unstable sentence structure and grammatical errors are the reasons why KoGPT2 shows low scores in n-gram-based measurements, and appear to be the result of insufficient common sense inference for given conceptual information. KoBART generated sentences containing all conceptual information, but it ended up repeating the same conceptual information more than necessary. These results indicate that further improvement is needed in combining given conceptual information to form sentences. mBART-50 generated sentences with a natural and stable structure compared to other models. However, it is a concise and easy expression compared to sentences written by humans, and it seems that improvements are needed to be able to generate sentences based on various words and rich expressions.

Figure 3 is a functional block diagram of a dataset generation device for a Korean language model, according to an embodiment of the present invention.

Referring to FIG. 3, the dataset creation device 100, which may be referred to as a language model evaluation device, etc., includes a data collection unit 110, a preprocessing unit 120, and a dataset extraction unit 130. Depending on the embodiment, the dataset generating device 100 may further include at least one of a learning unit 140, an evaluation unit 150, and a storage unit 160. The dataset generating device 100 may be implemented as a computing device including at least a processor and/or memory. Computing devices may include personal computers (PCs), laptop computers, note PCs, servers, etc.

The data collection unit 110 can collect (text) data through a predetermined wired or wireless communication network. To this end, the data collection unit 110 may collect (text) data from at least one server that provides text data through crawling.

As an example, the data collection unit 110 may collect caption information of a certain data set (eg, MS-COCO data set) through a wired or wireless communication network. As another example, the data collection unit 110 may collect text data from a server that holds and provides text data, such as Wikipedia. Data collected by the data collection unit 110 may be stored in the storage unit 160.

The preprocessor 120 may perform a preprocessing operation on the collected (text) data. Specifically, the preprocessor 120 can extract only the comment text from the collected data. To this end, the preprocessor 120 can extract only sentences whose endings end with “-da.”

Additionally, the preprocessor 120 may perform morpheme analysis and/or part-of-speech tagging operations. For example, Mecab-style morpheme segmentation can be performed using KoNLPy, a Korean morpheme analysis package. According to one embodiment, the preprocessor 120 may tag only body language and verbs. Tagged phrases and verbs constitute a set of conceptual information. Additionally, the concept information set may be configured to include pre-final endings, final endings, conjunctive endings, and pre-verbal endings for the corresponding verb. Accordingly, concept information set sentence pairs, each including a concept information set and a sentence, can be constructed.

Additionally, the preprocessor 120 may perform machine translation on text data in different languages, convert it into Korean, and then perform the above-described preprocessing operation.

Concept information set sentence pairs constructed by the preprocessor 120 may be stored in the storage unit 160.

The dataset extractor 130 may construct a dataset from concept information set sentence pairs. The data set may mean at least one of a training data set, a validation data set, and an evaluation data set.

Specifically, the dataset extractor 130 has a predetermined number of concept information (i.e., includes a predetermined number (e.g., 3 to 6) of verbs and/or nouns from the concept information set sentence pairs. ) A dataset can be constructed by extracting concept information set sentence pairs.

According to an embodiment, the concept information sets included in the learning dataset, validation dataset, and evaluation dataset may be constructed so as not to overlap. As an example, the evaluation dataset may be constructed to include (only) concept information that was not learned in the training dataset (i.e., not included in the training dataset). Additionally, the ratio of verbs and/or nouns included in the evaluation dataset may be constructed to be higher than the ratio of verbs and/or nouns included in the learning dataset.

The dataset constructed (or extracted) by the dataset extraction unit 130 may be stored in the storage unit 160.

The learning unit 140 may train at least one pre-trained (Korean) language model that is stored in advance using the generated training dataset. Learning can be done to reconstruct the corresponding sentence y from a set of input concepts.

At least one language model learned by the learning unit 140 may be stored in the storage unit 160.

The evaluation unit 150 may perform an evaluation operation on at least one language model learned using training data. Specifically, the evaluation unit 150 inputs the concept information set of the evaluation data set into each of the language models, and compares the output sentences (reconstructed sentences) with the sentences included in the concept information set sentence pair, so that each language model Evaluation indicators can be derived. The evaluation index may include at least one of ROUGE-2, ROUGE-L, BLEU 3, BLEU 4, METEOR, and Coverage. The output sentence of the language model selects the sentence that contains the largest set of input concept information among a plurality of sentences generated (reconstructed) by each language model (e.g., 10 sentences generated by the size of the beam), and , the language model can be evaluated based on the selected sentence.

The evaluation results by the evaluation unit 150 may be stored in the storage unit 160.

The storage unit 160 may store an operating system (OS), program, source code, etc. for operation of the data set generating device 100. Additionally, at least one pre-trained (Korean) language model may be stored in the storage unit 160. In addition, the storage unit 160 includes data collected by the data collection unit 110, data preprocessed by the preprocessor 120, a dataset extracted by the dataset extraction unit 130, and learning unit 140. Results learned by, that is, the learned language model, evaluation results by the evaluation unit 150, etc. may be stored.

The device described above may be implemented as a hardware component, a software component, and/or a set of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an Arithmetic Logic Unit (ALU), a Digital Signal Processor, a microcomputer, a Field Programmable Array (FPA), It may be implemented using one or more general-purpose or special-purpose computers, such as a Programmable Logic Unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device may include multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes specially configured hardware devices to store and execute program instructions, such as magneto-optical media, ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The present invention has been described with reference to the embodiments shown in the drawings, but these are merely exemplary, and those skilled in the art will understand that various modifications and other equivalent embodiments are possible therefrom. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent. Therefore, the true scope of technical protection of the present invention should be determined by the technical spirit of the attached registration claims.

100: Dataset creation device
110: data collection unit
120: preprocessing unit
130: Dataset extraction unit
140: Learning Department
150: Evaluation department
160: storage unit

Claims (7)

A data collection unit that collects text data;
a pre-processing unit that performs a pre-processing operation on the text data; and
It includes a dataset extraction unit that builds a dataset from preprocessed text data,
The dataset extractor extracts concept information set sentence pairs that satisfy predetermined conditions from concept information set sentence pairs that are generated as preprocessed results, each of which consists of a concept information set and a sentence corresponding to the concept information set, and sets the dataset. To build,
Dataset creation device. According to paragraph 1,
The preprocessor,
Extracting descriptive sentences from the text data,
Performing morpheme segmentation and part-of-speech tagging for each of the above descriptive sentences,
Dataset creation device. According to paragraph 2,
The preprocessor tags the part of speech for the declarative sentence and the verb for each of the descriptive sentences,
Dataset creation device. According to paragraph 1,
The concept information set includes pre-final endings, final endings, conjunctive endings, and prepositional endings for verbs included in the concept information set,
Dataset creation device. According to paragraph 1,
The data set extractor constructs concept information set sentence pairs containing a predetermined number of verbs and phrases from among the concept information set sentence pairs as the data set,
Dataset creation device. According to paragraph 1,
The dataset generating device further includes a learning unit that trains at least one pre-trained language model using training data included in the dataset.
Dataset creation device. According to clause 6,
The dataset generating device further includes an evaluation unit that evaluates the performance of at least one language model learned by the learning unit using an evaluation dataset included in the dataset,
Dataset creation device.

Images