JP7343566B2 - Data generation method, computer device, and computer program using language models - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act Website publication date April 18, 2021 Website address https://arxiv. org/abs/2104.08826 Publisher Yoo Kang-min, Park Dong-joo, Kang Jae-wook, Lee Sang-woo, Park Woo-myung, Website publication date May 25, 2021 Website address https://tv. naver. com/v/20349649 Published by Yoo Kang-min, Park Dong-joo, Kang Jae-wook, Lee Sang-woo, Park Woo-myung

The following description relates to techniques for generating text data.

In order to build an NLP (natural language processing) model, data necessary for model learning is secured through data design and data collection in advance. After learning a model based on the acquired data, we repeat improvements to data design and model learning while evaluating performance.

Since the advent of PLM (pretrained language model), a method of performing fine tuning using an NLP model that solves a specific domain or task based on PLM has been applied.

Additionally, data augmentation techniques have been used to efficiently learn NLP models.

As an example, a technology such as EDA (easy data augmentation), which generates text data by superficially manipulating phrases for model learning, produces sentences with poor grammar, and the meaning of the sentences may not match the intended meaning. There may be a large difference between the two.

As another example, a technology such as back-translation that uses a machine translation model to generate similar sentences utilizes a translation model learned from a translation corpus with specific linguistic characteristics. The disadvantage is that the generated writing style cannot reflect the linguistic characteristics of existing text data (to be generated), making general use impossible.

To provide a data expansion technology that can derive natural language generation results using a large-scale language model learned from a corpus of diverse language characteristics, and can extract new data from the derived generation results.

A data generation method performed by a computer device, the computer device including at least one processor configured to execute computer readable instructions contained in a memory, the data generation method comprising: one processor constructs a prompt to be an input sentence for a language model using original data; and the at least one processor inputs the prompt to the language model and generates new data from the language model. and generating label information for the new data.

According to one aspect, the generating step may include generating a soft label indicating a label distribution using a probability distribution for natural words corresponding to a label in the prompt.

According to another aspect, the data generation method may further include the step of the at least one processor selecting the original data from a training data set depending on the semantic diversity of the text.

According to another aspect, the selecting step may select the original data as many as the number of label types from the learning data set.

According to another aspect, the configuring step may configure the prompt in a format that includes a text type and a label type.

According to another aspect, the configuring step may configure the prompt in a format including a text type, a label type, and a label position verbizer.

According to another aspect, the step of configuring may process the original data and configure the prompt in a natural language form that is the same as the original data.

According to another aspect, the configuring step may configure the prompt by combining a task specification, the original data, and a prompt template.

According to another aspect, the generating step is an auto-regressive method in which a probability distribution of a previous token is transmitted as an input to a next token for a natural word corresponding to the text and label in the prompt. The method may include extracting the new data and the label information using a method.

According to still another aspect, the extracting step includes determining the probability of a top part of the probability distribution of the previous token as the input of the next token by beam search using a heuristic. may include the step of communicating as

A computer program for causing the computer device to execute the data generation method is provided.

A computing device including at least one processor configured to execute computer readable instructions contained in memory, the at least one processor utilizing source data to serve as input sentences for a language model. A computer device is provided that processes the steps of configuring a prompt, inputting the prompt into the language model, and generating new data and label information for the new data from the language model.

According to embodiments of the present invention, data collection is performed by generating new data by transforming or extending original data using a language model, and by transferring knowledge recognized by the language model through the new data. It is possible to significantly reduce the number of input man-hours and achieve the effect of reducing the weight of the model.

1 is a block diagram for explaining an example of the internal configuration of a computer device in an embodiment of the present invention. FIG. FIG. 2 is a diagram for explaining the concept of text expansion using a large-scale language model in an embodiment of the present invention. 1 is a flowchart illustrating an example of a data generation method that can be executed by a computer device according to an embodiment of the present invention. FIG. 3 is a diagram for explaining a process of configuring input prompts for a language model in an embodiment of the present invention. FIG. 3 is a diagram for explaining a data expansion process in an embodiment of the present invention. FIG. 3 is a diagram for explaining a data expansion process in an embodiment of the present invention. FIG. 3 is a diagram for explaining a process of generating a new sentence and label information using a language model in an embodiment of the present invention. FIG. 3 is a diagram for explaining a process of generating distributed soft labels in an embodiment of the present invention. FIG. 2 is a diagram showing an example of original data and an example of new data generated from the original data in an embodiment of the present invention.

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

Embodiments of the present invention relate to text data generation technology using a language model.

Embodiments including matters specifically disclosed in this specification utilize a large-scale language model to achieve consistency with the characteristics of existing text data, as well as high grammaticality and naturalness. Can generate sentences. Furthermore, high quality label information for the relevant text can also be generated.

FIG. 1 is a block diagram showing an example of a computer device in an embodiment of the present invention. For example, the data generation system according to the embodiment of the present invention may be realized by the computer device 100 shown in FIG.

As shown in FIG. 1, the computer device 100 includes a memory 110, a processor 120, a communication interface 130, and an input/output interface 140 as components for executing the data generation method according to the embodiment of the present invention. good.

Memory 110 is a computer readable storage medium and may include permanent mass storage devices such as random access memory (RAM), read only memory (ROM), and disk drives. Here, a permanent large capacity storage device such as a ROM or a disk drive may be included in the computer device 100 as a separate permanent storage device separate from the memory 110. Additionally, an operating system and at least one program code may be recorded in the memory 110. Such software components may be loaded into memory 110 from a computer-readable storage medium separate from memory 110. Such other computer-readable recording media may include computer-readable recording media such as floppy drives, disks, tapes, DVD/CD-ROM drives, memory cards, and the like. In other embodiments, software components may be loaded into memory 110 through communication interface 130 that is not a computer-readable storage medium. For example, software components may be loaded into memory 110 of computing device 100 based on a computer program installed by a file received over network 160.

Processor 110 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to processor 120 by memory 110 or communication interface 130. For example, processor 120 may be configured to execute instructions received according to program code recorded on a storage device, such as memory 110.

Communication interface 130 may provide functionality for computing device 100 to communicate with other devices and each other via network 160. As an example, requests, instructions, data, files, etc. generated by the processor 120 of the computer device 100 according to a program code recorded in a storage device such as the memory 110 are transmitted to others via the network 160 under the control of the communication interface 130. may be transmitted to the device. Conversely, signals, instructions, data, files, etc. from other devices may be received by the computing device 100 via the network 160 and through the communication interface 130 of the computing device 100 . Signals, instructions, data, etc. received through communication interface 130 may be communicated to processor 120 and memory 110, files, etc. may be transferred to a storage medium (such as a persistent storage device as described above) that computing device 100 may further include. May be recorded.

The communication method is not limited, and is not limited to communication methods that utilize communication networks that can be included in the network 160 (for example, mobile communication networks, wired Internet, wireless Internet, and broadcasting networks), as well as communication methods that utilize short distances between devices. Wireless communications may also be included. For example, the network 160 is a PAN (personal area network), a LAN (local area network), a CAN (campus area network), a MAN (metropolitan area network), or a WAN (wide area network). e area network), BBN (broadband network), the Internet, etc. may include any one or more of the networks. Further, network 160 may include any one or more of network topologies including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, and the like. It will not be done.

Input/output interface 140 may be a means for interfacing with input/output device 150. For example, input devices may include devices such as a microphone, keyboard, camera, or mouse, and output devices may include devices such as a display and speakers. As another example, input/output interface 140 may be a means for interfacing with a device that has integrated input and output functionality, such as a touch screen. Input/output device 150 may be configured as one device with computer device 100.

Also, in other embodiments, computing device 100 may include fewer or more components than those of FIG. However, most prior art components need not be clearly illustrated. For example, the computing device 100 may be implemented to include at least some of the input/output devices 150 described above, and may further include other components such as transceivers, cameras, various sensors, databases, etc. But that's fine.

The large-scale language model used in the present invention refers to a language model that can be inferred without fine-tuning using a method such as Few-shot Learning (FSL). It has more than 10 times as many mediating variables (for example, more than 100 billion mediating variables) than conventional general language models. For example, large-scale language models such as GPT-3 (Generative Pre-trained Transformer 3) and HyperClova are excellent few-shot learning machines that can be controlled with natural text prompts. In-context learning, which is the ability to understand patterns and solve NLP problems by simply using a computer, is possible.

This embodiment relates to a new data expansion technique that utilizes a large-scale language model to generate new data from original data. Further, by utilizing soft labels predicted by a language model, knowledge can be effectively distilled using a large-scale language model, and at the same time, textual perturbations can be generated.

FIG. 2 is a diagram for explaining the concept of text expansion using a large-scale language model in one embodiment of the present invention.

Referring to FIG. 2, in this embodiment, a large-scale language model 210 is used as a backbone for generating synthetic text data required for model learning.

According to this embodiment, by using the large-scale language model 210, it is possible to generate synthetic yet hyper-realistic new data from original data.

When there is text data in a format with or without a label, the corresponding data is converted into a prompt input sentence in the form of a natural language, and the converted prompt input sentence is given as input to the language model 210 to generate a subject natural word. You can derive results. The derived generation results may be analyzed to extract new data, where the new data is in the same form as the original data, with or without labels. The extracted new data can be added to the original text data to help with data collection, or if a model is generated using the relevant data, the performance of the model can be improved.

In other words, the present embodiment uses a prompt-based approach method to generate new data in the large-scale language model 210 for the purpose of data augmentation, and the new data is inspired from the original data. Knowledge distillation can be achieved by training a small-scale classification model using the soft labels predicted by the large-scale language model 210.

FIG. 3 is a flowchart illustrating an example of a data generation method that can be executed by a computer device in an embodiment of the present invention.

The data generation method according to this embodiment may be executed by the computer device 100 described above. In this case, the processor 120 of the computer device 100 may be implemented to execute control instructions according to the code of the operating system included in the memory 110 and the code of at least one program. Here, the processor 120 controls the computer device 100 so that the computer device 100 executes steps 310 to 340 included in the data generation method of FIG. It's fine.

The data generation method according to this embodiment can generate extremely fluent new data based on data distribution.

Referring to FIG. 3, at step 310, processor 120 may select original data from the data set for use in the prompt.

In the following, a method for selecting original data will be explained using data generation for a text classification task as an example. Given a classification task T, the training dataset is a set of pairs of text x and label y.

となる。

becomes.

As an example, the processor 120 may randomly select k pieces of original data from the training data set D. The processor 120 may select the k pieces of original data using a uniform distribution (Equation (1)).

The number of pieces of original data k may be determined in consideration of cost and performance. For example, the processor 120 may set k to 2 and select two pieces of original data from the training data set D.

As another example, the processor 120 may select the original data from the training data set D after considering the semantic diversity of the text. Semantic diversity is a measure of how diverse the meanings of a text are. When semantic diversity is low, the data expansion effect is low because the generated data has a high degree of similarity to the original of the existing dataset, but on the other hand, the higher the semantic diversity, the more likely it is that new data useful for data expansion will be obtained. increases.

As a method of calculating semantic diversity, a vector of each text is extracted using a sentence vector representation method (eg, bag-of-words, aggregate word2vec, BERT embedding, BLEURT, etc.). After calculating the distance between vectors using similarity (e.g., cosine similarity) or calculating the pairwise semantic distance of each pair using a network like BLEURT, the distance is Sampling may be performed by giving a higher weight to pairs that are far away (high semantic distance).

The processor 120 may select original data corresponding to the number of classes, that is, the types of labels, from the learning data set D. The processor 120 may preferentially select data with a high semantic distance between pairs of original data with the same label. An α (alpha) multiplier may be applied to the semantic distance, where α corresponds to the hyperparameter that needs to be optimized.

At step 320, processor 120 may configure prompts that are appropriate for the input of the language model. Processor 120 may utilize the source data selected in step 310 to construct input prompts for the language model. The processor 120 may create a dedicated prompt template that appropriately reflects the characteristics of a given NLP problem, and at this time, the prompt template may include the definition and meta information of the corresponding task. In other words, the processor 120 may process the original data selected from the data set to compose a natural language prompt, where the prompt is produced in a format that can be understood by the language model; Given as input to the language model. When the original data is labeled data, the processor 120 designs the prompt so that the input sentence is generated along with the label information.

The processor 120 uses original data sampled from the learning data set D.

が与えられるとき、説明ヘッダ（ｄｅｓｃｒｉｐｔｉｏｎ ｈｅａｄｅｒ）、テキスト－ラベル対リスト、拡張接頭辞（ａｕｇｍｅｎｔａｔｉｏｎ ｐｒｅｆｉｘ）で構成されたプロンプトを生成してよい。プロンプトは、言語モデルがデータ分布に対してさらに適切に一般化することが可能なように各タスクの情報を有しており、このようなタスク表示子（ｔａｓｋ ｉｎｄｉｃａｔｏｒ）は、タスクごとに固有であり、課題のメタ情報を提供してよい。

may generate a prompt consisting of a description header, a list of text-label pairs, and an augmentation prefix. The prompt contains information for each task to allow the language model to generalize better to the data distribution, and such task indicators are unique for each task. Yes, you may provide meta information about the assignment.

The format of the prompt itself may be configured in various ways, but as an example, the prompt can confirm the text type (e.g., review, article, etc.), the label type (e.g., sentiment, classification, etc.), and the label position. It may include a label-token verbizer.

The text type T is a meta type of the input text x, and for example, in video review sentiment analysis, the text type corresponds to a video review. Label type L is a metatype of label class y, and for example, in video review sentiment analysis, the label type corresponds to emotion. label-token section numerator v

においてプロンプトを公式化するためには、ラベルクラス

To formulate the prompt in the label class

と言語モデルの語彙

and language model vocabulary

で単語トークン間の１対１マッピングが必要となる。

requires a one-to-one mapping between word tokens.

The three meta information mentioned above are task specifications.

を構成する。各タスクＴは、プロンプトを公式化することのできる課題仕様

Configure. Each task T is a task specification that can formulate prompts.

を必要とする。プロセッサ１２０は、基本的に、一般タスク仕様である

Requires. Processor 120 is basically general task specific.

を使用してプロンプトを生成してよい。ここで、Ｉは、クラスラベルが語彙

You can generate prompts using Here, I is the class label is a vocabulary

に存在すると仮定する識別関数（ｉｄｅｎｔｉｔｙ ｆｕｎｃｔｉｏｎ）を意味する。

An identity function that is assumed to exist in .

In short, as shown in FIG. 4, for a given task T, the processor 120 considers the task specification 410, the data example 420 which is the original data sampled from the learning data set 400, and the characteristics of the given task T. The input prompts of the language model may be configured by combining the prompt templates 430.

As can be seen from the specific example of FIG. 5, the processor 120 utilizes the task specification 410, example data 420, and prompt template 430 to configure the input of the language model in a format that the language model can understand. A prompt 540 may be created. An example of the task specification 410 is as shown in Table 1, and an example of the prompt template 430 is as shown in Table 2. As data example 420

が与えられる場合、入力プロンプト５４０は表３のように構成されてよい。

, the input prompt 540 may be configured as in Table 3.

Referring again to FIG. 3, at step 330, processor 120 may input the prompts configured at step 320 into the language model and generate natural language containing new data from the language model. In other words, the processor 120 may input the prompt input sentence into the language model and then obtain the language generation result by the completion function of the language model.

At step 340, processor 120 may analyze the natural language generation results to extract new data. The language model may generate natural words according to the pattern of a prompt given as an input sentence, and may extract new data by pattern analysis of the generated natural words.

As shown in FIG. 6, the processor 120 inputs the prompt 540 into the language model 210 and analyzes the natural language pattern generated based on the language model 210 to generate extended data 650 (new sentences, corresponding Label information of sentences) pairs can be obtained. As an example, processor 120 may use a language modeling probability distribution of natural language tokens that correspond to labels in prompt 540 to obtain the label distribution.

In the prompt-based approach method, the extended text x' and the label y' are continuously generated as natural language text after the prompt. A predefined prompt template based on sampled original data may extract each value by pattern matching to provide input sentences such that the language model generates an (x', y') structure. Joint text and label generation also ensures that the generated text is concatenated with the correct label.

The prompt design of this embodiment is

に該当するラベルトークンがテキストｘ以後に生成されるように保障する。プロセッサ１２０は、言語モデルを利用して疑似ラベリング（ｐｓｅｕｄｏ－ｌａｂｅｌｉｎｇ）を実行してよく、拡張テキストｘ’のソフトラベル確率を得るためにラベル－トークンを生成する可能性（ｌｉｋｅｌｉｈｏｏｄ）を正規化してよい。

ensures that a label token corresponding to x is generated after text x. Processor 120 may utilize the language model to perform pseudo-labeling and normalize the likelihood of generating a label-token to obtain a soft label probability for the extended text x'. good.

The pseudo label probability that the extended text x' is labeled with the label y' is as shown in Equation (2).

ここで、

here,

は、言語モデリング可能性（ｌａｎｇｕａｇｅ ｍｏｄｅｌｉｎｇ ｌｉｋｅｌｉｈｏｏｄ）を示し、

indicates language modeling likelihood,

は、与えられたタスク仕様を構成する関数である。

are the functions that constitute the given task specification.

In this embodiment, text perturbation, pseudo-labeling, and knowledge distillation can be effectively combined in a single augmentation task. In fact, new data with pseudo-labels is trained with the original data using cross-entropy loss.

Referring to FIG. 7, the processor 120 may generate new data based on the probability distribution according to the completed function of the language model 210, and at this time, the processor 120 may generate new data based on the probability distribution based on the completed function of the language model 210. Probability may be used to generate soft labels, that is, labels with a distribution.

More specifically, the processor 120 converts all natural words provided as prompt input sentences into a particular tokenization scheme (e.g., constant form index, etc.) before inputting them into the language model 210. . In order to obtain a pair (new sentence, label information of the corresponding sentence), the probability distribution of the previous token is transmitted as the input of the next token based on autoregressive, but at this time, the heuristic The top n probabilities are used as input for the next token by beam search using (heuristic). By using the top n probabilities among the joint probabilities obtained by multiplying the own probability distribution of each token by the probability distribution of the previous token, sequences with high probability values can be extracted.

For example, as shown in FIG. 8, the processor 120 may construct a soft label using the probability of a token corresponding to the label. In other words, soft labels may be extracted from the normalized label token distribution predicted by language model 210.

Referring to FIG. 9, when data example 1 (910) classified as positive and data example 2 (920) classified as negative are given as original data, new data is generated from the language generation result using the language model 210. 930 may be extracted.

Processor 120 analyzes the results generated by the prompts and extracts the combined new data and label information. By repeating this process several times, you will be able to obtain diverse data and accurate classification information that did not originally exist in the data, and by mixing such data with existing data, you will be able to It can be used to fine tune NLP models. New data 930 is output data that is generated by appropriately referring to the linguistic and structural features of the two given original data 910 and 920, and appropriately mixes affirmation and negation in the new data 930. This results in completely new data that does not already exist.

By adding new data 930 to the learning data set, that is, by expanding the data, the performance of the final classifier can be improved.

The original data 910 and 920 may be generated as data with a hard label attached with a single label, and the new data 930 may be generated as data with a soft label with at least two labels distributed.

Since a soft label can be converted into a hard label by a max operation, the original data 910 and 920 and the new data 930 can all be used in the form of a hard label. In the learning process, a loss function such as cross entropy is used, so that both hard label format and soft label format can be utilized.

Thus, according to embodiments of the present invention, existing data may be transformed or extended based on a language model to generate augmented data, and knowledge known by the language model may be transferred through the augmented data. As a result, the man-hours required for data collection can be significantly reduced, and the model weight can be reduced.

The apparatus described above may be realized by hardware components, software components, and/or a combination of hardware and software components. For example, the devices and components described in the embodiments include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or may be implemented using one or more general purpose or special purpose computers, such as various devices capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that execute on the OS. The processing device may also be responsive to execution of the software to access, record, manipulate, process, and generate data. Although for convenience of understanding, one processing device may be described as being used, those skilled in the art will appreciate that a processing device may include multiple processing elements and/or multiple types of processing elements. You will understand. For example, a processing device may include multiple processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include computer programs, code, instructions, or a combination of one or more of these that configure a processing device or instruct a processing device, independently or collectively, to perform operations as desired. You may do so. The software and/or data may be embodied in a machine, component, physical device, computer storage medium or device of any kind for being interpreted by or providing instructions or data to a processing device. good. The software may be distributed on computer systems connected by a network, and may be recorded or executed in a distributed manner. The software and data may be recorded on one or more computer readable storage media.

Methods according to embodiments may be implemented in the form of program instructions executable by various computer means and recorded on computer-readable media. Here, the medium may be one that continuously records a computer-executable program, or one that temporarily records it for execution or download. Also, the medium may be a variety of recording or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may be distributed over a network. It may also exist. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, It may also include ROM, RAM, flash memory, etc., and may be configured to record program instructions. Further, other examples of the medium include an application store that distributes applications, a site that supplies or distributes various other software, and a recording medium or storage medium managed by a server.

As mentioned above, although the embodiments have been described based on limited embodiments and drawings, those skilled in the art will be able to make various modifications and variations based on the above description. For example, the techniques described may be performed in a different order than in the manner described, and/or components of the systems, structures, devices, circuits, etc. described may be performed in a different form than in the manner described. Even when combined or combined, opposed or replaced by other components or equivalents, suitable results can be achieved.

Therefore, even if the embodiments are different, if they are equivalent to the scope of the claims, they fall within the scope of the appended claims.

210: Language model 220: PLM

Claims (20)

A data generation method performed by a computer device, the method comprising:
The computer device includes at least one processor configured to execute computer-readable instructions contained in memory;
The data generation method includes:
the at least one processor utilizes source data to construct prompts that serve as input sentences for a language model; and the at least one processor inputs the prompts into the language model and extracts new data from the language model. and generating label information for the new data ,
The generating step is a step of generating a soft label indicating a label distribution using a probability distribution for natural words corresponding to the label in the prompt.
data generation methods, including ; The data generation method includes:
2. The data generation method of claim 1 , further comprising: the at least one processor selecting the original data from a training data set depending on the semantic diversity of the text. The selecting step includes:
3. The data generation method according to claim 2 , further comprising selecting the original data as many as the number of label types from the learning data set. The configuring step includes:
4. The data generation method according to claim 1 , wherein the prompt is configured in a format including a text type and a label type. The configuring step includes:
4. The data generation method according to claim 1 , wherein the prompt is configured in a format including a text type, a label type, and a label -token delimiter. The configuring step includes:
4. The data generation method according to claim 1 , further comprising processing the original data to configure the prompt in a natural language format that is the same as that of the original data. A data generation method performed by a computer device, the method comprising:
The computer device includes at least one processor configured to execute computer-readable instructions contained in memory;
The data generation method includes:
the at least one processor configuring a prompt to be an input sentence of a language model using original data, the step of configuring the prompt by combining a task specification, the original data, and a prompt template;
the at least one processor inputting the prompt into the language model and generating new data and label information for the new data from the language model;
data generation methods, including ; A data generation method performed by a computer device, the method comprising:
The computer device includes at least one processor configured to execute computer-readable instructions contained in memory;
The data generation method includes:
the at least one processor constructs a prompt to serve as an input sentence for the language model using the source data; and
the at least one processor inputting the prompt into the language model and generating new data and label information for the new data from the language model;
and the step of generating comprises :
extracting the new data and the label information using an autoregressive method that transmits a probability distribution of previous tokens as input for a next token for natural words corresponding to the text and label in the prompt; ,Data generation method. The extracting step includes:
9. The data generation method according to claim 8 , further comprising the step of transmitting the probability of a top portion of the probability distribution of the previous token as an input of the next token by beam search using a heuristic. A data generation method performed by a computer device, the method comprising:
The computer device includes at least one processor configured to execute computer-readable instructions contained in memory;
The data generation method includes:
the at least one processor selecting original data from the training data set according to the semantic diversity of the text;
the at least one processor constructs a prompt to be an input sentence for a language model using the original data; and
the at least one processor inputting the prompt into the language model and generating new data and label information for the new data from the language model;
data generation methods, including; A computer program for causing a computer device to execute the data generation method according to any one of claims 1 to 10. A computer device,
at least one processor configured to execute computer-readable instructions contained in the memory;
The at least one processor includes:
A process of configuring a prompt to be an input sentence of a language model using original data, and a process of inputting the prompt to the language model and generating new data and label information for the new data from the language model. ,
The at least one processor includes:
In the generating process, a computer device generates a soft label indicating a label distribution using a probability distribution for a natural word corresponding to a label in the prompt. The at least one processor includes:
13. The computer device according to claim 12 , wherein the original data is selected from a training data set depending on the semantic diversity of the text. The at least one processor includes:
14. The computer device according to claim 13 , wherein the original data as many as the number of label types are selected from the learning data set. The at least one processor includes:
The computer device according to any one of claims 12 to 14 , characterized in that the prompt is configured in a format including a text type and a label type. The at least one processor includes:
A computer device according to any one of claims 12 to 14 , characterized in that the prompt is configured in a format that includes a text type, a label type, and a label -token delimiter. A computer device,
at least one processor configured to execute computer-readable instructions contained in the memory;
The at least one processor includes:
a process of configuring a prompt to be an input sentence of a language model using original data, the process of configuring the prompt by combining a task specification, the original data, and a prompt template ;
inputting the prompt into the language model and generating new data and label information for the new data from the language model;
A computer device that processes A computer device,
at least one processor configured to execute computer-readable instructions contained in the memory;
The at least one processor includes:
The process of constructing prompts that serve as input sentences for the language model using the original data, and
inputting the prompt into the language model and generating new data and label information for the new data from the language model;
process,
The at least one processor includes:
In the generation process, the new data and the label information are generated using an autoregressive method that transmits the probability distribution of the previous token as input to the next token for the text in the prompt and the natural language corresponding to the label. A computer device characterized by extracting. The at least one processor includes:
19. The computer device according to claim 18 , wherein the probability of an upper part of the probability distribution of the previous token is transmitted as an input of the next token by beam search using a heuristic. A computer device,
at least one processor configured to execute computer-readable instructions contained in the memory;
including;
The at least one processor includes:
a process of selecting original data from the training dataset according to the semantic diversity of the text;
a process of configuring a prompt that becomes an input sentence for a language model using the original data;
inputting the prompt into the language model and generating new data and label information for the new data from the language model;
A computer device that processes

Images