WO2022110730A1 - Label-based optimization model training method, apparatus, device, and storage medium - Google Patents

Abstract

The present application relates to the field of artificial intelligence and discloses a label-based optimization model training method, an apparatus, a device, and a storage medium, which are used for improving the optimization flexibility of a target optimization model as well as the accuracy of an optimized text. The label-based optimization model training method comprises: obtaining original text data and comparison text data; inputting the original text data into a preconfigured encoder, and obtaining target content hidden layer vector groups; obtaining target label groups according to the target content hidden layer vector groups and a decoder; training an initial optimization model on the basis of the target label groups; obtaining text data to be evaluated according to the original text data and the initial optimization model, and determining whether the text data to be evaluated matches with the comparison text data; if target text data to be evaluated does not match with the comparison text data, performing adjustment on the initial optimization model, and obtaining a target optimization model. Additionally, the present application further relates to blockchain technology, and the text data to be evaluated may be stored in a blockchain.

Description

Label-based optimization model training method, device, equipment and storage medium

This application claims the priority of the Chinese patent application filed on November 27, 2020 with the application number 202011353108.0 and the invention titled "label-based optimization model training method, device, equipment and storage medium", the entire content of which is Incorporated in the application by reference.

technical field

The present application relates to the technical field of neural networks, and in particular, to a label-based optimization model training method, apparatus, device, and storage medium.

Background technique

In the current recruitment process, in order to simplify the recruitment process and improve work efficiency, many companies use artificial intelligence interview systems for interviews, which mainly convert the interviewee's voice into text through speech recognition. There will be errors in word and sentence recognition, so it is also necessary to use the recognition optimization model to correct the grammar of the text and adjust the sentence fluency.

The current method for identifying and optimizing models to correct grammar and adjust sentence fluency is to train a trainer for a specific task, and then use the trainer to correct grammar and adjust sentence fluency. The inventor realized that this adjustment method is only suitable for For a specific task, other tasks cannot be optimized, the flexibility of identifying and optimizing the model is low, and the accuracy of converting other tasks into text will be reduced.

SUMMARY OF THE INVENTION

The present application provides a label-based optimization model training method, device, equipment and storage medium, which improves the flexibility of identifying the optimization model and improves the accuracy of converting other tasks into text.

In order to achieve the above purpose, a first aspect of the present application provides a label-based optimization model training method, comprising: acquiring multiple original text data and multiple comparison text data, one original text data corresponding to one comparison text data; Each original text data is input into the preset encoder, and based on the self-attention mechanism and the query attention mechanism, multiple target content hidden layer vector groups are obtained; each target content hidden layer vector group is input into the preset decoder. , perform label calculation in combination with the autoregressive mechanism to obtain multiple target label groups; train a model based on the multiple target label groups to obtain an initial optimization model; input the plurality of original text data into the initial optimization model in turn to obtain A plurality of text data to be detected, and determine whether each text data to be detected matches the corresponding comparison text data; if the target text data to be detected does not match the corresponding comparison text data, then adjust the initial optimization model. parameters to obtain the target optimization model.

A second aspect of the present application provides a label-based optimization model training device, comprising a memory, a processor, and computer-readable instructions stored on the memory and executable on the processor, the processor executing the When the computer-readable instructions are described, the following steps are implemented: obtaining multiple original text data and multiple comparison text data, one original text data corresponding to one comparison text data; inputting each original text data into a preset encoder, based on The self-attention mechanism and the inquiry attention mechanism can obtain multiple target content hidden layer vector groups; input each target content hidden layer vector group into the preset decoder, and combine the autoregressive mechanism to perform label calculation to obtain multiple target labels group; train the model based on the multiple target label groups to obtain an initial optimization model; input the multiple original text data into the initial optimization model in turn to obtain multiple text data to be detected, and determine each text to be detected Whether the data matches the corresponding comparison text data; if the target text data to be detected does not match the corresponding comparison text data, adjust the parameters of the initial optimization model to obtain the target optimization model.

A third aspect of the present application provides a computer-readable storage medium, where computer instructions are stored in the computer-readable storage medium, and when the computer instructions are executed on a computer, the computer is caused to perform the following steps: acquiring a plurality of original texts data and multiple comparison text data, one original text data corresponds to one comparison text data; input each original text data into the preset encoder, and obtain multiple target contents based on the self-attention mechanism and the inquiry attention mechanism Hidden layer vector group; input each target content hidden layer vector group into the preset decoder, and perform label calculation in combination with the autoregressive mechanism to obtain multiple target label groups; train the model based on the multiple target label groups to obtain the initial optimization model; inputting the plurality of original text data into the initial optimization model in turn, obtaining a plurality of text data to be detected, and judging whether each text data to be detected matches the corresponding comparison text data; If the detected text data does not match the corresponding comparison text data, the parameters of the initial optimization model are adjusted to obtain the target optimization model.

A fourth aspect of the present application provides a label-based optimization model training device, comprising: an acquisition module for acquiring a plurality of original text data and a plurality of comparison text data, where one original text data corresponds to one comparison text data; The layer vector calculation module is used to input each original text data into the preset encoder, and based on the self-attention mechanism and the query attention mechanism, multiple target content hidden layer vector groups are obtained; the label group calculation module is used to Each target content hidden layer vector group is input into the preset decoder, and the label calculation is performed in combination with the autoregressive mechanism to obtain multiple target label groups; the training module is used to train the model based on the multiple target label groups to obtain the initial optimization. model; a judgment module for sequentially inputting the plurality of original text data into the initial optimization model, obtaining a plurality of text data to be detected, and judging whether each text data to be detected matches the corresponding comparison text data ; Adjustment module, if the target text data to be detected does not match the corresponding comparison text data, it is used to adjust the parameters of the initial optimization model to obtain the target optimization model.

In the technical solution provided by the present application, a plurality of original text data and a plurality of comparison text data are obtained, one original text data corresponds to one comparison text data; each original text data is input into a preset encoder, based on self-attention Force mechanism and query attention mechanism to obtain multiple target content hidden layer vector groups; input each target content hidden layer vector group into the preset decoder, and combine the autoregressive mechanism to perform label calculation to obtain multiple target label groups; The model is trained based on the multiple target label groups to obtain an initial optimization model; the multiple original text data are sequentially input into the initial optimization model to obtain multiple text data to be detected, and it is determined whether each text data to be detected is Match with the corresponding comparison text data; if the target text data to be detected does not match the corresponding comparison text data, then adjust the parameters of the initial optimization model to obtain the target optimization model. In the embodiment of the present application, combining the self-attention mechanism of the encoder, the query attention mechanism of the encoder, and the autoregressive mechanism of the decoder, multiple target label groups corresponding to multiple original text data are calculated, and then according to the multiple target labels The initial optimization model is trained in groups, and finally the initial optimization model is adjusted based on the comparison text data and the text data to be detected output by the initial optimization model to obtain the target optimization model, which makes the target optimization model suitable for a variety of optimization tasks and improves the optimization of the target optimization model. Flexibility and accuracy of optimized text.

Description of drawings

1 is a schematic diagram of an embodiment of a label-based optimization model training method in an embodiment of the present invention;

2 is a schematic diagram of another embodiment of a label-based optimization model training method in an embodiment of the present invention;

3 is a schematic diagram of calculating a target content hidden layer vector group in an embodiment of the present invention;

4 is a schematic diagram of an embodiment of a label-based optimization model training device in an embodiment of the present invention;

5 is a schematic diagram of another embodiment of a label-based optimization model training apparatus in an embodiment of the present invention;

FIG. 6 is a schematic diagram of an embodiment of a label-based optimization model training device in an embodiment of the present invention.

Detailed ways

The embodiments of the present application provide a label-based optimization model training method, apparatus, device, and storage medium. "Three", "fourth", etc., if present, are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed steps or units, but may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

For ease of understanding, the specific flow of the embodiment of the present application is described below, referring to Fig. 1, the first embodiment of the label-based optimization model training method in the embodiment of the present application includes:

101. Obtain multiple original text data and multiple comparison text data, where one original text data corresponds to one comparison text data;

The server obtains multiple pieces of original text data and multiple pieces of comparison text data that correspond one-to-one with the multiple pieces of original text data.

Multiple raw text data are used to train the optimized model, and multiple comparison text data are used to adjust the optimized model. A plurality of original text data are text data obtained by music data being recognized by a speech recognition model. In this embodiment, the original text data are interview scene data, such as "I was born in 2000, and I am studying in college", "My The graduate school is school A, and I work in company B” and so on. The comparison text data corresponding to the original text data of "I was born in 2000 and I am studying in university" are "I was born in 2000 and I am studying in university now", "My graduation school is school A, and I am in company B The comparison text data corresponding to the original text data of "work" is "My graduation school is school A, and now I work in company B".

It can be understood that the execution body of the present application may be a label-based optimization model training device, and may also be a terminal or a server, which is not specifically limited here. The embodiments of the present application take the server as an execution subject as an example for description.

102. Input each original text data into a preset encoder, and obtain multiple target content hidden layer vector groups based on the self-attention mechanism and the inquiry-attention mechanism;

Input each original text data into the preset encoder, calculate the content hidden layer vector based on the encoder's self-attention mechanism and query attention mechanism, and obtain multiple target content hidden layer vector groups.

It should be noted that the self-attention mechanism and the inquiry attention mechanism are collectively referred to as the dual-stream attention mechanism, through which the dual-stream attention mechanism can be used to predict what the next word is through the above. The self-attention mechanism is the traditional self-attention mechanism, and the inquiry-attention mechanism modifies some things on the basis of the self-attention mechanism, so that it cannot see itself in the global information when predicting what the next word is, so that it can Use the above to predict what the next word will be. The server inputs a raw text data into the preset encoder, and iteratively calculates the content hidden layer vector of each layer in the encoder based on the self-attention mechanism and the query attention mechanism in the encoder, so as to obtain one of the target content hidden layers vector group, the server then inputs other original text data into the preset encoder to obtain other target content hidden layer vector groups, of which one target content hidden layer vector group and other target content hidden layer vector groups constitute multiple target content hidden layers Layer vector set.

103. Input each target content hidden layer vector group into the preset decoder, and perform label calculation in combination with the autoregressive mechanism to obtain multiple target label groups;

The server inputs each target content hidden layer vector group into the preset decoder for decoding, and combines the decoder's autoregressive mechanism to calculate labels based on each target content hidden layer vector group to obtain multiple target label groups. .

It should be noted that there are two types of decoders, one of which is a decoder with an autoregressive mechanism, and the other is a decoder with a feed-forward mechanism. In this embodiment, the decoder with autoregressive mechanism is mainly described. The autoregressive mechanism can be understood as using a layer of decoding and encoding attention mechanism to decode the target content hidden layer vector group to obtain the corresponding target label group. In other embodiments, the decoder may also be a decoder with a feedforward mechanism, and the decoder with a feedforward mechanism mainly superimposes a layer of Softmax for the target content hidden layer vector group to obtain the corresponding target label group.

For example, multiple content hidden layer vector groups are C1, C2, and C3. Input C1, C2, and C3 into the preset decoder in turn. First, the autoregressive mechanism is used to decode C1 to obtain a target label group D1, and then The autoregressive mechanism is used to decode C2 to obtain a target label group D2, and finally the autoregressive mechanism is used to decode C3 to obtain a target label group D3.

104. Train a model based on multiple target label groups to obtain an initial optimization model;

The server performs model training based on multiple target label groups to obtain an initial optimized model.

The server uses multiple target label groups to perform multiple iterative training to obtain an initial optimized model.

105. Inputting a plurality of original text data into the initial optimization model in turn, obtaining a plurality of text data to be detected, and judging whether each text data to be detected matches the corresponding comparison text data;

The server sequentially inputs a plurality of original text data into the initial optimization model for optimization, obtains a plurality of text data to be detected, and then determines whether each text data to be detected matches the corresponding comparison text data.

Inputting each original text data into the initial optimization model can obtain a plurality of text data to be detected. By comparing the text data to be detected with the comparison text data, the accuracy of the initial optimization model can be judged. The parameters of the initial text optimization model are updated and adjusted.

For example, the original text data is "I was born in 2000, and I am studying in college", and the corresponding comparison text data is "I was born in 2000, and I am studying in college now". Input the original text data of "I was born in 2000, and I am studying in college" into the initial optimization model, and the text data to be detected is "I was born in 2000, and I am studying in college", and the server judges "I was born in In 2000, whether the text data to be tested is the same as the text data to be tested for "I was born in 2000 and I am currently studying in college".

106. If the target text data to be detected does not match the corresponding comparison text data, adjust the parameters of the initial optimization model to obtain the target optimization model.

If the server determines that the target text data to be detected does not match the corresponding comparison text data, the parameters of the initial optimization model are adjusted to obtain the target optimization model.

This embodiment also uses the example of step 105. The text data to be detected is “I was born in 2000 and I am studying at a university”, and the comparison text data is “I was born in 2000 and I am studying in a university now”, and the server determines that it is to be detected. If the text data does not match the comparison text data, it means that the optimization accuracy of the initial optimization model is low. In this case, the parameters of the initial optimization model need to be adjusted to obtain the target optimization model.

It should be noted that, in this embodiment, only one example is used for description. In fact, the basis for adjusting the initial optimization model is a plurality of original text data and a plurality of corresponding comparison text data, a plurality of original text data The process of optimizing the initial optimization model is the same as the corresponding comparison file data, so other optimization processes will not be described in detail in this embodiment.

In the embodiment of the present application, combining the self-attention mechanism of the encoder, the query attention mechanism of the encoder, and the autoregressive mechanism of the decoder, multiple target label groups corresponding to multiple original text data are calculated, and then according to the multiple target labels The initial optimization model is trained in groups, and finally the initial optimization model is adjusted based on the comparison text data and the text data to be detected output by the initial optimization model to obtain the target optimization model, which makes the target optimization model suitable for a variety of optimization tasks and improves the optimization of the target optimization model. Flexibility and accuracy of optimized text.

Referring to FIG. 2, another embodiment of the label-based optimization model training method in the embodiment of the present application includes:

201. Obtain multiple original text data and multiple comparison text data, where one original text data corresponds to one comparison text data;

The server obtains multiple pieces of original text data and multiple pieces of comparison text data that correspond one-to-one with the multiple pieces of original text data.

Multiple raw text data are used to train the optimized model, and multiple comparison text data are used to adjust the optimized model. A plurality of original text data are text data obtained by music data being recognized by a speech recognition model. In this embodiment, the original text data are interview scene data, such as "I was born in 2000, and I am studying in college", "My The graduate school is school A, and I work in company B” and so on. The comparison text data corresponding to the original text data of "I was born in 2000 and I am studying in university" are "I was born in 2000 and I am studying in university now", "My graduation school is school A, and I am in company B The comparison text data corresponding to the original text data of "work" is "My graduation school is school A, and now I work in company B".

202. Input each original text data into a preset encoder, and obtain multiple target content hidden layer vector groups based on the self-attention mechanism and the inquiry-attention mechanism;

Input each original text data into the preset encoder, calculate the content hidden layer vector based on the encoder's self-attention mechanism and query attention mechanism, and obtain multiple target content hidden layer vector groups.

It should be noted that the self-attention mechanism and the inquiry attention mechanism are collectively referred to as the dual-stream attention mechanism, through which the dual-stream attention mechanism can be used to predict what the next word is through the above. The self-attention mechanism is the traditional self-attention mechanism, and the inquiry-attention mechanism modifies some things on the basis of the self-attention mechanism, so that it cannot see itself in the global information when predicting what the next word is, so that it can Use the above to predict what the next word will be. The server inputs a raw text data into the preset encoder, and iteratively calculates the content hidden layer vector of each layer in the encoder based on the self-attention mechanism and the query attention mechanism in the encoder, so as to obtain one of the target content hidden layers vector group, the server then inputs other original text data into the preset encoder to obtain other target content hidden layer vector groups, of which one target content hidden layer vector group and other target content hidden layer vector groups constitute multiple target content hidden layers Layer vector set.

Specifically, the server extracts the corresponding original text sequence from each original text data; the server inputs each original text sequence into the preset encoder, and determines the corresponding original text sequence based on the attention mask mechanism of the encoder and each original text sequence. The server calculates the hidden layer of each input sequence based on the self-attention mechanism and the inquiry attention mechanism, generates the corresponding content hidden layer vector group, and obtains multiple target content hidden layer vector groups.

For example, if the original text sequence is [This,is,a,sentence], the server inputs the original text sequence into the preset encoder, and changes the order of the original text sequence based on the encoder's attention mechanism, so as to obtain the corresponding input sequence , assuming the resulting input sequence is:

The server calculates the input sequence in multiple hidden layers based on the self-attention mechanism and the query attention mechanism, and generates a content hidden layer vector group corresponding to the original text sequence. The content hidden layer vector group is obtained, and multiple content hidden layer vector groups are obtained.

It should be noted that the hidden layers in this embodiment are 12 layers.

The server inputs each original text sequence into the preset encoder, and the specific process of determining the corresponding input sequence based on the encoder's attention mask mechanism and each original text sequence is as follows:

The server first inputs each original text sequence into the preset encoder, and combines the attention mask mechanism to perform multiple iterative predictions on each original text sequence to obtain multiple corresponding position masks; then the server integrates each original text. Multiple position masks corresponding to the sequence to obtain the input sequence corresponding to each original text sequence.

For ease of understanding, the following description is combined with specific scenarios:

Suppose there is an original text sequence [This,is,a,sentence], when predicting the result based on the existing model, we usually get 4! permutations of possible outcomes. In this embodiment, the attention mask mechanism of the encoder changes the order of objects in the original text sequence through a mask matrix to obtain a new input sequence. Suppose, when the server predicts the position of "a", there is no information in front of "a", so the corresponding position mask is [0,0,0,0]; when the server predicts the position of "is", it needs to use |The location of "a", the server determines that the corresponding location mask is [0, 0, 1, 0]; when the server predicts the location of "sentence", combining the locations of "is" and "a", the server obtains the corresponding location The location mask of This is [0,1,1,0]. Similarly, when the server predicts the location of This, the corresponding location mask is [0,1,1,1]. Finally, the server integrates these position masks to obtain the input sequence corresponding to the original text sequence of [This,is,a,sentence] as:

The server calculates the hidden layer for each input sequence based on the self-attention mechanism and the inquiry attention mechanism, generates the corresponding content hidden layer vector group, and obtains multiple target content hidden layer vector groups. The specific process is as follows:

The server extracts the corresponding input vector group based on each input sequence, and adopts the self-attention mechanism and the inquiry attention mechanism to calculate the target input vector group and the preset initialization vector in the first hidden layer, and obtain the corresponding first The content hidden layer vector group and the corresponding first query hidden layer vector group; the server adopts the self-attention mechanism and the inquiry attention mechanism, in the second hidden layer, the corresponding first content hidden layer vector group and the corresponding first query The hidden layer vector group is calculated, and the corresponding second content hidden layer vector group and the corresponding second query hidden layer vector group are obtained; The content hidden layer vector group and the corresponding query hidden layer vector group are calculated until the last hidden layer, and the corresponding target content hidden layer vector group is generated, and the corresponding target content hidden layer vector group is the corresponding target content hidden layer. Content hidden layer vector group; finally, the server uses the self-attention mechanism and the query attention mechanism to calculate other input sequences according to the above steps, and obtains multiple target content hidden layer vector groups.

Please refer to Figure 3, e(x ₁ ), e(x ₂ ), e(x ₃ ) and e(x ₄ ) are the input vectors extracted based on the target input sequence respectively, w is the preset initialization vector, the server adopts The self-attention mechanism and the query attention mechanism calculate the target input vector and the preset initialization vector in the first hidden layer of the encoder, and obtain the first content hidden layer vector group g ₁ ⁽¹⁾ , g ₂ ⁽¹⁾ , g ₃ ⁽¹⁾ and g ₄ ⁽¹⁾ , the first query hidden layer vector groups h ₁ ⁽¹⁾ , h ₂ ⁽¹⁾ , h ₃ ⁽¹⁾ and h ₄ ⁽¹⁾ ; the server adopts a self-attention mechanism And the query attention mechanism hides the first content hidden layer vector group g ₁ ⁽¹⁾ , g ₂ ⁽¹⁾ , g ₃ ⁽¹⁾ and g ₄ ⁽¹⁾ and the first query hidden layer in the second hidden layer of the encoder The layer vector groups h ₁ ⁽¹⁾ , h ₂ ⁽¹⁾ , h ₃ ⁽¹⁾ and h ₄ ⁽¹⁾ are calculated to obtain the second content hidden layer vector groups g ₁ ⁽²⁾ , g ₂ ⁽²⁾ , g ₃ ⁽²⁾ and g ₄ ⁽³⁾ , the second query hidden layer vector group is obtained as h ₁ ⁽²⁾ , h ₂ ⁽²⁾ , h ₃ ⁽²⁾ and h ₄ ⁽²⁾ ; according to this method, the above one The output of the hidden layer of each layer is used as the input of the next hidden layer, and is calculated by combining the self-attention mechanism and the query attention mechanism to obtain the query hidden layer vector group of each hidden layer and the content hidden layer vector of each hidden layer. Group. The content hidden layer vector group output by the hidden layer of the last layer (the 12th layer) is used as the target content hidden layer vector group, namely x ₁ , x ₂ , x ₃ and x ₄ in the figure.

203. Input each target content hidden layer vector group into a preset decoder, and perform label calculation in combination with an autoregressive mechanism to obtain multiple target label groups;

The server inputs each target content hidden layer vector group into the preset decoder for decoding, and combines the decoder's autoregressive mechanism to calculate labels based on each target content hidden layer vector group to obtain multiple target label groups. .

It should be noted that there are two types of decoders, one of which is a decoder with an autoregressive mechanism, and the other is a decoder with a feed-forward mechanism. In this embodiment, the decoder with autoregressive mechanism is mainly described. The autoregressive mechanism can be understood as using a layer of decoding and encoding attention mechanism to decode the target content hidden layer vector group to obtain the corresponding target label group. In other embodiments, the decoder may also be a decoder with a feedforward mechanism, and the decoder with a feedforward mechanism mainly superimposes a layer of Softmax for the target content hidden layer vector group to obtain the corresponding target label group.

For example, multiple content hidden layer vector groups are C1, C2, and C3. Input C1, C2, and C3 into the preset decoder in turn. First, the autoregressive mechanism is used to decode C1 to obtain a target label group D1, and then The autoregressive mechanism is used to decode C2 to obtain a target label group D2, and finally the autoregressive mechanism is used to decode C3 to obtain a target label group D3.

Specifically, the corresponding content hidden layer dimension is read from each target content hidden layer vector group to obtain multiple content hidden layer dimensions; the multiple content hidden layer dimensions are sequentially input into the preset decoder, and the autoregressive mechanism is combined. Generate multiple decoding tag groups and corresponding multiple decoding tag probability groups; based on the decoding tag probability group corresponding to each decoding tag group, determine the target tag group corresponding to each original text data from each decoding tag group, and obtain Multiple target label groups.

In order to facilitate understanding, the description is combined with specific scenarios:

Assuming that the server obtains the target content hidden layer vector groups x ₁ , x ₂ , x ₃ and x ₄ from the above step 202, the read content hidden layer dimension is 1*4*768, where 1 represents the dimension of the sentence, and 4 represents the dimension of the sentence Sentence length, 768 represents the preset word vector dimension. Then the server inputs the hidden layer dimension of the content into the preset decoder, and combines the autoregressive mechanism to generate the decoding dimension and the corresponding decoding label probability group. The decoding dimension is 1*4*(2*D ^v ), where 1 represents the dimension of the sentence, 4 represents the length of the sentence, and 2*D ^v is the number of tags in the decoding tag group corresponding to the target content hidden vector group. Based on the probability of each corresponding decoded label, the label group determines the decoded label with the highest probability as the target label, thereby obtaining a target label group corresponding to the target original text data. The server also performs the same calculation for other target content hidden layer vector groups, thereby obtaining multiple target label groups.

204. Train a model based on multiple target label groups to obtain an initial optimization model;

The server performs model training based on multiple target label groups to obtain an initial optimized model.

The server uses multiple target label groups to perform multiple iterative training to obtain an initial optimized model.

205. Input each original text data into the initial optimization model in turn to generate a plurality of tag groups to be replaced, and each tag group to be replaced at least includes a reserved tag, a deletion tag and/or a phrase tag;

The server sequentially inputs each original text data into the initial optimization model for optimization, and obtains a plurality of tag groups to be replaced that at least include retained tags, deleted tags and/or phrase tags.

Keep tags and delete tags are basic tags, and phrase tags are additional tags. The phrase label in this embodiment is calculated based on the longest common subsequence, and the phrase label corresponds to a phrase, and the phrase needs to meet three conditions: a. The amount of data is small enough to prevent the generation of some irrelevant words; b. For The current original text data needs to have a high enough coverage; c. The frequency of occurrence is high. The specific process of generating phrase labels is as follows: using the longest common subsequence to compare the original text data with the corresponding comparison text data, extracting words that do not belong to the original text data from the comparison text sequence, and then using the label corresponding to the word Add to the initial set of phrase tags, and finally sort the set of phrase tags according to the frequency of word occurrence to obtain the final set of phrase tags.

It should be noted that phrase tags can be combined with retention tags and delete tags, for example, ^{'Keep now} and ^' Delete now', where ^'now is a phrase tag, Keep is a retention tag, and Delete is a deletion tag.

206. Replacing a plurality of label groups to be replaced in sequence according to a preset replacement rule, obtaining a plurality of text data to be detected, and judging whether each text data to be detected matches the corresponding comparison text data;

Specifically, the target phrase corresponding to the phrase tag is determined in the preset phrase set; the sub-text data corresponding to the reserved tag is retained in each original text data, the sub-text data corresponding to the deletion tag is deleted, and the sub-text data corresponding to the phrase tag is deleted in each original text data. The corresponding sub-text data is replaced with the target phrase, the text data to be detected corresponding to each original text data is generated, and a plurality of text data to be detected is obtained.

For example the raw text data is: [I was born in 2000, and I am in college. ], the corresponding label group to be replaced is: [Keep Keep Keep Keep Keep Keep Keep Delete Delete ^'Now Keep Keep Keep Keep], the server determines in the preset phrase set that the target phrase corresponding to ^'now is "now"; the server will keep the label The sub-text data corresponding to Keep is retained, the sub-text data corresponding to Delete is deleted, and the sub-text data corresponding to ^{' is} replaced by the target phrase, so as to obtain the text data to be detected as [I was born in 2000, now in college. ]. Then, the server determines that the text data to be detected all match the corresponding comparison text data.

207. If the target text data to be detected does not match the corresponding comparison text data, adjust the parameters of the initial optimization model to obtain the target optimization model.

If the server determines that the target text data to be detected does not match the corresponding comparison text data, the parameters of the initial optimization model are adjusted to obtain the target optimization model.

This embodiment also uses the example of step 206. The text data to be detected is "I was born in 2000, and I am in college now.", and the comparison text data is "I was born in 2000, and I am currently studying in college". The server determines that If the text data to be detected matches the comparison text data, it means that the optimization accuracy of the initial optimization model is high, and the initial optimization model is determined as the target optimization model.

It should be noted that, in this embodiment, only one example is used for description. In fact, the basis for adjusting the initial optimization model is a plurality of original text data and a plurality of corresponding comparison text data, a plurality of original text data The process of optimizing the initial optimization model is the same as the corresponding comparison file data, so other optimization processes will not be described in detail in this embodiment.

In the embodiment of the present application, combining the self-attention mechanism of the encoder, the query attention mechanism of the encoder, and the autoregressive mechanism of the decoder, multiple target label groups corresponding to multiple original text data are calculated, and then according to the multiple target labels The initial optimization model is trained in groups, and finally the initial optimization model is adjusted based on the comparison text data and the text data to be detected output by the initial optimization model to obtain the target optimization model, which makes the target optimization model suitable for a variety of optimization tasks and improves the optimization of the target optimization model. Flexibility and accuracy of optimized text.

The label-based optimization model training method in the embodiment of the present application has been described above, and the label-based optimization model training device in the embodiment of the present application is described below. Please refer to FIG. 4 , the label-based optimization model training device in the embodiment of the present application. One embodiment includes:

an acquisition module 401, configured to acquire multiple original text data and multiple comparison text data, where one original text data corresponds to one comparison text data;

The hidden layer vector calculation module 402 is used to input each original text data into the preset encoder, and obtain multiple target content hidden layer vector groups based on the self-attention mechanism and the inquiry attention mechanism;

The tag group calculation module 403 is configured to input each target content hidden layer vector group into the preset decoder, and perform tag calculation in combination with the autoregressive mechanism to obtain multiple target tag groups;

A training module 404, configured to train a model based on the multiple target label groups to obtain an initial optimization model;

Judging module 405, configured to sequentially input the plurality of original text data into the initial optimization model, obtain a plurality of text data to be detected, and determine whether each text data to be detected matches the corresponding comparison text data;

The adjustment module 406 is configured to adjust the parameters of the initial optimization model to obtain the target optimization model if the target text data to be detected does not match the corresponding comparison text data.

In the embodiment of the present application, combining the self-attention mechanism of the encoder, the query attention mechanism of the encoder, and the autoregressive mechanism of the decoder, multiple target label groups corresponding to multiple original text data are calculated, and then according to the multiple target labels The initial optimization model is trained in groups, and finally the initial optimization model is adjusted based on the comparison text data and the text data to be detected output by the initial optimization model to obtain the target optimization model, which makes the target optimization model suitable for a variety of optimization tasks and improves the optimization of the target optimization model. Flexibility and accuracy of optimized text.

Referring to FIG. 5, another embodiment of the label-based optimization model training device in the embodiment of the present application includes:

an acquisition module 401, configured to acquire multiple original text data and multiple comparison text data, where one original text data corresponds to one comparison text data;

The hidden layer vector calculation module 402 is used to input each original text data into the preset encoder, and obtain multiple target content hidden layer vector groups based on the self-attention mechanism and the inquiry attention mechanism;

The tag group calculation module 403 is configured to input each target content hidden layer vector group into the preset decoder, and perform tag calculation in combination with the autoregressive mechanism to obtain multiple target tag groups;

A training module 404, configured to train a model based on the multiple target label groups to obtain an initial optimization model;

Judging module 405, configured to sequentially input the plurality of original text data into the initial optimization model, obtain a plurality of text data to be detected, and determine whether each text data to be detected matches the corresponding comparison text data;

The adjustment module 406 is configured to adjust the parameters of the initial optimization model to obtain the target optimization model if the target text data to be detected does not match the corresponding comparison text data.

Optionally, the hidden layer vector calculation module 402 includes:

Extraction unit 4021, for extracting the corresponding original text sequence from each original text data;

The input sequence determination unit 4022 is used to input each original text sequence into a preset encoder, and determine the corresponding input sequence based on the attention mask mechanism and each original text sequence;

The hidden layer vector calculation unit 4023 is configured to perform hidden layer calculation on each input sequence based on the self-attention mechanism and the query attention mechanism, generate a corresponding content hidden layer vector group, and obtain multiple target content hidden layer vector groups.

Optionally, the input sequence determination unit 4022 can also be specifically used for:

Input each original text sequence into the preset encoder, and combine the attention mask mechanism to perform multiple iterative predictions on each original text sequence to obtain corresponding multiple position masks;

Integrate multiple position masks corresponding to each original text sequence to obtain the input sequence corresponding to each original text sequence.

Optionally, the hidden layer vector calculation unit 4023 can also be specifically used for:

Based on each input sequence, the corresponding input vector group is extracted, and the self-attention mechanism and the inquiry attention mechanism are used to calculate the target input vector group and the preset initialization vector in the first hidden layer to obtain the corresponding first content the hidden layer vector group and the corresponding first query hidden layer vector group;

Using the self-attention mechanism and the query attention mechanism, the corresponding first content hidden layer vector group and the corresponding first query hidden layer vector group are calculated in the second hidden layer to obtain the corresponding The second content hidden layer vector group and the corresponding second query hidden layer vector group;

Using the self-attention mechanism and the query-attention mechanism, the corresponding content hidden layer vector group and the corresponding query hidden layer vector group are calculated in other hidden layers according to the above steps, until the last hidden layer is generated. The corresponding target content hidden layer vector group, the corresponding target content hidden layer vector group is the content hidden layer vector group corresponding to the last hidden layer;

The self-attention mechanism and the query-attention mechanism are used to calculate other input sequences according to the above steps to obtain multiple target content hidden layer vector groups.

Optionally, the tag group calculation module 403 can also be specifically used for:

Read the corresponding content hidden layer dimension from each target content hidden layer vector group to obtain multiple content hidden layer dimensions;

Inputting the multiple content hidden layer dimensions into the preset decoder in turn, and combining the autoregressive mechanism to generate multiple decoding label groups and corresponding multiple decoding label probability groups;

Based on the decoding label probability group corresponding to each decoding label group, a target label group corresponding to each original text data is determined from each decoding label group, and a plurality of target label groups are obtained.

Optionally, the judgment module 405 includes:

The tag group generation unit 4051 to be replaced is used to sequentially input each original text data into the initial optimization model, and generate a plurality of tag groups to be replaced, each tag group to be replaced at least includes a reserved tag, a deletion tag and/or a phrase tag;

The replacement unit 4052 is used to sequentially replace the plurality of label groups to be replaced according to the preset replacement rules, obtain a plurality of text data to be detected, and determine whether each text data to be detected is consistent with the corresponding comparison text data. match.

Optionally, the replacement unit 4052 can also be specifically used for:

Determine the target phrase corresponding to the phrase tag in the preset phrase set;

In each original text data, the sub-text data corresponding to the reserved label is retained, the sub-text data corresponding to the deletion label is deleted, and the sub-text data corresponding to the phrase label is replaced with the target phrase, and the corresponding sub-text data corresponding to each original text data is generated. The text data to be detected is obtained, and a plurality of text data to be detected is obtained.

In the embodiment of the present application, combining the self-attention mechanism of the encoder, the query attention mechanism of the encoder, and the autoregressive mechanism of the decoder, multiple target label groups corresponding to multiple original text data are calculated, and then according to the multiple target labels The initial optimization model is trained in groups, and finally the initial optimization model is adjusted based on the comparison text data and the text data to be detected output by the initial optimization model to obtain the target optimization model, which makes the target optimization model suitable for a variety of optimization tasks and improves the optimization of the target optimization model. Flexibility and accuracy of optimized text.

4 and 5 above describe in detail the label-based optimization model training device in the embodiment of the present application from the perspective of modular functional entities, and the following describes the label-based optimization model training device in the embodiment of the present application from the perspective of hardware processing in detail. describe.

6 is a schematic structural diagram of a label-based optimization model training device provided by an embodiment of the present application. The label-based optimization model training device 600 may vary greatly due to different configurations or performances, and may include one or more than one Central processing units (CPU) 610 (eg, one or more processors) and memory 620, one or more storage media 630 (eg, one or more mass storage devices) that store application programs 633 or data 632. Among them, the memory 620 and the storage medium 630 may be short-term storage or persistent storage. The program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the label-based optimization model training device 600 . Furthermore, the processor 610 may be configured to communicate with the storage medium 630 to execute a series of instruction operations in the storage medium 630 on the label-based optimization model training device 600 .

The label-based optimization model training device 600 may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input and output interfaces 660, and/or, one or more operating systems 631, For example Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that the structure of the label-based optimization model training device shown in FIG. 5 does not constitute a limitation on the label-based optimization model training device, and may include more or less components than those shown in the figure, or a combination of certain components may be included. some components, or a different arrangement of components.

Further, the computer usable storage medium may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function, and the like; using the created data, etc.

The present application also provides a label-based optimization model training device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected by lines; the at least one processor The processor invokes the instructions in the memory, so that the label-based optimization model training device executes the steps in the label-based optimization model training method described above.

The present application also provides a computer-readable storage medium, and the computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on the computer, the computer performs the following steps:

Obtain multiple original text data and multiple comparison text data, one original text data corresponds to one comparison text data;

Input each original text data into the preset encoder, and obtain multiple target content hidden layer vector groups based on the self-attention mechanism and the inquiry-attention mechanism;

Input each target content hidden layer vector group into the preset decoder, and combine the autoregressive mechanism to perform label calculation to obtain multiple target label groups;

Based on the multiple target label group training models, an initial optimization model is obtained;

Inputting the plurality of original text data into the initial optimization model in turn, obtaining a plurality of text data to be detected, and judging whether each text data to be detected matches the corresponding comparison text data;

If the target text data to be detected does not match the corresponding comparison text data, the parameters of the initial optimization model are adjusted to obtain the target optimization model.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above may refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes .

The blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information to verify its Validity of information (anti-counterfeiting) and generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: The technical solutions recorded in the embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the present application.

Claims (20)

1. A label-based optimization model training method, including:

   Obtain multiple original text data and multiple comparison text data, one original text data corresponds to one comparison text data;

   Input each original text data into the preset encoder, and obtain multiple target content hidden layer vector groups based on the self-attention mechanism and the inquiry-attention mechanism;

   Input each target content hidden layer vector group into the preset decoder, and combine the autoregressive mechanism to perform label calculation to obtain multiple target label groups;

   Based on the multiple target label group training models, an initial optimization model is obtained;

   Inputting the plurality of original text data into the initial optimization model in turn, obtaining a plurality of text data to be detected, and judging whether each text data to be detected matches the corresponding comparison text data;

   If the target text data to be detected does not match the corresponding comparison text data, the parameters of the initial optimization model are adjusted to obtain the target optimization model.
2. The label-based optimization model training method according to claim 1, wherein the inputting each original text data into a preset encoder, based on a self-attention mechanism and an inquiry-attention mechanism, obtains multiple target content hidden The layer vector group includes:

   Extract the corresponding original text sequence from each original text data;

   Input each original text sequence into the preset encoder, and determine the corresponding input sequence based on the attention mask mechanism and each original text sequence;

   Based on the self-attention mechanism and the query attention mechanism, the hidden layer calculation is performed for each input sequence, and the corresponding content hidden layer vector group is generated, and multiple target content hidden layer vector groups are obtained.
3. The label-based optimization model training method according to claim 2, wherein each original text sequence is input into a preset encoder, and the corresponding input sequence is determined based on an attention mask mechanism and each original text sequence include:

   Input each original text sequence into the preset encoder, and combine the attention mask mechanism to perform multiple iterative predictions on each original text sequence to obtain corresponding multiple position masks;

   Integrate multiple position masks corresponding to each original text sequence to obtain the input sequence corresponding to each original text sequence.
4. The label-based optimization model training method according to claim 3, wherein the hidden layer calculation is performed on each input sequence based on the self-attention mechanism and the inquiry-attention mechanism, and a corresponding content hidden layer vector group is generated to obtain multiple The target content hidden layer vector group includes:

   Based on each input sequence, the corresponding input vector group is extracted, and the self-attention mechanism and the inquiry attention mechanism are used to calculate the target input vector group and the preset initialization vector in the first hidden layer to obtain the corresponding first content the hidden layer vector group and the corresponding first query hidden layer vector group;

   Using the self-attention mechanism and the query attention mechanism, the corresponding first content hidden layer vector group and the corresponding first query hidden layer vector group are calculated in the second hidden layer to obtain the corresponding The second content hidden layer vector group and the corresponding second query hidden layer vector group;

   Using the self-attention mechanism and the query-attention mechanism, the corresponding content hidden layer vector group and the corresponding query hidden layer vector group are calculated in other hidden layers according to the above steps, until the last hidden layer is generated. The corresponding target content hidden layer vector group, the corresponding target content hidden layer vector group is the content hidden layer vector group corresponding to the last hidden layer;

   The self-attention mechanism and the query-attention mechanism are used to calculate other input sequences according to the above steps to obtain multiple target content hidden layer vector groups.
5. The label-based optimization model training method according to claim 1, wherein the hidden layer vector group of each target content is input into a preset decoder, and a label calculation is performed in combination with an autoregressive mechanism to obtain a plurality of target label groups include:

   Read the corresponding content hidden layer dimension from each target content hidden layer vector group to obtain multiple content hidden layer dimensions;

   Inputting the multiple content hidden layer dimensions into the preset decoder in turn, and combining the autoregressive mechanism to generate multiple decoding label groups and corresponding multiple decoding label probability groups;

   Based on the decoding label probability group corresponding to each decoding label group, a target label group corresponding to each original text data is determined from each decoding label group, and a plurality of target label groups are obtained.
6. The label-based optimization model training method according to any one of claims 1-5, wherein the plurality of original text data are sequentially input into the initial optimization model to obtain a plurality of text data to be detected, And judging whether each text data to be detected matches the corresponding comparison text data includes:

   Input each original text data into the initial optimization model in turn, and generate a plurality of tag groups to be replaced, and each tag group to be replaced includes at least a reserved tag, a deletion tag and/or a phrase tag;

   The multiple tag groups to be replaced are sequentially replaced according to a preset replacement rule to obtain multiple text data to be detected, and it is determined whether each text data to be detected matches the corresponding comparison text data.
7. The label-based optimization model training method according to claim 6, wherein the plurality of label groups to be replaced are sequentially replaced according to a preset replacement rule, to obtain a plurality of text data to be detected, and to determine each Whether the text data to be detected matches the corresponding comparison text data includes:

   Determine the target phrase corresponding to the phrase tag in the preset phrase set;

   In each original text data, the sub-text data corresponding to the reserved label is retained, the sub-text data corresponding to the deletion label is deleted, and the sub-text data corresponding to the phrase label is replaced with the target phrase, and the corresponding sub-text data corresponding to each original text data is generated. The text data to be detected is obtained, and a plurality of text data to be detected is obtained.
8. A label-based optimization model training device, comprising a memory, a processor, and computer-readable instructions stored on the memory and executable on the processor, which are implemented when the processor executes the computer-readable instructions Follow the steps below:

   Obtain multiple original text data and multiple comparison text data, one original text data corresponds to one comparison text data;

   Input each original text data into the preset encoder, and obtain multiple target content hidden layer vector groups based on the self-attention mechanism and the inquiry-attention mechanism;

   Input each target content hidden layer vector group into the preset decoder, and combine the autoregressive mechanism to perform label calculation to obtain multiple target label groups;

   Based on the multiple target label group training models, an initial optimization model is obtained;

   Inputting the plurality of original text data into the initial optimization model in turn, obtaining a plurality of text data to be detected, and judging whether each text data to be detected matches the corresponding comparison text data;

   If the target text data to be detected does not match the corresponding comparison text data, the parameters of the initial optimization model are adjusted to obtain the target optimization model.
9. The label-based optimization model training device according to claim 8, wherein the processor further implements the following steps when executing the computer program:

   Extract the corresponding original text sequence from each original text data;

   Input each original text sequence into the preset encoder, and determine the corresponding input sequence based on the attention mask mechanism and each original text sequence;

   Based on the self-attention mechanism and the query attention mechanism, the hidden layer calculation is performed for each input sequence, and the corresponding content hidden layer vector group is generated, and multiple target content hidden layer vector groups are obtained.
10. The label-based optimization model training device according to claim 9, wherein the processor further implements the following steps when executing the computer program:

    Input each original text sequence into the preset encoder, and combine the attention mask mechanism to perform multiple iterative predictions on each original text sequence to obtain corresponding multiple position masks;

    Integrate multiple position masks corresponding to each original text sequence to obtain the input sequence corresponding to each original text sequence.
11. The label-based optimization model training device according to claim 10, wherein the processor further implements the following steps when executing the computer program:

    Based on each input sequence, the corresponding input vector group is extracted, and the self-attention mechanism and the inquiry attention mechanism are used to calculate the target input vector group and the preset initialization vector in the first hidden layer to obtain the corresponding first content the hidden layer vector group and the corresponding first query hidden layer vector group;

    Using the self-attention mechanism and the query attention mechanism, the corresponding first content hidden layer vector group and the corresponding first query hidden layer vector group are calculated in the second hidden layer to obtain the corresponding The second content hidden layer vector group and the corresponding second query hidden layer vector group;

    Using the self-attention mechanism and the query-attention mechanism, the corresponding content hidden layer vector group and the corresponding query hidden layer vector group are calculated in other hidden layers according to the above steps, until the last hidden layer is generated. The corresponding target content hidden layer vector group, the corresponding target content hidden layer vector group is the content hidden layer vector group corresponding to the last hidden layer;

    The self-attention mechanism and the query-attention mechanism are used to calculate other input sequences according to the above steps to obtain multiple target content hidden layer vector groups.
12. The label-based optimization model training device according to claim 8, wherein the processor further implements the following steps when executing the computer program:

    Read the corresponding content hidden layer dimension from each target content hidden layer vector group to obtain multiple content hidden layer dimensions;

    Inputting the multiple content hidden layer dimensions into the preset decoder in turn, and combining the autoregressive mechanism to generate multiple decoding label groups and corresponding multiple decoding label probability groups;

    Based on the decoding label probability group corresponding to each decoding label group, a target label group corresponding to each original text data is determined from each decoding label group, and a plurality of target label groups are obtained.
13. The label-based optimization model training device according to claims 8-12, wherein the processor further implements the following steps when executing the computer program:

    Input each original text data into the initial optimization model in turn, and generate a plurality of tag groups to be replaced, and each tag group to be replaced includes at least a reserved tag, a deletion tag and/or a phrase tag;

    The multiple tag groups to be replaced are sequentially replaced according to a preset replacement rule to obtain multiple text data to be detected, and it is determined whether each text data to be detected matches the corresponding comparison text data.
14. The label-based optimization model training device according to claim 13, wherein the processor further implements the following steps when executing the computer program:

    Determine the target phrase corresponding to the phrase tag in the preset phrase set;

    In each original text data, the sub-text data corresponding to the reserved label is retained, the sub-text data corresponding to the deletion label is deleted, and the sub-text data corresponding to the phrase label is replaced with the target phrase, and the corresponding sub-text data corresponding to each original text data is generated. The text data to be detected is obtained, and a plurality of text data to be detected is obtained.
15. A computer-readable storage medium, storing computer instructions in the computer-readable storage medium, when the computer instructions are executed on a computer, the computer is made to perform the following steps:

    Obtain multiple original text data and multiple comparison text data, one original text data corresponds to one comparison text data;

    Input each original text data into the preset encoder, and obtain multiple target content hidden layer vector groups based on the self-attention mechanism and the inquiry-attention mechanism;

    Input each target content hidden layer vector group into the preset decoder, and combine the autoregressive mechanism to perform label calculation to obtain multiple target label groups;

    Based on the multiple target label group training models, an initial optimization model is obtained;

    Inputting the plurality of original text data into the initial optimization model in turn, obtaining a plurality of text data to be detected, and judging whether each text data to be detected matches the corresponding comparison text data;

    If the target text data to be detected does not match the corresponding comparison text data, the parameters of the initial optimization model are adjusted to obtain the target optimization model.
16. The computer-readable storage medium of claim 15, when the computer instructions are executed on a computer, causing the computer to further perform the following steps:

    Extract the corresponding original text sequence from each original text data;

    Input each original text sequence into the preset encoder, and determine the corresponding input sequence based on the attention mask mechanism and each original text sequence;

    Based on the self-attention mechanism and the query attention mechanism, the hidden layer calculation is performed for each input sequence, and the corresponding content hidden layer vector group is generated, and multiple target content hidden layer vector groups are obtained.
17. The computer-readable storage medium of claim 16, when the computer instructions are executed on a computer, causing the computer to further perform the following steps:

    Input each original text sequence into the preset encoder, and combine the attention mask mechanism to perform multiple iterative predictions on each original text sequence to obtain corresponding multiple position masks;

    Integrate multiple position masks corresponding to each original text sequence to obtain the input sequence corresponding to each original text sequence.
18. The computer-readable storage medium of claim 17, which, when executed on a computer, causes the computer to further perform the following steps:

    Based on each input sequence, the corresponding input vector group is extracted, and the self-attention mechanism and the inquiry attention mechanism are used to calculate the target input vector group and the preset initialization vector in the first hidden layer to obtain the corresponding first content the hidden layer vector group and the corresponding first query hidden layer vector group;

    Using the self-attention mechanism and the query attention mechanism, the corresponding first content hidden layer vector group and the corresponding first query hidden layer vector group are calculated in the second hidden layer to obtain the corresponding The second content hidden layer vector group and the corresponding second query hidden layer vector group;

    Using the self-attention mechanism and the inquiry attention mechanism, according to the above steps, the corresponding content hidden layer vector groups and the corresponding query hidden layer vector groups are calculated in other hidden layers, until the last hidden layer is generated. The corresponding target content hidden layer vector group, the corresponding target content hidden layer vector group is the content hidden layer vector group corresponding to the last hidden layer;

    The self-attention mechanism and the query-attention mechanism are used to calculate other input sequences according to the above steps to obtain multiple target content hidden layer vector groups.
19. The computer-readable storage medium of claim 15, when the computer instructions are executed on a computer, causing the computer to further perform the following steps:

    Read the corresponding content hidden layer dimension from each target content hidden layer vector group to obtain multiple content hidden layer dimensions;

    Inputting the multiple content hidden layer dimensions into the preset decoder in turn, and generating multiple decoding tag groups and corresponding multiple decoding tag probability groups in combination with the autoregressive mechanism;

    Based on the decoding label probability group corresponding to each decoding label group, a target label group corresponding to each original text data is determined from each decoding label group, and a plurality of target label groups are obtained.
20. A label-based optimization model training device, the label-based optimization model training device includes:

    an acquisition module, used for acquiring multiple original text data and multiple comparison text data, one original text data corresponds to one comparison text data;

    The hidden layer vector calculation module is used to input each original text data into the preset encoder, and obtain multiple target content hidden layer vector groups based on the self-attention mechanism and the inquiry attention mechanism;

    The label group calculation module is used to input each target content hidden layer vector group into the preset decoder, and combine the autoregressive mechanism to perform label calculation to obtain multiple target label groups;

    a training module for training a model based on the multiple target label groups to obtain an initial optimization model;

    a judgment module, configured to sequentially input the plurality of original text data into the initial optimization model, obtain a plurality of text data to be detected, and judge whether each text data to be detected matches the corresponding comparison text data;

    The adjustment module is used to adjust the parameters of the initial optimization model to obtain the target optimization model if the target text data to be detected does not match the corresponding comparison text data.

Images