CN113761842A - Data processing method and device and electronic equipment - Google Patents
Data processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN113761842A CN113761842A CN202111045487.1A CN202111045487A CN113761842A CN 113761842 A CN113761842 A CN 113761842A CN 202111045487 A CN202111045487 A CN 202111045487A CN 113761842 A CN113761842 A CN 113761842A
- Authority
- CN
- China
- Prior art keywords
- data
- text
- value
- fluency
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 9
- 238000006243 chemical reaction Methods 0.000 claims abstract description 56
- 238000000034 method Methods 0.000 claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 13
- 238000012549 training Methods 0.000 claims description 169
- 230000005284 excitation Effects 0.000 claims description 133
- 230000006870 function Effects 0.000 claims description 36
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 9
- 238000013256 Gubra-Amylin NASH model Methods 0.000 description 7
- 238000004519 manufacturing process Methods 0.000 description 5
- 238000009826 distribution Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000000638 stimulation Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/151—Transformation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/355—Class or cluster creation or modification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Machine Translation (AREA)
Abstract
The application discloses a data processing method, a data processing device and electronic equipment, wherein the method comprises the following steps: obtaining target table data; performing text conversion on the target table data to obtain target text data corresponding to the target table data; the value of the fluency attribute of the target text data is greater than or equal to a fluency threshold, the value of the consistency attribute of the target text data is greater than or equal to a consistency threshold, the magnitude of the value of the fluency attribute represents the text fluency of the target text data, and the magnitude of the value of the consistency attribute represents the similarity between the target text data and the target table data.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data processing method and apparatus, and an electronic device.
Background
Currently, Table-to-Text is an important research field in natural language processing (nlp) and aims to convert tabular data into Text information convenient for a user to read and understand.
At present, a neural network model is mostly adopted to process table information into text information, but the situations of low fluency and semantic consistency of the text information still exist.
Disclosure of Invention
In view of the above, the present application provides a data processing method, an apparatus and an electronic device, as follows:
a method of data processing, comprising:
obtaining target table data;
performing text conversion on the target table data to obtain target text data corresponding to the target table data;
the value of the fluency attribute of the target text data is greater than or equal to a fluency threshold, the value of the consistency attribute of the target text data is greater than or equal to a consistency threshold, the magnitude of the value of the fluency attribute represents the text fluency of the target text data, and the magnitude of the value of the consistency attribute represents the similarity between the target text data and the target table data.
Preferably, the method for performing text conversion on the target table data to obtain target text data corresponding to the target table data includes:
performing text conversion on the target table data by using a generating model to obtain target character data corresponding to the target table data;
the generative model is constructed based on a neural network and obtained by training a plurality of training samples, wherein the training samples comprise form training data and text training data, the generative model is also trained by using excitation training data corresponding to the training samples, and the excitation training data represent fluency attributes and consistency attributes of the generative model for text conversion of the form training data.
The method preferably, the motivational training data is obtained by:
obtaining a plurality of candidate text data obtained by performing text conversion on the table training data by the generating model;
obtaining a fluency excitation value of each candidate text data, wherein the fluency excitation value represents the text fluency of the candidate text data;
obtaining a consistency incentive value between each candidate text data and the corresponding text training data, wherein the consistency incentive value represents the similarity between the candidate text data and the text training data;
and obtaining an overall excitation value according to the fluency excitation value and the consistency excitation value, wherein the overall excitation value is used as the excitation training data and is used for updating the model parameters of the generated model.
In the method, preferably, the fluency attribute of the target text data is obtained by a discriminant model;
the discriminant model is obtained by training the text training data and text sample data corresponding to the text training data, and the text sample data is obtained by performing text conversion on the form training data by using the generation model.
In the above method, preferably, the discriminant model is trained by:
obtaining a loss function value of the discriminant model according to the text sample data and the text training data, wherein the size of the loss function value represents the error size of the obtained fluency attribute of the discriminant model;
and updating the model parameters of the discriminant model by using the loss function values.
Preferably, the method for obtaining the loss function value of the discriminant model according to the text sample data and the text training data includes:
obtaining a first excitation value of the text training data and a second excitation value of the text sample data by using the discriminant model;
obtaining a loss function value based on at least the first and second excitation values.
The above method, preferably, obtaining the loss function value at least according to the first excitation value and the second excitation value, includes:
obtaining expected values of the logarithm values of the first incentive values corresponding to all the text training data to obtain first expected values;
subtracting the second excitation value corresponding to each text sample data by using 1 respectively to obtain a new second excitation value; obtaining expected values of the logarithm values of the new second excitation values corresponding to all the text sample data to obtain second expected values;
and obtaining the loss function value according to the first expected value and the second expected value.
The above method, preferably, obtaining an overall excitation value according to the fluency excitation value and the consistency excitation value, includes:
weighting and summing the fluency excitation value and the consistency excitation value by using an excitation weight parameter to obtain an overall excitation value;
wherein the excitation weight parameter is related to at least the text training data and the candidate text data.
A data processing apparatus comprising:
a target obtaining unit for obtaining target table data;
the text conversion unit is used for performing text conversion on the target table data to obtain target text data corresponding to the target table data;
the value of the fluency attribute of the target text data is greater than or equal to a fluency threshold, the value of the consistency attribute of the target text data is greater than or equal to a consistency threshold, the magnitude of the value of the fluency attribute represents the text fluency of the target text data, and the magnitude of the value of the consistency attribute represents the similarity between the target text data and the target table data.
An electronic device, comprising:
a memory for storing an application program and data generated by the application program running;
a processor for executing the application to implement: obtaining target table data; performing text conversion on the target table data to obtain target text data corresponding to the target table data; the value of the fluency attribute of the target text data is greater than or equal to a fluency threshold, the value of the consistency attribute of the target text data is greater than or equal to a consistency threshold, the magnitude of the value of the fluency attribute represents the text fluency of the target text data, and the magnitude of the value of the consistency attribute represents the similarity between the target text data and the target table data.
According to the technical scheme, after the target table data is obtained, the target text data is obtained by performing text conversion on the target table data, and the fluency attribute and the consistency attribute of the target text data can be larger than or equal to respective threshold values through the text conversion. Therefore, the fluency and semantic consistency of the obtained text can be higher through text conversion in the application.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present application;
FIG. 2 is a table data diagram in an embodiment of the present application;
FIG. 3 is an exemplary diagram of training a generative model in an embodiment of the present application;
fig. 4 and fig. 5 are partial flow charts of a data processing method according to an embodiment of the present application;
FIG. 6 is an exemplary diagram of training a generative model and a discriminative model in an embodiment of the present application;
fig. 7 is a schematic structural diagram of a data processing apparatus according to a second embodiment of the present application;
fig. 8 is another schematic structural diagram of a data processing apparatus according to a second embodiment of the present application;
fig. 9 is a schematic structural diagram of an electronic device according to a third embodiment of the present application;
fig. 10-12 are diagrams respectively illustrating examples of a server for text conversion of a table according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, a flowchart of an implementation of a data processing method provided in an embodiment of the present application is shown, where the method may be applied to an electronic device capable of performing data processing, such as a mobile phone, a pad, a notebook, a computer, or a server. The technical scheme in the embodiment is mainly used for improving the fluency and consistency of the text converted from the form.
Specifically, the method in this embodiment may include the following steps:
step 101: target table data is obtained.
The target table data refers to table data including a plurality of cells and having characters in the cells, as shown in fig. 2. The target table data is table data that needs to be converted into a text sentence.
Specifically, in the present embodiment, the target table data in the designated storage location may be obtained by data reading or the like.
Step 102: and performing text conversion on the target table data to obtain target text data corresponding to the target table data.
Specifically, in this embodiment, feature extraction may be performed on the target table data to obtain a plurality of table features corresponding to at least one feature layer, and then the table features are processed at least according to the feature weight parameters corresponding to the feature layers, so as to obtain target text data corresponding to the target table data. The obtained target text data has a fluency attribute and a consistency attribute, and the fluency attribute value of the target text data is greater than or equal to a fluency threshold value and the consistency attribute value is greater than or equal to a consistency threshold value through text conversion in the embodiment.
The fluency attribute value represents the text fluency of the target text data, and the higher the fluency attribute value is, the higher the text fluency representing the target text data is, so that the reading of a user is more convenient; the value of the consistency attribute represents the similarity between the target text data and the target table data, so that the semantic consistency between the target text data and the target table data can be represented, and the larger the value of the consistency attribute is, the higher the voice consistency between the target text data and the target table data is.
According to the above scheme, in the data processing method provided by the first embodiment of the present application, after the target table data is obtained, the target text data is obtained by performing text conversion on the target table data, and the fluency attribute and the consistency attribute of the target text data can be greater than or equal to respective thresholds through the text conversion in the present application. Therefore, the text conversion in the embodiment can enable the fluency and semantic consistency of the obtained text to be higher.
In one implementation manner, when performing text conversion on the target table data in step 102, the target text data corresponding to the target table data may be obtained by:
and performing text conversion on the target table data by using the generating model to obtain target character data corresponding to the target table data.
The generative model is constructed on the basis of the neural network, so that at least one characteristic layer is contained in the generative model. In addition, the generative model is obtained by training a plurality of training samples, the training samples include form training data and text training data, the text training data is set by manual labeling and the like aiming at the form training data, and the text training data is set aiming at the form training data and simultaneously satisfies that the value of the fluency attribute is greater than or equal to the fluency threshold value and the value of the consistency attribute is greater than or equal to the consistency threshold value.
It should be noted that, the generative model is trained by using the excitation training data corresponding to the training samples in addition to the form training data and the text training data in the training samples, where the excitation training data represents the fluency attribute and the consistency attribute of the generative model for performing text conversion on the form training data. That is, the generative model has two training directions, the first of which consists in: training is performed using the table training data and the text training data in the training sample, and the second training direction is as follows: after training the production model using the form training data and the text training data, excitation training data capable of characterizing the text conversion performance of the trained production model, for example, an overall excitation value characterizing the fluency attribute and the consistency attribute of the production model performing text conversion on the form training data, is obtained, and the production model is trained again using the excitation training data, as shown in fig. 3, so that the fluency attribute value and the consistency attribute value of the target text data obtained by performing text conversion on the target form data by the trained production model are greater than or equal to respective corresponding threshold values.
First, in this embodiment, the generative model may be used to learn the table training data and the text training data, so that the model parameters in the generative model are optimized. Specifically, in this embodiment, the table training data may be used as an input sample, and the text training data may be used as an output sample, so as to optimize the model parameters of the generated model. The model parameters of the generative model may include feature weight parameters corresponding to each feature layer in the neural network structure, and so on.
Further, the motivational training data may be obtained by the following procedure, as shown in fig. 4:
step 401: and obtaining a plurality of candidate text data obtained by performing text conversion on the table training data by the generating model.
Specifically, when text conversion is performed on the form training data by using the generative model, the generative model predicts candidate words used for forming the candidate text data by using the monte carlo search algorithm MC, for example, first a plurality of first candidate words are predicted, then second candidate words are predicted based on the first candidate words, a plurality of second candidate words corresponding to each first candidate word are predicted, and so on, as shown in fig. 5, the first candidate word to the last candidate word form the candidate text data, and thus a plurality of candidate text data are obtained.
Step 402: and obtaining a fluency excitation value of each candidate text data.
And the fluency excitation value represents the text fluency of the candidate text data.
Specifically, in this embodiment, the fluency attribute of each candidate text data may be obtained through a discriminant model, and the discriminant model may be constructed based on a classification algorithm, based on which the discriminant model takes each candidate text data as input, and may output a respective fluency excitation value of each candidate text, so as to characterize the text fluency of the corresponding candidate text data.
For example, the fluency activation value of each candidate text data can be obtained by adopting the following formula (1) in the embodiment:
wherein t is the serial number of the candidate word, Y is the sentence in the candidate text data, and the sentence is composed of the candidate words. T is the number of candidate words contained in Y, i.e., the candidate word length of the candidate text data. N is the nth result of MC, i.e. the nth candidate word, N is the total number of search results of MC (monte carlo), for example, N may be 8, i.e. it means that there are 8 predictions for each candidate word, and the value of N may be set according to the requirement. DΦIs a discriminant model, GθThe generative model, a, predicts the action of the next word. Beta represents the model parameters of the generative model,phi denotes the model parameters of the discriminant model. Based on this, the obtained Q is the fluency incentive value of the candidate text data.
Step 403: and obtaining a consistency incentive value between each candidate text data and the corresponding text training data.
The size of the consistency excitation value represents the similarity between the candidate text data and the text training data, namely the semantic consistency.
Specifically, in this embodiment, a semantic similarity between each candidate text data and the text training data may be calculated by using a bleu-related algorithm, so as to obtain a consistency incentive value between each candidate text data and the text training data.
For example, the following formula (2) may be adopted in the present embodiment to obtain the consistency incentive value between each candidate text data and the text training data:
wherein, BtAnd the bleu _ table algorithm is used for calculating a semantic consistency incentive value between the text and the table, and the obtained O is the consistency incentive value between the candidate text data and the text training data.
It should be noted that, the execution sequence of the step 402 and the step 403 may not be limited by the sequence shown in the drawing, and different technical solutions formed by executing the step 403 first and then executing the step 402, or simultaneously executing the step 403 and the step 402, and the execution sequence of the step 402 and the step 403 is different are all within the protection scope of the present application.
Step 404: and obtaining an integral excitation value according to the fluency excitation value and the consistency excitation value.
And the integral excitation value is used as excitation training data for updating or optimizing the model parameters of the generated model.
In one implementation, the fluency excitation value and the consistency excitation value may be averaged to obtain an overall excitation value in this embodiment. For example, the fluency excitation value and the consistency excitation value are natural numbers greater than 0 and less than 1, respectively, and the overall excitation value can be obtained by averaging the two natural numbers.
In another implementation manner, in this embodiment, the excitation weight parameter may be used to perform weighted summation on the smoothness excitation value and the consistency excitation value to obtain an overall excitation value. The excitation weight parameter includes a weight value of the fluency excitation value, and the weight value is subtracted by 1, so that a weight value of the consistency excitation value can be obtained, or the excitation weight parameter includes a weight value of the consistency excitation value, and the weight value is subtracted by 1, so that a weight value of the fluency excitation value can be obtained.
In a particular implementation, the magnitude of the incentive weight parameter is related to at least the text training data and the candidate text data. For example, the following formula (3) may be used in the present embodiment to obtain the weight value of the coherence excitation value:
wherein LCS (the changest common subsequence) is the longest common subsequence algorithm,
is the kth text, G, in the text training dataiIs the ith result from the MC search, and λ is the weight value of the consensus excitation value.
Based on this, the following equation (4) can be used in the present embodiment to obtain the overall excitation value:
r＝(1-λ)×rd+λ×rbleu-t (4)
wherein r isbleu-tRefers to a consistent stimulus value, r, between the text training data and the candidate text datadAnd the fluency excitation value output by the discriminant model is referred to, and based on the fluency excitation value, r is an integral excitation value.
Based on the above implementation, in this embodiment, after obtaining the overall excitation value, model parameters of the generative model, such as weight parameters of each layer in the neural network structure, may be updated through a policy gradient scheme, so as to obtain a trained generative model. The generation model is used for performing text conversion on the target table data to obtain target text data with higher respective values of the fluency attribute and the consistency attribute.
In an implementation manner, in this embodiment, the fluency attribute of the target text data may be obtained by using a discriminant model, and the consistency attribute between the target text data and the target table data may be obtained by using a bleu _ talbe algorithm, where the obtained fluency attribute value and the consistency attribute value are used to ensure the performance of text conversion performed by the generation model. Based on this, the discriminant model may be obtained by training text training data and text sample data corresponding to the text training data, where the text sample data is obtained by performing text conversion on the table training data by using the generated model, and may also be referred to as negative sample data. Therefore, in this embodiment, the discriminant model is trained by using the text training data as positive sample data and the text training data generated by the corresponding table training data as negative sample data, so that the model parameters of the discriminant model are optimized.
Specifically, the discriminant model may be trained in the following manner, as shown in FIG. 5:
step 501: and obtaining a loss function value of the discrimination model according to the text sample data and the text training data.
And the size of the loss function value represents the error of the fluency attribute obtained by the discrimination model.
Specifically, in this embodiment, a cross entropy loss function may be used to obtain a loss function value representing an error between text sample data and text training data. For example, first, a first excitation value of text training data and a second excitation value of text sample data are obtained using a discriminant model, and then a loss function value is obtained based on at least the first excitation value and the second excitation value.
For example, first, a judgment model is used to respectively obtain fluency excitation values for text training data and text sample data to obtain a first excitation value and a second excitation value, since the text training data and the text sample data are paired and have a plurality of excitation values, the obtained first excitation value and the obtained second excitation value are also respectively multiple, based on which, in this embodiment, an expectation value is obtained for the logarithm values of the first excitation values corresponding to all the text training data to obtain a first expectation value, and a new second excitation value is obtained by subtracting the second excitation value corresponding to each text sample data by 1, so that an expectation value is obtained for the logarithm values of the new second excitation values corresponding to all the text sample data to obtain a second expectation value, and finally, a loss function value is obtained according to the first expectation value and the second expectation value. For example, the first expected value and the second expected value are summed and then negative to obtain the loss function value.
In a specific implementation, the following formula (5) may be used to obtain the loss function value in this embodiment:
wherein the content of the first and second substances,
for the first expected value corresponding to the text training data,
and min (phi) is a second expected value corresponding to the text sample data, and is a loss function value.
Step 502: and updating the model parameters of the discriminant model by using the loss function values.
Based on this, in this embodiment, according to the above training scheme, the generation model and the discrimination model may be alternately trained multiple times by using multiple sets of table training data and text training data until the overall excitation value and the loss function value both satisfy the corresponding convergence condition. For example, in training the generative and discriminative models, until: the integral excitation value corresponding to the generated model approaches to 1 or the variation of each round of training is smaller than the corresponding threshold, and the loss function value of the discriminant model approaches to 0 or the variation of each round of training is smaller than the corresponding threshold.
For example, as shown in fig. 6, the generative model and the discriminative model are alternately trained using sets of training samples consisting of tabular training data and textual training data according to the following training procedure until the two models converge:
firstly, training a generated model by using table training data and text training data in a training sample to update model parameters of the generated model;
secondly, performing text conversion on the form training data by using a trained generation model to obtain candidate text data in the text conversion process of the generation model, on the basis, obtaining a fluency excitation value of the candidate text data by using a discrimination model, obtaining a consistency excitation value between the candidate text data and text sample data by using a bleu _ table algorithm, and updating model parameters of the generation model again by using the integral excitation value corresponding to the fluency excitation value and the consistency excitation value to obtain a generation model after primary training;
and then, performing text conversion on the table training data by using the generated model after the primary training to obtain text sample data, taking the text sample data as a negative sample and the text training data as a positive sample to obtain a loss function value of the discriminant model, and updating the model parameters of the discriminant model based on the loss function value.
Fig. 7 is a schematic structural diagram of a data processing apparatus according to a second embodiment of the present disclosure, where the apparatus may be applied to an electronic device capable of performing data processing, such as a mobile phone, a pad, a notebook, a computer, or a server. The technical scheme in the embodiment is mainly used for improving the fluency and consistency of the text converted from the form.
Specifically, the apparatus in this embodiment may include the following units:
a target obtaining unit 701 for obtaining target table data;
a text conversion unit 702, configured to perform text conversion on the target table data to obtain target text data corresponding to the target table data;
the value of the fluency attribute of the target text data is greater than or equal to a fluency threshold, the value of the consistency attribute of the target text data is greater than or equal to a consistency threshold, the magnitude of the value of the fluency attribute represents the text fluency of the target text data, and the magnitude of the value of the consistency attribute represents the similarity between the target text data and the target table data.
As can be seen from the foregoing solutions, in the data processing apparatus provided in the second embodiment of the present application, after the target form data is obtained, the target text data is obtained by performing text conversion on the target form data, and the fluency attribute and the consistency attribute of the target text data can be greater than or equal to respective thresholds through the text conversion in the present application. Therefore, the text conversion in the embodiment can enable the fluency and semantic consistency of the obtained text to be higher.
In one implementation, the text conversion unit 702 is specifically configured to: performing text conversion on the target table data by using a generating model to obtain target character data corresponding to the target table data;
based on this, the following units may also be provided in the present embodiment, as shown in fig. 8:
and a model training unit 703 for training the generated model and the discriminant model.
The generative model is constructed based on a neural network and obtained by training a plurality of training samples, wherein the training samples comprise form training data and text training data, the generative model is also trained by using excitation training data corresponding to the training samples, and the excitation training data represent fluency attributes and consistency attributes of the generative model for text conversion of the form training data.
Optionally, the incentive training data is obtained by: obtaining a plurality of candidate text data obtained by performing text conversion on the table training data by the generating model; obtaining a fluency excitation value of each candidate text data, wherein the fluency excitation value represents the text fluency of the candidate text data; obtaining a consistency incentive value between each candidate text data and the corresponding text training data, wherein the consistency incentive value represents the similarity between the candidate text data and the text training data; and obtaining an overall excitation value according to the fluency excitation value and the consistency excitation value, wherein the overall excitation value is used as the excitation training data and is used for updating the model parameters of the generated model.
In one implementation, the fluency attribute of the target text data is obtained through a discriminant model; the discriminant model is obtained by training the text training data and text sample data corresponding to the text training data, and the text sample data is obtained by performing text conversion on the form training data by using the generation model.
Optionally, the discriminant model is trained in the following manner: obtaining a loss function value of the discriminant model according to the text sample data and the text training data, wherein the size of the loss function value represents the error size of the obtained fluency attribute of the discriminant model; and updating the model parameters of the discriminant model by using the loss function values.
Based on the above implementation, the loss function value is obtained by: obtaining a first excitation value of the text training data and a second excitation value of the text sample data by using the discriminant model; obtaining a loss function value based on at least the first and second excitation values. For example, an expectation value is taken for the logarithm values of the first incentive values corresponding to all the text training data to obtain a first expectation value; subtracting the second excitation value corresponding to each text sample data by using 1 respectively to obtain a new second excitation value; obtaining expected values of the logarithm values of the new second excitation values corresponding to all the text sample data to obtain second expected values; and obtaining the loss function value according to the first expected value and the second expected value.
In one implementation, the overall excitation value is obtained by: weighting and summing the fluency excitation value and the consistency excitation value by using an excitation weight parameter to obtain an overall excitation value; wherein the excitation weight parameter is related to at least the text training data and the candidate text data.
It should be noted that, for the specific implementation of the above units, reference may be made to the corresponding contents in the foregoing, and details are not described here.
Referring to fig. 9, a schematic structural diagram of an electronic device according to a third embodiment of the present disclosure is provided, where the electronic device may be an electronic device capable of performing data processing, such as a mobile phone, a pad, a notebook, a computer, or a server. The technical scheme in the embodiment is mainly used for improving the fluency and consistency of the text converted from the form.
Specifically, the electronic device in this embodiment may include the following structure:
a memory 901 for storing an application program and data generated by the application program running;
a processor 902 for executing the application to implement: obtaining target table data; performing text conversion on the target table data to obtain target text data corresponding to the target table data; the value of the fluency attribute of the target text data is greater than or equal to a fluency threshold, the value of the consistency attribute of the target text data is greater than or equal to a consistency threshold, the magnitude of the value of the fluency attribute represents the text fluency of the target text data, and the magnitude of the value of the consistency attribute represents the similarity between the target text data and the target table data.
According to the scheme, in the electronic device provided by the third embodiment of the application, after the target table data is obtained, the target text data is obtained by performing text conversion on the target table data, and the fluency attribute and the consistency attribute of the target text data can be larger than or equal to respective threshold values through the text conversion in the application. Therefore, the text conversion in the embodiment can enable the fluency and semantic consistency of the obtained text to be higher.
Specifically, taking an electronic device as an example, the following describes the technical solution in the present application in detail:
first, the technical solution in the present application trains the generator and the discriminator therein by establishing a counterstudy mechanism based on the GAN model, as shown in fig. 10. The GAN model mainly includes two modules: and the generator G is a generation model in the text, and the discriminator D is a discrimination model in the text. The generator can generate corresponding text description according to the input form data; the discriminator can evaluate whether the input text description is real data, namely the accurate text description of manual labeling, so as to obtain a fluency incentive value, and meanwhile, a bleu correlation algorithm is used for comparing the input text with a corresponding table, so as to obtain a semantic information consistency incentive value through calculation, and the semantic information consistency incentive value and the corresponding table are effectively fused so as to be used for updating the generator G. Based on this, the technical scheme of this application mainly has following core main points:
1. pre-training a generator G in the GAN model on the basis of a training data set, wherein the training data set is a training sample in the preceding text;
2. generating a negative sample by using G, and using the negative sample to pre-train a discriminator D in the GAN model;
3. generating a plurality of candidate text descriptions (data) using an MC search and calculating their total excitation, i.e. overall excitation values;
4. updating a generator G by adopting strategy gradient, and generating a negative sample for updating the discriminator D by using the updated G;
5. continuously and iteratively updating G and D to train the whole GAN framework;
6. a text description of the input form is generated using a trained generator.
In connection with the training flow shown in fig. 11, formula (1) is used to calculate fluency incentive for generating text and formula (2) is used to calculate semantic information consistency of the generated text with the corresponding table content during the training of the model. To effectively fuse the above two excitations, the final excitation values can be calculated using equations (3) and (4), and the generator parameters are updated using a strategic gradient method. We then use the updated G to generate negative samples, and train the arbiter using a loss function such as equation (5).
The algorithm of the whole training process is as follows:
inputting: form training data R and text training data S corresponding to the form training data R;
training a target: joint training generator G and discriminator D
1. Initializing G and D by using random parameters, namely initializing model parameters beta and phi of G and D respectively;
2. pre-training generator G using R and S;
3. saving the model parameters of G to beta;
4. inputting R to G, wherein the generated text is used as a negative sample, and S is used as a positive sample;
5. discriminator D is pre-trained using positive and negative samples.
The algorithm of the iterative training generator is as follows:
1. when each word is generated by G, 8 candidates are obtained by using the MC algorithm and form (incomplete) sentences with the previous texts, namely candidate text data in the previous texts;
2. calculating fluency stimuli of each candidate text data using formula (1);
3. calculating information consistency excitation of each candidate text data and the original Table data by using a BLEU-Table algorithm;
4. fusing the two excitations by using a formula (4) to obtain an integral excitation;
5. model parameters of the generator are updated based on the overall excitation using the policy gradients.
The algorithm of the iterative training arbiter is as follows:
1. generating negative examples using updated generators
2. Combining positive samples such as S and negative samples to form training data;
3. the k-round discriminator is trained using a cross entropy loss function, where k may be 3.
Thereafter, the algorithm of the iterative training generator and the algorithm of the iterative training discriminator are repeatedly executed, thereby alternately training the generator and the discriminator until the entire GAN model converges.
In summary, the GAN model in the present application can be applied to various application scenarios, for example, in writing english composition, a model sentence with complete semantics and smooth grammar can be generated according to the keywords provided in the table, so as to guide the writing of student composition, as shown in fig. 12. Therefore, the method and the device can simultaneously guarantee fluency and semantic information consistency of the generated text by adopting the GAN model, and can reasonably fuse fluency stimulation and semantic consistency stimulation and improve the performance of the model by providing a stimulation fusion method in specific implementation.
When the technical solution of the present application is used, statistics may be performed on distribution information of text data generated on a test set, such as probability distributions of words of different levels (the level of a word is determined by looking up a dictionary) and probability distributions of different sentence structures (obtained by a syntax parser), and in addition, statistics may be performed on distribution information of text data generated on the test set by using other methods, and if it is found that statistical information obtained by other methods is consistent with statistical information obtained in the technical solution of the present application, the other methods are considered to belong to the same inventive concept as the technical solution of the present application.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method of data processing, comprising:
obtaining target table data;
performing text conversion on the target table data to obtain target text data corresponding to the target table data;
the value of the fluency attribute of the target text data is greater than or equal to a fluency threshold, the value of the consistency attribute of the target text data is greater than or equal to a consistency threshold, the magnitude of the value of the fluency attribute represents the text fluency of the target text data, and the magnitude of the value of the consistency attribute represents the similarity between the target text data and the target table data.
2. The method of claim 1, performing text conversion on the target table data to obtain target text data corresponding to the target table data, comprising:
performing text conversion on the target table data by using a generating model to obtain target character data corresponding to the target table data;
the generative model is constructed based on a neural network and obtained by training a plurality of training samples, wherein the training samples comprise form training data and text training data, the generative model is also trained by using excitation training data corresponding to the training samples, and the excitation training data represent fluency attributes and consistency attributes of the generative model for text conversion of the form training data.
3. The method of claim 2, the motivational training data obtained by:
obtaining a plurality of candidate text data obtained by performing text conversion on the table training data by the generating model;
obtaining a fluency excitation value of each candidate text data, wherein the fluency excitation value represents the text fluency of the candidate text data;
obtaining a consistency incentive value between each candidate text data and the corresponding text training data, wherein the consistency incentive value represents the similarity between the candidate text data and the text training data;
and obtaining an overall excitation value according to the fluency excitation value and the consistency excitation value, wherein the overall excitation value is used as the excitation training data and is used for updating the model parameters of the generated model.
4. The method of claim 2 or 3, wherein the fluency attribute of the target text data is obtained by a discriminant model;
the discriminant model is obtained by training the text training data and text sample data corresponding to the text training data, and the text sample data is obtained by performing text conversion on the form training data by using the generation model.
5. The method of claim 4, the discriminant model being trained by:
obtaining a loss function value of the discriminant model according to the text sample data and the text training data, wherein the size of the loss function value represents the error size of the obtained fluency attribute of the discriminant model;
and updating the model parameters of the discriminant model by using the loss function values.
6. The method of claim 5, obtaining a loss function value for the discriminative model from the text sample data and the text training data, comprising:
obtaining a first excitation value of the text training data and a second excitation value of the text sample data by using the discriminant model;
obtaining a loss function value based on at least the first and second excitation values.
7. The method of claim 6, obtaining a loss function value from at least the first and second excitation values, comprising:
obtaining expected values of the logarithm values of the first incentive values corresponding to all the text training data to obtain first expected values;
subtracting the second excitation value corresponding to each text sample data by using 1 respectively to obtain a new second excitation value; obtaining expected values of the logarithm values of the new second excitation values corresponding to all the text sample data to obtain second expected values;
and obtaining the loss function value according to the first expected value and the second expected value.
8. The method of claim 3, deriving an overall incentive value from the fluency incentive value and the conformance incentive value, comprising:
weighting and summing the fluency excitation value and the consistency excitation value by using an excitation weight parameter to obtain an overall excitation value;
wherein the excitation weight parameter is related to at least the text training data and the candidate text data.
9. A data processing apparatus comprising:
a target obtaining unit for obtaining target table data;
the text conversion unit is used for performing text conversion on the target table data to obtain target text data corresponding to the target table data;
the value of the fluency attribute of the target text data is greater than or equal to a fluency threshold, the value of the consistency attribute of the target text data is greater than or equal to a consistency threshold, the magnitude of the value of the fluency attribute represents the text fluency of the target text data, and the magnitude of the value of the consistency attribute represents the similarity between the target text data and the target table data.
10. An electronic device, comprising:
a memory for storing an application program and data generated by the application program running;
a processor for executing the application to implement: obtaining target table data; performing text conversion on the target table data to obtain target text data corresponding to the target table data; the value of the fluency attribute of the target text data is greater than or equal to a fluency threshold, the value of the consistency attribute of the target text data is greater than or equal to a consistency threshold, the magnitude of the value of the fluency attribute represents the text fluency of the target text data, and the magnitude of the value of the consistency attribute represents the similarity between the target text data and the target table data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111045487.1A CN113761842A (en) | 2021-09-07 | 2021-09-07 | Data processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111045487.1A CN113761842A (en) | 2021-09-07 | 2021-09-07 | Data processing method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113761842A true CN113761842A (en) | 2021-12-07 |
Family
ID=78793668
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111045487.1A Pending CN113761842A (en) | 2021-09-07 | 2021-09-07 | Data processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113761842A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109829164A (en) * | 2019-02-01 | 2019-05-31 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating text |
| CN110688479A (en) * | 2019-08-19 | 2020-01-14 | 中国科学院信息工程研究所 | Evaluation method and sequencing network for generating abstract |
| CN111651970A (en) * | 2020-07-14 | 2020-09-11 | 华侨大学 | Method, device and equipment for generating text based on table type data and storage medium |
| US20210056417A1 (en) * | 2019-08-22 | 2021-02-25 | Google Llc | Active learning via a sample consistency assessment |
| CN112612868A (en) * | 2020-11-24 | 2021-04-06 | 中国传媒大学 | Text rapid generation method, computer readable storage medium and electronic device |
-
2021
- 2021-09-07 CN CN202111045487.1A patent/CN113761842A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109829164A (en) * | 2019-02-01 | 2019-05-31 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating text |
| CN110688479A (en) * | 2019-08-19 | 2020-01-14 | 中国科学院信息工程研究所 | Evaluation method and sequencing network for generating abstract |
| US20210056417A1 (en) * | 2019-08-22 | 2021-02-25 | Google Llc | Active learning via a sample consistency assessment |
| CN111651970A (en) * | 2020-07-14 | 2020-09-11 | 华侨大学 | Method, device and equipment for generating text based on table type data and storage medium |
| CN112612868A (en) * | 2020-11-24 | 2021-04-06 | 中国传媒大学 | Text rapid generation method, computer readable storage medium and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109783817B (en) | Text semantic similarity calculation model based on deep reinforcement learning | |
| CN111460807B (en) | Sequence labeling method, device, computer equipment and storage medium | |
| CN111241243A (en) | Knowledge measurement-oriented test question, knowledge and capability tensor construction and labeling method | |
| CN113435211B (en) | Text implicit emotion analysis method combined with external knowledge | |
| WO2021174827A1 (en) | Text generation method and appartus, computer device and readable storage medium | |
| CN111694937A (en) | Interviewing method and device based on artificial intelligence, computer equipment and storage medium | |
| CN112069295A (en) | Similar question recommendation method and device, electronic equipment and storage medium | |
| CN113326374B (en) | Short text emotion classification method and system based on feature enhancement | |
| CN111368082A (en) | Emotion analysis method for domain adaptive word embedding based on hierarchical network | |
| CN115309910B (en) | Language-text element and element relation joint extraction method and knowledge graph construction method | |
| CN110991193A (en) | Translation matrix model selection system based on OpenKiwi | |
| CN110852071A (en) | Knowledge point detection method, device, equipment and readable storage medium | |
| CN111353040A (en) | GRU-based attribute level emotion analysis method | |
| CN117094383B (en) | Joint training method, system, equipment and storage medium for language model | |
| CN116910185B (en) | Model training method, device, electronic equipment and readable storage medium | |
| CN110929532B (en) | Data processing method, device, equipment and storage medium | |
| CN113705207A (en) | Grammar error recognition method and device | |
| Arifin et al. | Automatic essay scoring for Indonesian short answers using siamese Manhattan long short-term memory | |
| CN115840884A (en) | Sample selection method, device, equipment and medium | |
| CN114065741B (en) | Method, device, apparatus and medium for verifying authenticity of a representation | |
| CN116822530A (en) | Knowledge graph-based question-answer pair generation method | |
| CN113761842A (en) | Data processing method and device and electronic equipment | |
| CN115114937A (en) | Text acquisition method and device, computer equipment and storage medium | |
| WO2021176714A1 (en) | Learning device, information processing device, learning method, information processing method, and program | |
| CN110633363B (en) | Text entity recommendation method based on NLP and fuzzy multi-criterion decision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |