CN115204118A - Article generation method and device, computer equipment and storage medium - Google Patents
Article generation method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN115204118A CN115204118A CN202210816045.0A CN202210816045A CN115204118A CN 115204118 A CN115204118 A CN 115204118A CN 202210816045 A CN202210816045 A CN 202210816045A CN 115204118 A CN115204118 A CN 115204118A
- Authority
- CN
- China
- Prior art keywords
- target
- sentence
- training
- vocabulary
- distribution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/284—Lexical analysis, e.g. tokenisation or collocates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention relates to the technical field of artificial intelligence, and discloses an article generation method, an article generation device, computer equipment and a storage medium. The method comprises the steps of obtaining a target theme, and processing the target theme, target external data corresponding to the target theme and target text data corresponding to the current moment by adopting a target vocabulary encoder to obtain target structure distribution; a target sentence encoder is adopted to obtain a target sentence sequence corresponding to the current moment according to the target structure distribution corresponding to the current moment, the target subject and the target upper text data corresponding to the current moment; and obtaining corresponding sentence data corresponding to the current moment after processing, obtaining target structure distribution with stronger correlation based on the sentence data corresponding to the current moment and the target upper text data corresponding to the current moment, and generating corresponding sentence data by combining the target structure distribution with the upper text so as to generate an article with higher correlation with the theme.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an article generation method, an article generation device, computer equipment and a storage medium.
Background
Natural Language Generation (NLG) is a fundamental and challenging task in the field of natural language processing, and nowadays creativity is written as a hot spot area for exploring the margin of AI cognitive intelligence. Topic-to-Essay Generation (TEG) technology aims to generate a semantically coherent article with strong diversity by a series of subject words, so as to imitate human logic deduction and subject creation. Article generation techniques can provide support for many downstream applications, such as news editing, story generation, and also heuristic teaching.
As a mature end-to-end sequence generation architecture is applied in the field of machine translation, the generation effect is greatly improved on a TEG task by introducing RNN, CNN and a transform seq2seq model. However, in the prior art, an article generation model based on a deep neural network exists, an information amount gap exists between an input topic and an output article, and the obtained structure distribution is not obvious enough during training, so that the relevance between words is low, and the relevance between the generated overall article and the topic is poor.
Disclosure of Invention
The embodiment of the invention provides an article generation method, an article generation device, computer equipment and a storage medium, and solves the problem that the relevance between an article generated by an existing article generation model and a theme is poor.
The embodiment of the invention provides an article generation method, which comprises the following steps:
acquiring a target theme;
processing the target theme, target external data corresponding to the target theme and target upper data corresponding to the current moment by adopting a target vocabulary encoder to obtain target structure distribution corresponding to the current moment; the target upper data corresponding to the current moment is the summary of all sentence data before the current moment
Processing the target theme, the target structure distribution corresponding to the current moment and the target upper data corresponding to the current moment by adopting a target sentence encoder to obtain a target sentence sequence corresponding to the current moment;
processing the target sentence sequence corresponding to the current moment to acquire sentence data corresponding to the current moment;
and acquiring a target article based on the sentence data corresponding to the current moment and the target upper text data corresponding to the current moment.
An embodiment of the present invention further provides an article generating apparatus, including:
the target theme acquisition module is used for acquiring a target theme;
a target structure distribution acquisition module which adopts a target vocabulary encoder to process the target theme, target external data corresponding to the target theme and target upper data corresponding to the current moment so as to acquire target structure distribution corresponding to the current moment;
a target sentence sequence acquisition module which adopts a target sentence encoder to process the target theme, the target structure distribution corresponding to the current moment and the target upper text data corresponding to the current moment so as to acquire a target sentence sequence corresponding to the current moment;
the sentence data acquisition module is used for processing the target sentence sequence corresponding to the current moment to acquire sentence data corresponding to the current moment;
and the target article acquisition module is used for acquiring a target article based on the sentence data corresponding to the current moment and the target upper text data corresponding to the current moment.
An embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the article generation method when executing the computer program.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the article generation method described above.
According to the article generation method, the article generation device, the computer equipment and the storage medium, the target theme is obtained, and the target external data corresponding to the target theme and the target text data corresponding to the current moment are processed by adopting the target vocabulary encoder, so that the target structure distribution corresponding to the current moment corresponding to each vocabulary with strong correlation is obtained; a target sentence encoder is adopted to obtain a target sentence sequence corresponding to the current moment according to the target structure distribution corresponding to the current moment, the target subject and the target upper text data corresponding to the current moment; the method comprises the steps of obtaining corresponding sentence data corresponding to the current moment after processing, and generating a target article with high theme relevance based on the sentence data corresponding to the current moment and the target upper text data corresponding to the current moment.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive labor.
FIG. 1 is a diagram of an application environment of an article generation method according to an embodiment of the present invention;
FIG. 2 is a flow chart of an article generation method in one embodiment of the present invention;
FIG. 3 is another flow chart of a method for article generation in an embodiment of the invention;
FIG. 4 is another flow chart of a method for article generation in an embodiment of the present invention;
FIG. 5 is another flow chart of a method for article generation in an embodiment of the invention;
FIG. 6 is a diagram of an article generation apparatus according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a computer device according to an embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The article generation method provided by the embodiment of the invention can be applied to the application environment shown in fig. 1. As shown in fig. 1, a client (computer device) communicates with a server through a network. The client (computer device) includes, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, cameras, and portable wearable devices. The server may be an independent server, or may be a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like.
The article generation method provided by the embodiment of the invention can be applied to an application environment shown in fig. 1. Specifically, the article generation method is applied to an article generation system, the article generation system includes a client and a server shown in fig. 1, the client and the server communicate with each other through a network, and a corresponding target article is acquired for a target topic by using an article generation model, so that the relevance between the generated article and the target topic is improved.
In an embodiment, as shown in fig. 2, an article generation method is provided, which is described by taking the application of the method to the server in fig. 1 as an example, and includes the following steps:
s201: acquiring a target theme;
s202: processing a target theme, target external data corresponding to the target theme and target upper text data corresponding to the current moment by adopting a target vocabulary encoder to obtain target structure distribution corresponding to the current moment; and the target upper data corresponding to the current moment is the summary of all sentence data before the current moment.
S203: processing a target theme, target structure distribution corresponding to the current moment and target upper text data corresponding to the current moment by adopting a target sentence encoder to obtain a target sentence sequence corresponding to the current moment;
s204: processing a target sentence sequence corresponding to the current moment to acquire sentence data corresponding to the current moment;
s205: and acquiring the target article based on the sentence data corresponding to the current moment and the target upper text data corresponding to the current moment.
Wherein, the structural distribution is used to express the distribution of the examples on the text and the dependency relationship between the entities.
As an example, in step S201, the server obtains a target topic, the target topic is determined according to input or selection of a user, and an article generation model is input according to the corresponding target topic to generate a final target article related to the target topic, target context data corresponding to the current time is a summary of sentence data generated by the target sentence encoder before the current time, and if there is no target context data, corresponding target external data is directly obtained according to the target topic to obtain corresponding target structure distribution.
As an example, in step S202, after the target topic is confirmed, the server processes the target topic, the target external data corresponding to the target topic, and the target context data corresponding to the current time by using a trained target vocabulary encoder, so as to provide a target structure distribution suitable for the corresponding at present, which is used for generating a sentence at the current time, and determining a corresponding range and structure. In the present example of the present invention,
the vocabulary Encoder includes, but is not limited to, a variational auto-Encoder (VAE), a Conditional variational auto-Encoder (CVAE), a Generative adaptive network model (GAN), or a Conditional Generative adaptive network model (CGAN).
In the example, a vocabulary encoder is built by using a CVAE model, the vocabulary encoder is named as word-Graph CVAE, the CVAE model is one of generated models, the main aim is to generate new sampling data in sample data learning distribution, extract structural distribution by using an encoder network through a sample with a label, and transmit the structural distribution to a decoder network to obtain new sample data similar to the sample data.
As an example, in step S203, after obtaining the target structure distribution corresponding to the current time, the server uses a trained target sentence encoder to process the target topic, the target structure distribution corresponding to the current time, and the target text data corresponding to the current time, and after splicing the target vocabulary sequences according to the corresponding structures, obtains the corresponding target sentence sequence.
In the example, according to a target theme and target context data corresponding to the current time, sampling processing is performed on target structure distribution corresponding to the current time, and at least one target vocabulary sequence is obtained; and splicing at least one target vocabulary sequence to obtain a target sentence sequence. And confirming a corresponding target vocabulary sequence through the target structure distribution corresponding to the current moment determined by the target vocabulary encoder so as to ensure the correlation between the finally output sentence data and the target theme.
In this example, the vocabulary Encoder is named a sense CVAE by constructing the vocabulary Encoder using a CVAE model, wherein the sentence Encoder is constructed by, but not limited to, a Variational Auto-Encoder (VAE), a conditional Variational Auto-Encoder (CVAE), a Generative Accommodation Network (GAN), or a Conditional Generative Accommodation Network (CGAN).
As an example, in step S204, the server performs decoding processing on the acquired target sentence sequence corresponding to the current time, converts the target sentence sequence into sentence data corresponding to the current time, and inputs the decoded sentence data into the tail end of the above data corresponding to the current time.
As an example, in step S204, based on the obtained sentence data corresponding to the current time and the target text data corresponding to the current time, if the sentence data corresponding to the previous time is generated and the text generation task is completed, the server splices the sentence data corresponding to the current time and the target text data corresponding to the current time to obtain the target text corresponding to the target subject. The target subject and the corresponding target article can also be used as training data for subsequent confrontation training so as to improve the subject relevance of the article generation model.
In this example, a target topic is obtained, and a target vocabulary encoder is adopted to process the target topic, target external data corresponding to the target topic, and target context data corresponding to a current time, so as to obtain target structure distribution corresponding to the current time corresponding to each vocabulary with strong correlation; a target sentence encoder is adopted to obtain a target sentence sequence corresponding to the current moment according to the target structure distribution corresponding to the current moment, the target subject and the target upper text data corresponding to the current moment; the method comprises the steps of obtaining corresponding sentence data corresponding to the current moment after processing, and generating a target article with high theme relevance based on the sentence data corresponding to the current moment and the target upper text data corresponding to the current moment.
In an embodiment, as shown in fig. 3, in step S202, processing the target topic, the target external data corresponding to the target topic, and the target context data corresponding to the current time by using the target vocabulary encoder, and acquiring the target structure distribution corresponding to the current time includes:
s301: acquiring the upper structure distribution corresponding to the current moment according to the target upper data corresponding to the current moment;
s302: acquiring external common sense data of the target according to the target theme and the target upper data corresponding to the current moment;
s303: and acquiring target structure distribution corresponding to the current moment based on the external common sense data and the above structure distribution corresponding to the current moment.
As an example, in step S301, the server obtains, according to the obtained target context data corresponding to the current time, the context distribution corresponding to the current time corresponding to the target context data corresponding to the current time, so as to be used for obtaining the subsequent target context distribution.
As an example, in step S302, the server obtains the corresponding target external common sense data according to the target topic and the target context data corresponding to the current time, and may use a preset external common sense database or a cloud external common sense database to obtain the target external common sense data. The conventional generative model only utilizes input information, the understanding level is low, and if external common sense data are introduced into the model, the generative model learns more features, the understanding level of the model is improved, and the task effect can be improved.
As an example, in step S303, the server obtains the target structure distribution corresponding to the current time based on the external common sense data and the above structure distribution corresponding to the current time.
In this example, the target context data corresponding to the current time and the target topic are obtained to obtain the corresponding target external common sense data, and after the target context data corresponding to the current time is converted into the context distribution, the target structure distribution corresponding to the current time is obtained based on the external common sense data and the context distribution corresponding to the current time. And by utilizing external common knowledge data, the generated model learns more features, and the richness of the distribution of the target structure is improved.
In an embodiment, as shown in fig. 4, in step S205, acquiring a target article based on sentence data corresponding to a current time and target upper text data corresponding to the current time, including:
s401: acquiring an original article based on sentence data corresponding to the current moment and target upper text data corresponding to the current moment;
s402: judging whether the original article meets the format condition corresponding to the target theme or not;
s403: if the original article meets the format condition, determining the original article as a target article;
s404: and if the original article does not meet the format condition, determining the original article as the target upper text data at the next moment.
As an example, in step S401, the server performs a concatenation process based on the sentence data corresponding to the current time and the target text data corresponding to the current time. Usually, the sentence data corresponding to the current moment is a sentence after the target text data corresponding to the current moment, and the original article is obtained by adding the sentence data corresponding to the current moment into the tail end of the target text data corresponding to the current moment.
As an example, in step S402, the server determines whether the original article satisfies the format condition corresponding to the target topic according to the acquired original article. The format conditions include, but are not limited to, the number of words required by the user, the chapter required by the user, and the preset specific identifier.
In this example, the end of generation is indicated by setting the < eos > flag, and the end of output flag is deemed to be the end of generation when the next sentence sequence is generated. Since the output sequence is output on a word-by-word (word) basis when the output sequence is decoded, the tag corresponds to a word, and the article generation is considered to be completed when the < eos > tag is output, which can be learned during training.
As an example, in step S403, if the original article satisfies the format condition according to the determination result, the server determines the original article as the target article.
As an example, in step S404, if the original sentence does not satisfy the format condition according to the determination result, the server determines the original sentence as the target text data at the next time, and continues to loop through steps S202 to S205 in this example.
In the example, the sentence data corresponding to the current moment and the target text data corresponding to the current moment are spliced to obtain the original article, whether the original article is finished is judged according to the format condition, and then the article generation is circulated, so that the setting is effectively carried out according to the user requirement, and the article generation efficiency is improved.
In another embodiment, as shown in fig. 5, before acquiring the target topic in step S201, the article generating method further includes:
s501: acquiring training external common sense data and training articles related to a training subject;
s502: training the vocabulary encoder by using training external common sense data to obtain hidden distribution of a vocabulary structure;
s503: training a sentence encoder by using a training article to obtain the hidden distribution of sentence structures;
s504: and performing countermeasure training according to the hidden distribution of the vocabulary structure and the hidden distribution of the sentence structure to obtain a target vocabulary encoder and a target sentence encoder.
The structure hidden distribution is a dependency relationship among all features formed by unobservable random variables and is obtained by deducing through the current existing sample.
As an example, in step S501, the server obtains training external knowledge data and training articles related to a training topic for training the article generation model. In this example, the corresponding training external common sense data is matched in the external common sense database according to the training topic and the corresponding training article.
As an example, in step S502, the server trains the vocabulary encoder according to the acquired training external common sense data, and the corresponding training subjects and training articles, so as to capture and acquire hidden distribution of vocabulary structure.
As an example, in step S503, the server trains the sentence encoder according to the obtained training article and the corresponding training topic, and obtains the sentence structure hidden distribution.
As an example, in step S504, the server performs countermeasure training according to the hidden distribution of vocabulary structure and the hidden distribution of sentence structure, and obtains a target vocabulary encoder and a target sentence encoder. Among them, the countermeasure training of the article generation model can be performed by the countermeasure network or by this KL divergence training.
In this example, training external common sense data and training articles related to a training topic are obtained to train a corresponding vocabulary encoder and sentence encoder, obtain vocabulary structure implicit distribution and sentence structure implicit distribution, and after performing countermeasure training, obtain a target vocabulary encoder and a target sentence encoder with more significant characteristics.
In an embodiment, step S502, training the vocabulary encoder by using the training external common sense data to obtain the hidden distribution of the vocabulary structure includes:
s5021: the vocabulary encoder comprises a vocabulary prior neural network and a vocabulary posterior neural network; inputting training upper-text data in a training article into a vocabulary prior neural network for model training to obtain first vocabulary hidden distribution;
s5022: processing the training external common sense data to obtain training external common sense relation embedding;
s5023: embedding and training external common sense data in a training theme, training external common sense data and training external common sense relation, inputting a vocabulary posterior neural network for model training, and acquiring second vocabulary hidden distribution;
s5024: and acquiring the hidden distribution of the vocabulary structure according to the hidden distribution of the first vocabulary and the hidden distribution of the second vocabulary.
As an example, in step S5021, the server inputs training context data in a training article into a vocabulary prior neural network for model training, and obtains a first vocabulary implicit distribution. In this example, the vocabulary encoder includes a vocabulary prior neural network and a vocabulary posterior neural network, the prior neural network is used for learning a prediction method according to the target subject and the above data, the insights network is used for learning a prediction method under the condition of training external common knowledge data, and the prediction accuracy of the prior neural network is improved after the vocabulary encoder is closed.
In this example, in the process of generating the article, the article generation process is performed only by using the prior neural network, and the posterior neural network is not required to be used, and the relevance of the topic generated by the article is ensured by approaching the posterior neural network parameters in the training to the prior neural network parameters. Wherein the Prior neural network is set as a priority-network, and the posterior neural network is set as a posterior neural network
Recognition-network, whereas prior and posterior neural networks, can be built by artificial neural networks or linear layers. The vocabulary prior neural network and the sentence prior neural network are constructed on the basis of the structure of the prior neural network, and the vocabulary posterior neural network and the sentence posterior neural network are constructed on the basis of the structure of the posterior neural network.
As an example, in step S5022, the server processes the training external common sense data, and converts the training external common sense data into corresponding embedding vectors to obtain training external common sense relationship embedding for subsequent model training.
As an example, in step S5023, the server embeds the training topic, the training external common sense data, and the training external common sense relationship into and trains the external common sense data, inputs the vocabulary posterior neural network to perform model training, and obtains the second vocabulary hidden distribution.
As an example, in step S5024, the server performs a splicing process according to the first hidden distribution of vocabulary and the second hidden distribution of vocabulary, so as to obtain hidden distribution of vocabulary structure with more prominent features.
In the example, training upper data in a training article is input into a vocabulary prior neural network for model training to obtain first vocabulary implicit distribution, then training external common sense data, training external common sense relationship embedding, training subjects and training external common sense data are utilized, the vocabulary prior neural network is input for model training to obtain second vocabulary implicit distribution, and vocabulary structure implicit distribution with more obvious characteristics is obtained after splicing processing and is used for model training.
In one embodiment, step S503, training the vocabulary encoder by using the training external common sense data to obtain the hidden distribution of the vocabulary structure, includes:
s5031: the sentence encoder comprises a sentence prior neural network and a sentence posterior neural network; inputting training subject and training upper data in a training article into a sentence prior neural network for model training to obtain first sentence hidden distribution;
s5032: processing the training upper data to obtain training sentence prediction data;
s5033: inputting a training subject, training upper data and training sentence prediction data into a sentence prior neural network for model training to obtain hidden distribution of a second sentence;
s5034: and obtaining the hidden distribution of sentence structures according to the hidden distribution of the first sentence and the hidden distribution of the second sentence.
As an example, in step S5031, the server inputs the training topic and the training context data in the training article into a sentence prior neural network for model training, and obtains a first sentence hidden distribution. In this example, the sentence encoder includes a sentence prior neural network and a sentence posterior neural network, the prior neural network is used for learning a prediction method according to the target subject and the above data, the posterior neural network is used for learning a prediction method under the condition of structure distribution with the next time, and the prediction accuracy of the prior neural network is improved after the prior neural network is closed.
As an example, in step S5032, the server processes the training text data to obtain training sentence prediction data.
As an example, in step S5033, the server inputs the training topic, the training context data, and the training sentence prediction data into the sentence apriori neural network for model training, and obtains the second sentence implicit distribution.
In this example, the target sentence encoder encodes the training topic, the training upper data, and the training sentence prediction data by setting a bi-directional RNN encoder, and feeds the encoded vectors into the sentence prior neural network and the sentence posterior neural network.
As an example, in step S5034, the server obtains the sentence structure hidden distribution according to the first sentence hidden distribution and the second sentence hidden distribution.
In the example, a sentence prior neural network is input for model training by training the training topic and the training upper data in the training article to obtain a first sentence implicit distribution, then the training topic, the training upper data and the training sentence prediction data are input for model training by the sentence prior neural network to obtain a second sentence implicit distribution, and the sentence structure implicit distribution with more obvious characteristics is obtained after splicing processing and used for model training.
In an embodiment, step S504, training the vocabulary encoder by using the training external common sense data to obtain the hidden distribution of the vocabulary structure includes:
s5041: splicing the vocabulary structure implicit distribution and the sentence structure implicit distribution to obtain training structure implicit distribution;
s5042: adopting a discriminator to classify the subjects of the hidden distribution of the training structure to obtain a subject classification result;
s5043: according to the subject identification result, carrying out countermeasure training on a vocabulary prior neural network and a vocabulary posterior neural network in a vocabulary encoder to obtain a target vocabulary encoder;
s5044: and according to the subject identification result, carrying out countermeasure training on the sentence prior neural network and the sentence posterior neural network in the sentence encoder to obtain the target sentence encoder.
As an example, in step S5041, the server performs a concatenation process on the hidden distribution of the vocabulary structure and the hidden distribution of the sentence structure to obtain the hidden distribution of the training structure.
As an example, in step S5042, the server uses a discriminator to perform topic classification on the implicit distribution of the training structure, and obtains a topic classification result. In this example, the output of the classifier and the training articles and training topic input corresponding to the generated sentence sequence establish loss for the counterstudy.
As an example, in step S5043, according to the topic identification result, the server performs countermeasure training on the vocabulary prior neural network and the vocabulary posterior neural network in the vocabulary encoder by using KL divergence measurement loss, and the two parameters are approximated in the training, so as to obtain the target vocabulary encoder.
The first vocabulary implicit distribution and the second vocabulary implicit distribution which are required to be obtained by the vocabulary encoder based on the CVAE model are obtained by following Gaussian distribution, mean parameters and variance parameters are more critical in the Gaussian distribution, and the loss is measured through the KL divergence, so that the mean parameters and the variance parameters of the vocabulary prior neural network and the vocabulary posterior neural network are adjusted, and the first vocabulary implicit distribution and the second vocabulary implicit distribution are close to each other.
As an example, in step S5044, according to the topic identification result, the server measures loss through KL divergence for the sentence prior neural network and the sentence posterior neural network in the sentence encoder, and performs countermeasure training by approximating these two parameters, thereby obtaining the target sentence encoder. In this example, since the training sentence prediction data is not generated in the process of generating the article, only the sentence prior neural network is used for generating the article, and the topic relevance generated by the article is ensured under the condition that only the sentence prior neural network is used by approximating two parameters, namely the sentence prior neural network and the sentence posterior neural network.
The first sentence implicit distribution and the second sentence implicit distribution which are required to be obtained by the sentence encoder based on the CVAE model are obtained by following Gaussian distribution, mean parameters and variance parameters are more critical in the Gaussian distribution, and loss is measured through KL divergence to adjust the mean parameters and the variance parameters of the sentence prior neural network and the sentence posterior neural network so as to realize the approach of the first sentence implicit distribution and the second sentence implicit distribution.
In the present example, the conventional sequence generation model is replaced by using CVAE (dual conditional variable automatic encoder). Meanwhile, in order to better capture the internal structural relationship, an internal structural representation is constructed by using the appearance condition of a topic related entity in a sentence, and is dynamically updated according to a generated new sequence, so that the recall process of external knowledge is guided. By utilizing the internal structure diagram, the CVAE generation process can be better controlled, and the diversity of generated texts is enhanced on the premise of controllable semantics.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
In an embodiment, an article generation apparatus is provided, and the article generation apparatus corresponds to the article generation methods in the above embodiments one to one. As shown in fig. 6, the article generating apparatus includes a target topic acquisition module 801, a target structure distribution acquisition module 802, a target sentence sequence acquisition module 803, a sentence data acquisition module 804, and a target article acquisition module 805. The detailed description of each functional module is as follows:
a target theme acquisition module 801 for acquiring a target theme;
a target structure distribution obtaining module 802, which uses a target vocabulary encoder to process a target theme, target external data corresponding to the target theme, and target context data corresponding to the current time, and obtains target structure distribution corresponding to the current time;
a target sentence sequence obtaining module 803, which uses a target sentence encoder to process the target theme, the target structure distribution corresponding to the current time and the target upper data corresponding to the current time, so as to obtain a target sentence sequence corresponding to the current time;
a sentence data obtaining module 804, configured to process the target sentence sequence corresponding to the current time to obtain sentence data corresponding to the current time;
the target article obtaining module 805 obtains a target article based on the sentence data corresponding to the current time and the target text data corresponding to the current time.
In one embodiment, the target structure distribution obtaining module 802 includes:
the upper structure distribution acquisition unit acquires the upper structure distribution corresponding to the current moment according to the target upper data corresponding to the current moment;
the target external common sense data acquisition unit is used for acquiring target external common sense data according to a target theme and target upper text data corresponding to the current moment;
and the target structure distribution acquisition unit is used for acquiring the target structure distribution corresponding to the current moment based on the external common sense data and the above structure distribution corresponding to the current moment.
In one embodiment, the target article obtaining module 805 includes:
an original article acquisition unit which acquires an original article based on sentence data corresponding to the current time and target upper text data corresponding to the current time;
the format condition judging unit is used for judging whether the original article meets the format condition corresponding to the target theme or not;
the target article acquisition unit is used for determining the original article as the target article if the original article meets the format condition;
and the target context data acquisition unit is used for determining the original article as the target context data of the next moment if the original article does not meet the format condition.
In another embodiment, the article generation method further comprises:
the training subject acquisition module is used for acquiring training external common sense data and training articles related to a training subject;
the vocabulary structure hidden distribution acquisition module is used for training the vocabulary encoder by utilizing training external common knowledge data to acquire vocabulary structure hidden distribution;
the sentence structure hidden distribution acquisition module is used for training a sentence encoder by utilizing a training article to acquire sentence structure hidden distribution;
and the target vocabulary encoder and target sentence encoder acquisition module is used for performing countermeasure training according to the vocabulary structure hidden distribution and the sentence structure hidden distribution to acquire a target vocabulary encoder and a target sentence encoder.
In one embodiment, the vocabulary structure hidden distribution acquiring module includes:
the device comprises a first vocabulary hidden distribution acquisition unit, a first vocabulary encoder and a second vocabulary hidden distribution acquisition unit, wherein the vocabulary encoder comprises a vocabulary prior neural network and a vocabulary posterior neural network; inputting training upper text data in a training article into a vocabulary prior neural network for model training to obtain a first vocabulary implicit distribution;
the training external common sense relation embedding acquisition unit is used for processing training external common sense data and acquiring training external common sense relation embedding;
the second vocabulary hidden distribution acquisition unit is used for embedding and training the training theme, training external common sense data and training external common sense relation, inputting a vocabulary posterior neural network for model training and acquiring second vocabulary hidden distribution;
and the vocabulary structure hidden distribution acquisition unit acquires the vocabulary structure hidden distribution according to the first vocabulary hidden distribution and the second vocabulary hidden distribution.
In one embodiment, the sentence structure implicit distribution obtaining module includes:
a first sentence hidden distribution acquisition unit, wherein the sentence encoder comprises a sentence prior neural network and a sentence posterior neural network; inputting training subject and training upper data in a training article into a sentence prior neural network for model training to obtain first sentence hidden distribution;
the training sentence prediction data acquisition unit is used for processing the training upper data to acquire training sentence prediction data;
the second sentence hidden distribution acquisition unit inputs the training subject, the training upper data and the training sentence prediction data into a sentence prior neural network for model training to acquire second sentence hidden distribution;
and the sentence structure hidden distribution acquisition unit acquires the sentence structure hidden distribution according to the first sentence hidden distribution and the second sentence hidden distribution.
In one embodiment, the target vocabulary encoder and target sentence encoder acquisition module comprises:
the training structure hidden distribution acquisition unit is used for splicing the vocabulary structure hidden distribution and the sentence structure hidden distribution to acquire training structure hidden distribution;
the theme classification result acquisition unit is used for carrying out theme classification on the hidden distribution of the training structure by adopting a discriminator to acquire a theme classification result;
the target vocabulary encoder obtaining unit is used for carrying out countermeasure training on a vocabulary prior neural network and a vocabulary posterior neural network in the vocabulary encoder according to the theme identification result to obtain a target vocabulary encoder;
and the target sentence encoder acquisition unit is used for carrying out countermeasure training on the sentence prior neural network and the sentence posterior neural network in the sentence encoder according to the theme identification result to acquire the target sentence encoder.
For the specific limitations of the article generation apparatus, reference may be made to the above limitations of the article generation method, which are not described herein again. The modules in the article generation apparatus can be implemented in whole or in part by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent of a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for executing data adopted or generated in the article generation method process. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an article generation method.
In an embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the processor executes the computer program, the article generation method in the foregoing embodiments is implemented, for example, as shown in S201-S205 in fig. 2, or as shown in fig. 3 to fig. 5, which is not repeated here to avoid repetition. Alternatively, when the processor executes the computer program, the functions of the modules/units in the embodiment of the article generating apparatus are implemented, for example, the functions of the target topic obtaining module 801, the target structure distribution obtaining module 802, the target sentence sequence obtaining module 803, the sentence data obtaining module 804 and the target article obtaining module 805 shown in fig. 6 are not described herein again to avoid repetition.
In an embodiment, a computer-readable storage medium is provided, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the article generation method in the foregoing embodiments is implemented, for example, S201 to S205 shown in fig. 2, or shown in fig. 3 to fig. 5, which is not described herein again to avoid repetition. Alternatively, the computer program, when being executed by the processor, implements the functions of the modules/units in the embodiment of the article generating apparatus, such as the functions of the target topic obtaining module 801, the target structure distribution obtaining module 802, the target sentence sequence obtaining module 803, the sentence data obtaining module 804 and the target article obtaining module 805 shown in fig. 6, which are not repeated herein for avoiding repetition.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, databases or other media used in the embodiments provided herein may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambuS (RambuS) direct RAM (RDRAM), direct RambuS Dynamic RAM (DRDRAM), and RambuS Dynamic RAM (RDRAM).
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules, so as to perform all or part of the functions described above.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. An article generation method, comprising:
acquiring a target theme;
processing the target theme, the target external data corresponding to the target theme and the target upper data corresponding to the current moment by adopting a target vocabulary encoder to obtain target structure distribution corresponding to the current moment; the target upper data corresponding to the current moment is the summary of all sentence data before the current moment;
processing the target theme, the target structure distribution corresponding to the current moment and the target upper text data corresponding to the current moment by adopting a target sentence encoder to obtain a target sentence sequence corresponding to the current moment;
processing the target sentence sequence corresponding to the current moment to acquire sentence data corresponding to the current moment;
and acquiring a target article based on the sentence data corresponding to the current moment and the target upper text data corresponding to the current moment.
2. The article generation method of claim 1, wherein processing the target topic, target external data corresponding to the target topic, and target context data corresponding to the current time to obtain target structure distribution corresponding to the current time comprises:
acquiring the upper structure distribution corresponding to the current moment according to the target upper data corresponding to the current moment;
acquiring external common sense data of the target according to the target theme and the target upper data corresponding to the current moment;
and acquiring target structure distribution corresponding to the current moment based on the external common sense data and the above structure distribution corresponding to the current moment.
3. The article generation method of claim 1, wherein the obtaining of the target article based on the sentence data corresponding to the current time and the target text data corresponding to the current time comprises:
acquiring an original article based on sentence data corresponding to the current moment and target upper text data corresponding to the current moment;
judging whether the original article meets the format condition corresponding to the target theme or not;
if the original article meets the format condition, determining the original article as a target article;
and if the original article does not meet the format condition, determining the original article as the target text data at the next moment.
4. The article generation method of claim 1, wherein prior to the obtaining the target topic, the article generation method further comprises:
acquiring training external common sense data and training articles related to a training subject;
training the vocabulary encoder by using training external common knowledge data to obtain vocabulary structure hidden distribution;
training a sentence encoder by using the training article to acquire hidden distribution of sentence structures;
and performing countermeasure training according to the hidden distribution of the vocabulary structure and the hidden distribution of the sentence structure to obtain a target vocabulary encoder and a target sentence encoder.
5. The article generation method of claim 4, wherein training the lexical encoder using training external common sense data to obtain lexical structure hidden distributions comprises:
the vocabulary encoder comprises a vocabulary prior neural network and a vocabulary posterior neural network;
inputting training upper data in the training article into a vocabulary prior neural network for model training to obtain a first vocabulary hidden distribution;
processing training external common sense data to obtain training external common sense relationship embedding;
inputting the training theme, the training external common sense data, the training external common sense relationship embedding and the training external common sense data into a vocabulary posterior neural network for model training to obtain a second vocabulary hidden distribution;
and acquiring hidden distribution of a vocabulary structure according to the hidden distribution of the first vocabulary and the hidden distribution of the second vocabulary.
6. The article generation method of claim 4, wherein said training a sentence encoder using said training article to obtain a hidden distribution of sentence structure comprises:
wherein the sentence encoder comprises a sentence prior neural network and a sentence posterior neural network;
inputting sentence prior neural networks into the training subjects and the training upper text data in the training articles for model training to obtain first sentence implicit distribution;
processing the training upper text data to obtain training sentence prediction data;
inputting the training subject, the training upper data and the training sentence prediction data into a sentence prior neural network for model training to obtain a second sentence hidden distribution;
and acquiring sentence structure hidden distribution according to the first sentence hidden distribution and the second sentence hidden distribution.
7. The article generating method of claim 4, wherein said performing countermeasure training based on said hidden distribution of vocabulary structure and said hidden distribution of sentence structure to obtain a target vocabulary encoder and a target sentence encoder comprises:
splicing the vocabulary structure implicit distribution and the sentence structure implicit distribution to obtain training structure implicit distribution;
adopting a discriminator to classify the subjects of the hidden distribution of the training structure to obtain a subject classification result;
according to the subject identification result, carrying out countermeasure training on a vocabulary prior neural network and a vocabulary posterior neural network in the vocabulary encoder to obtain a target vocabulary encoder;
and according to the subject identification result, carrying out antagonistic training on the sentence prior neural network and the sentence posterior neural network in the sentence encoder to obtain a target sentence encoder.
8. An article generation apparatus, comprising:
the target theme acquisition module is used for acquiring a target theme;
a target structure distribution acquisition module which adopts a target vocabulary encoder to process the target theme, target external data corresponding to the target theme and target upper data corresponding to the current moment so as to acquire target structure distribution corresponding to the current moment;
a target sentence sequence acquisition module which adopts a target sentence encoder to process the target theme, the target structure distribution corresponding to the current moment and the target upper data corresponding to the current moment so as to acquire a target sentence sequence corresponding to the current moment;
the sentence data acquisition module is used for processing the target sentence sequence corresponding to the current moment to acquire sentence data corresponding to the current moment;
and the target article acquisition module is used for acquiring a target article based on the sentence data corresponding to the current moment and the target upper text data corresponding to the current moment.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the article generation method of any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, implements the article generation method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210816045.0A CN115204118B (en) | 2022-07-12 | 2022-07-12 | Article generation method, apparatus, computer device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210816045.0A CN115204118B (en) | 2022-07-12 | 2022-07-12 | Article generation method, apparatus, computer device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115204118A true CN115204118A (en) | 2022-10-18 |
| CN115204118B CN115204118B (en) | 2023-06-27 |
Family
ID=83579803
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210816045.0A Active CN115204118B (en) | 2022-07-12 | 2022-07-12 | Article generation method, apparatus, computer device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115204118B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287489A (en) * | 2019-06-24 | 2019-09-27 | 北京大米科技有限公司 | Document creation method, device, storage medium and electronic equipment |
| KR102078505B1 (en) * | 2019-05-28 | 2020-02-17 | 주식회사 에이플에이디 | System and method for generating article |
| CN111046645A (en) * | 2019-12-11 | 2020-04-21 | 浙江大搜车软件技术有限公司 | Method and device for generating article, computer equipment and storage medium |
| CN111428472A (en) * | 2020-03-13 | 2020-07-17 | 浙江华坤道威数据科技有限公司 | Article automatic generation system and method based on natural language processing and image algorithm |
| US20210064823A1 (en) * | 2018-05-15 | 2021-03-04 | Beijing Sankuai Online Technology Co., Ltd | Article generation |
| US20210117431A1 (en) * | 2019-10-16 | 2021-04-22 | International Business Machines Corporation | Structured article generation |
-
2022
- 2022-07-12 CN CN202210816045.0A patent/CN115204118B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210064823A1 (en) * | 2018-05-15 | 2021-03-04 | Beijing Sankuai Online Technology Co., Ltd | Article generation |
| KR102078505B1 (en) * | 2019-05-28 | 2020-02-17 | 주식회사 에이플에이디 | System and method for generating article |
| CN110287489A (en) * | 2019-06-24 | 2019-09-27 | 北京大米科技有限公司 | Document creation method, device, storage medium and electronic equipment |
| US20210117431A1 (en) * | 2019-10-16 | 2021-04-22 | International Business Machines Corporation | Structured article generation |
| CN111046645A (en) * | 2019-12-11 | 2020-04-21 | 浙江大搜车软件技术有限公司 | Method and device for generating article, computer equipment and storage medium |
| CN111428472A (en) * | 2020-03-13 | 2020-07-17 | 浙江华坤道威数据科技有限公司 | Article automatic generation system and method based on natural language processing and image algorithm |
Non-Patent Citations (1)
| Title |
|---|
| 张青;韩立新;勾智楠;: "基于词向量和变分自动编码器的短文本主题模型", 河北工业科技, no. 06 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115204118B (en) | 2023-06-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022095682A1 (en) | Text classification model training method, text classification method and apparatus, device, storage medium, and computer program product | |
| CN111930940B (en) | Text emotion classification method and device, electronic equipment and storage medium | |
| JP6207733B2 (en) | System and method for creating and implementing an artificial intelligent agent or system | |
| CN110276259A (en) | Lip reading recognition methods, device, computer equipment and storage medium | |
| CN114245203B (en) | Video editing method, device, equipment and medium based on script | |
| CN112528637B (en) | Text processing model training method, device, computer equipment and storage medium | |
| CN110162766B (en) | Word vector updating method and device | |
| CN112380837B (en) | Similar sentence matching method, device, equipment and medium based on translation model | |
| CN113297366B (en) | Emotion recognition model training method, device, equipment and medium for multi-round dialogue | |
| CN112149651B (en) | Facial expression recognition method, device and equipment based on deep learning | |
| WO2023137911A1 (en) | Intention classification method and apparatus based on small-sample corpus, and computer device | |
| CN114120978A (en) | Emotion recognition model training and voice interaction method, device, equipment and medium | |
| CN113505198A (en) | Keyword-driven generating type dialogue reply method and device and electronic equipment | |
| CN112652295A (en) | Language model training method, device, equipment and medium, and video subtitle checking method, device and medium | |
| CN116975288A (en) | Text processing method and text processing model training method | |
| CN113449081A (en) | Text feature extraction method and device, computer equipment and storage medium | |
| US11036996B2 (en) | Method and apparatus for determining (raw) video materials for news | |
| CN115204118B (en) | Article generation method, apparatus, computer device and storage medium | |
| CN116415597A (en) | Speech translation and simultaneous interpretation method | |
| CN114743539A (en) | Speech synthesis method, apparatus, device and storage medium | |
| CN114969544A (en) | Hot data-based recommended content generation method, device, equipment and medium | |
| KR102120936B1 (en) | System for providing customized character doll including smart phone | |
| CN116913278B (en) | Voice processing method, device, equipment and storage medium | |
| CN113011555B (en) | Data processing method, device, equipment and storage medium | |
| CN116431773A (en) | Dialogue flow extraction method and device, computer readable storage medium and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |