CN110245224B - Dialog generation method and device - Google Patents
Dialog generation method and device Download PDFInfo
- Publication number
- CN110245224B CN110245224B CN201910537494.XA CN201910537494A CN110245224B CN 110245224 B CN110245224 B CN 110245224B CN 201910537494 A CN201910537494 A CN 201910537494A CN 110245224 B CN110245224 B CN 110245224B
- Authority
- CN
- China
- Prior art keywords
- sample
- identifier
- parameter value
- dialog
- scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
Abstract
The embodiment of the disclosure provides a dialog generation method and a device, which are applied to a dialog generation system, wherein the method comprises the following steps: obtaining dialogue information, wherein the dialogue information comprises dialogue sentences and dialogue scene marks; acquiring a target parameter value corresponding to the conversation scene identifier, wherein the target parameter value is a parameter value of a preset model, and the preset model is applicable to a plurality of conversation scenes; processing the dialogue sentences by adopting a preset model of the target parameter values, and determining reply sentences corresponding to the dialogue sentences; and returning a reply sentence corresponding to the conversation sentence. For improving the versatility of the dialog generating system and the dialog generating method.
Description
Technical Field
The embodiment of the disclosure relates to the field of machine conversation, and in particular relates to a conversation generation method and device.
Background
The dialogue generation system is a system that can generate a response sentence from a sentence input by a user. For example, a robot provided with a dialog generation system may generate a response sentence from a sentence input by a user and reply the response sentence to the user. Currently, dialog generation systems are classified into the following 5 types according to application scenarios: a single-round dialog generation system, a multi-round dialog generation system, a dialog generation system based on tag control, a dialog generation system based on personalized background information, and a knowledge-based dialog generation system.
In practical applications, a robot provided with a dialog generation system can reply a response sentence corresponding to the dialog generation system to a user only by a dialog generation method corresponding to the dialog generation system based on a sentence input by the user. For example, a robot provided with a single-round dialog generation system may reply a response sentence corresponding to the single-round dialog generation system to a user by using a dialog generation method corresponding to the single-round dialog generation system, based on a sentence input by the user. However, the robot provided with the one-round dialog generation system cannot reply a response sentence corresponding to the dialog generation system based on the tag control to the user by the dialog generation method corresponding to the one-round dialog generation system, and thus the versatility of the dialog generation system and the dialog generation method is low.
Disclosure of Invention
The embodiment of the disclosure provides a dialog generation method and a dialog generation device, which are used for improving the universality of a dialog generation system and the dialog generation method.
In a first aspect, an embodiment of the present disclosure provides a dialog generation method, which is applied to a dialog generation system, and includes:
obtaining dialogue information, wherein the dialogue information comprises dialogue sentences and dialogue scene marks;
acquiring a target parameter value corresponding to the conversation scene identifier, wherein the target parameter value is a parameter value of a preset model, and the preset model is applicable to a plurality of conversation scenes;
processing the dialogue sentences by adopting a preset model of the target parameter values, and determining reply sentences corresponding to the dialogue sentences;
and returning a reply sentence corresponding to the conversation sentence.
In an optional implementation manner, before the obtaining of the target parameter value matching the dialog scenario identifier, the method further includes:
pre-training the text corpus through a generative pre-training GPT model to determine an intermediate parameter value, wherein the intermediate parameter value is a parameter value of the GPT model;
obtaining a sample dialogue in a sample dialogue scene, and determining a corresponding relation between a sample parameter value and an identifier of the sample dialogue scene according to the sample dialogue in the sample dialogue scene and a GPT (general packet transport) model with the intermediate parameter value, wherein the sample dialogue comprises a sample dialogue statement, a sample reply statement and a sample dialogue scene identifier;
and updating the GPT model according to the sample parameter value to obtain the preset model.
In another optional implementation, the pre-training the text corpus by using the generative pre-trained GPT model to determine an intermediate parameter value includes:
performing sentence division processing on the text corpus to determine M groups of sentences, wherein each group of sentences comprises at least one sentence;
pre-training the M sets of sentences for N times simultaneously through the GPT model to obtain N sets of loss values of the GPT model, wherein each set of loss values comprises M loss values, and each loss value corresponds to one set of sentences and candidate intermediate parameter values;
obtaining an average loss value corresponding to each group of loss values according to the M loss values included in each group of loss values;
and determining the intermediate parameter value according to the average loss value corresponding to each group of loss values and the candidate intermediate parameter value, wherein M and N are integers greater than or equal to 1.
In another optional implementation manner, the determining the intermediate parameter value according to the average loss value corresponding to each group of loss values and the candidate intermediate parameter value includes:
determining a minimum average loss value in the average loss values corresponding to each group of loss values;
acquiring a target loss value group where the minimum average loss value is located;
determining a minimum loss value among the set of target loss values;
and determining the candidate intermediate parameter value corresponding to the minimum loss value as the intermediate parameter value.
In another optional implementation manner, the obtaining a sample dialogue in a sample dialogue scene, and determining, according to the sample dialogue in the sample dialogue scene and a GPT model with the intermediate parameter value, a correspondence between a sample parameter value and an identifier of the sample dialogue scene includes:
according to a preset splicing rule, carrying out splicing processing on the identifier of the sample conversation scene and the sample conversation in the sample conversation scene to obtain P groups of splicing conversations;
performing Q training treatments on the P groups of splicing dialogues simultaneously through the GPT model with the intermediate parameter values to obtain Q groups of loss values of the GPT model with the intermediate parameter values, wherein each group of loss values comprises P loss values, the P loss values respectively correspond to one group of splicing dialogues and candidate sample parameter values, and each candidate sample parameter value has an identifier;
obtaining average loss values corresponding to the Q groups of loss values according to the P loss values included in the Q groups of loss values;
and determining the sample parameter value according to the average loss value corresponding to each of the Q groups of loss values and each candidate sample parameter value, and determining the corresponding relation between the identifier of the sample parameter value and the identifier of the sample dialogue scene according to the identifier of the sample dialogue scene and the identifier of each candidate sample parameter value, wherein P and Q are integers greater than or equal to 1.
In another optional implementation manner, the determining, according to the average loss value corresponding to each of the Q groups of loss values and the each candidate sample parameter value, the correspondence between the identifier of the sample parameter value and the identifier of the sample dialog scene according to the identifier of the sample dialog scene and the identifier of each candidate sample parameter value includes:
determining a minimum average loss value in the average loss values corresponding to the Q groups of loss values;
acquiring a target loss value group where the minimum average loss value is located;
determining a minimum loss value among the set of target loss values;
determining the candidate sample parameter value corresponding to the minimum loss value as the sample parameter value, determining the identifier of the candidate sample parameter value corresponding to the minimum loss value as the identifier of the sample parameter value,
and storing the identifier of the sample parameter and the identifier of the sample conversation scene to obtain the corresponding relation between the identifier of the sample parameter value and the identifier of the sample conversation scene.
In another optional implementation, the obtaining of the session information includes:
after the dialog statement is obtained, obtaining a current dialog scene identifier of the system, wherein the current dialog scene identifier is a preset dialog scene identifier;
and acquiring the dialogue information according to the dialogue statement and the current dialogue scene identifier.
In another optional implementation manner, the preset dialog scene identifier is any one of a single-round dialog scene identifier, a multi-round dialog scene identifier, a dialog scene identifier based on tag control, a dialog scene identifier based on personalized background information, and a dialog scene identifier based on knowledge.
In a second aspect, an embodiment of the present disclosure provides a dialog generating apparatus, which is applied to a dialog generating system, and the apparatus includes: a first obtaining module, a second obtaining module, a first determining module and a feedback module, wherein,
the first acquisition module is used for acquiring dialogue information, and the dialogue information comprises dialogue sentences and dialogue scene marks;
the second obtaining module is configured to obtain a target parameter value corresponding to the dialog scene identifier, where the target parameter value is a parameter value of a preset model, and the preset model is applicable to multiple dialog scenes;
the first determining module is used for processing the dialogue sentences by adopting a preset model of the target parameter values and determining reply sentences corresponding to the dialogue sentences;
and the feedback module is used for returning a reply sentence corresponding to the conversation sentence.
In an optional embodiment, the method further comprises: a second determination module to:
before the target parameter value matched with the conversation scene identification is obtained, pre-training processing is carried out on the text corpus through a generative pre-training GPT model, and an intermediate parameter value is determined, wherein the intermediate parameter value is the parameter value of the GPT model;
obtaining a sample dialogue in a sample dialogue scene, and determining a corresponding relation between a sample parameter value and an identifier of the sample dialogue scene according to the sample dialogue in the sample dialogue scene and a GPT (general packet transport) model with the intermediate parameter value, wherein the sample dialogue comprises a sample dialogue statement, a sample reply statement and a sample dialogue scene identifier;
and updating the GPT model according to the sample parameter value to obtain the preset model.
In another optional implementation manner, the second determining module is specifically configured to:
performing sentence division processing on the text corpus to determine M groups of sentences, wherein each group of sentences comprises at least one sentence;
pre-training the M sets of sentences for N times simultaneously through the GPT model to obtain N sets of loss values of the GPT model, wherein each set of loss values comprises M loss values, and each loss value corresponds to one set of sentences and candidate intermediate parameter values;
obtaining an average loss value corresponding to each group of loss values according to the M loss values included in each group of loss values;
and determining the intermediate parameter value according to the average loss value corresponding to each group of loss values and the candidate intermediate parameter value, wherein M and N are integers greater than or equal to 1.
In another optional implementation manner, the second determining module is specifically configured to:
determining a minimum average loss value in the average loss values corresponding to each group of loss values;
acquiring a target loss value group where the minimum average loss value is located;
determining a minimum loss value among the set of target loss values;
and determining the candidate intermediate parameter value corresponding to the minimum loss value as the intermediate parameter value.
In another optional embodiment, the second determining module is further configured to:
according to a preset splicing rule, carrying out splicing processing on the identifier of the sample conversation scene and the sample conversation in the sample conversation scene to obtain P groups of splicing conversations;
performing Q training treatments on the P groups of splicing dialogues simultaneously through the GPT model with the intermediate parameter values to obtain Q groups of loss values of the GPT model with the intermediate parameter values, wherein each group of loss values comprises P loss values, the P loss values respectively correspond to one group of splicing dialogues and candidate sample parameter values, and each candidate sample parameter value has an identifier;
obtaining average loss values corresponding to the Q groups of loss values according to the P loss values included in the Q groups of loss values;
and determining the sample parameter value according to the average loss value corresponding to each of the Q groups of loss values and each candidate sample parameter value, and determining the corresponding relation between the identifier of the sample parameter value and the identifier of the sample dialogue scene according to the identifier of the sample dialogue scene and the identifier of each candidate sample parameter value, wherein P and Q are integers greater than or equal to 1.
In another optional embodiment, the second determining module is further configured to:
determining a minimum average loss value in the average loss values corresponding to the Q groups of loss values;
acquiring a target loss value group where the minimum average loss value is located;
determining a minimum loss value among the set of target loss values;
determining the candidate sample parameter value corresponding to the minimum loss value as the sample parameter value, determining the identifier of the candidate sample parameter value corresponding to the minimum loss value as the identifier of the sample parameter value,
and storing the identifier of the sample parameter and the identifier of the sample conversation scene to obtain the corresponding relation between the identifier of the sample parameter value and the identifier of the sample conversation scene.
In another optional implementation manner, the first obtaining module is specifically configured to:
after the dialog statement is obtained, obtaining a current dialog scene identifier of the system, wherein the current dialog scene identifier is a preset dialog scene identifier;
and acquiring the dialogue information according to the dialogue statement and the current dialogue scene identifier.
In another optional implementation manner, the preset dialog scene identifier is any one of a single-round dialog scene identifier, a multi-round dialog scene identifier, a dialog scene identifier based on tag control, a dialog scene identifier based on personalized background information, and a dialog scene identifier based on knowledge.
In a third aspect, an embodiment of the present disclosure provides a dialog generating device, including:
a memory for storing a program;
a processor for executing the program stored in the memory, the processor being configured to perform the dialog generation method according to any of the above first aspects when the program is executed.
In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, which includes instructions that, when executed on a computer, cause the computer to perform the dialog generation method according to any one of the above first aspects.
The dialog generation method and device provided by the embodiment of the disclosure comprise the following steps: and acquiring dialogue information which comprises dialogue sentences and dialogue scene marks. And acquiring a target parameter value corresponding to the conversation scene identification, wherein the target parameter value is a parameter value of a preset model, and the preset model is applicable to a plurality of conversation scenes. And processing the dialogue sentences by adopting a preset model of the target parameter values to determine reply sentences corresponding to the dialogue sentences. And returning a reply sentence corresponding to the dialogue sentence. The method and the device avoid the need of using different dialog generation systems and different dialog generation methods in different dialog scenes in the prior art, and improve the universality of the dialog generation systems and the dialog generation methods.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to the drawings without inventive exercise.
Fig. 1 is a schematic view of an application scenario of a dialog generation method provided in an embodiment of the present disclosure;
fig. 2 is a first flowchart of a dialog generation method according to an embodiment of the present disclosure;
fig. 3 is a schematic flowchart of a dialog generation method according to an embodiment of the present disclosure;
fig. 4 is a first schematic structural diagram of a dialog generating device according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a dialog generating device according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram of a hardware structure of a dialog generating device according to an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it should be understood that the described embodiments are some embodiments of the present disclosure, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 is a schematic view of an application scenario of a dialog generation method according to an embodiment of the present disclosure. As shown in fig. 1, includes: electronic device 11, dialog generating system 12 operating in electronic device 11.
Alternatively, the electronic device 11 may be a terminal device or an electronic device such as a robot. For example, the terminal device may be a smart phone, a laptop computer, a desktop computer, and the like. For example, the robot may be a housekeeping robot, an emotional robot, and the like.
It should be noted that the dialog generation system 12 is applicable to a variety of dialog scenarios. For example, a single-turn dialog scenario, a multi-turn dialog scenario, a tag control-based dialog scenario, a personalized context information-based dialog scenario, and a knowledge-based dialog scenario.
During the operation of the dialog generating system 12 on the electronic device 11, a user may preset a dialog scene of the dialog generating system 12 through the electronic device 11 (for example, the user clicks a multi-turn dialog scene button in a dialog scene list on the electronic device 11, and may realize a dialog scene of the dialog generating system 12 that is preset), where the dialog scene has a preset dialog scene identifier. After determining the dialog scenario, the electronic device 11 may determine a target parameter value of a preset model in the dialog generating system 12 according to the dialog scenario identifier, and assign the target parameter value to the preset model in the dialog generating system 12. After the user inputs a dialogue sentence into the electronic device, the dialogue generating system 12 may generate a response sentence corresponding to the dialogue sentence in a preset dialogue scene. Alternatively, the dialog sentence may be a text sentence or a voice sentence, wherein the sentence type may be input into the electronic device 11 by the user. For example, the electronic device 11 shown in the embodiment of the present disclosure is a robot, and the dialogue sentences are speech sentences input by the user into the electronic device 11.
In the above process, the user may preset the dialog scenario of the dialog generating system, so that the dialog generating system generates the response sentence corresponding to the dialog sentence in the dialog scenario after receiving the dialog sentence input by the user, thereby avoiding the problem that different dialog generating systems are required to be used in different dialog scenarios in the prior art, and improving the universality of the dialog generating system.
A dialog generation method that can be applied to the dialog generation system in the embodiments of the present disclosure will be described in detail below with specific embodiments.
It should be noted that the following embodiments may be combined with each other, and the description of the same or similar contents in different embodiments is not repeated.
Fig. 2 is a first flowchart of a dialog generation method according to an embodiment of the present disclosure. As shown in fig. 2, the dialog generation method includes:
s201: and acquiring dialogue information which comprises dialogue sentences and dialogue scene marks.
Alternatively, the execution subject of the embodiment of the present disclosure may be a dialog generation system, and may be a dialog generation apparatus provided in the dialog generation system, and the dialog generation apparatus may be implemented by a combination of software and/or hardware.
The dialog sentences are dialog sentences input by a user into the dialog generating system, the dialog scene identifiers are preset identifiers, the dialog scene identifiers correspond to the dialog scenes, and after the user selects a certain dialog scene, the dialog generating system can determine the dialog scene identifiers corresponding to the dialog scene according to the dialog scene selected by the user. As shown in fig. 1, the user selects a multi-turn dialog scenario, and the dialog generation system may determine a multi-turn dialog scenario identification from the multi-turn dialog scenario.
For example, when the dialog scene is a single-turn dialog scene, the corresponding dialog scene identifier may be < a >.
For example, when the dialog scene is a multi-turn dialog scene, the corresponding dialog scene identifier may be < b >.
It should be noted that the dialog information may also include a start symbol and a sentence break symbol.
Alternatively, the start character and the sentence separator may be represented by a combination of letters and symbols.
For example, < s > may be used to represent the initiator and < p > may be used to represent the statement spacer. Specifically, the type and combination of the letters and symbols are not limited in the embodiments of the present disclosure.
In the embodiment of the present disclosure, the dialog sentences, the dialog scene identifiers, the start symbols, and the sentence spacers may be spliced according to a preset splicing method to obtain the dialog information.
Specifically, in the embodiment of the present disclosure, a method for obtaining dialog information by splicing dialog statements, dialog scene identifiers, start symbols, and statement interval symbols in different dialog scenes is provided, as shown in table 1.
TABLE 1 dialog messages in different dialog scenarios
S202: and acquiring a target parameter value corresponding to the conversation scene identification, wherein the target parameter value is a parameter value of a preset model, and the preset model is applicable to a plurality of conversation scenes.
In this embodiment, a plurality of sets of sample parameter values are stored in the dialog generation system, each set of sample parameter value has an identifier, and the identifier of one set of sample parameter value has a corresponding relationship with one dialog scene identifier.
Alternatively, the correspondence may be that the identity of a set of sample parameter values is the same as a dialog scenario identity, or that the identity of a set of sample parameter values is not the same as a dialog scenario identity.
For example, the identity of a set of sample parameter values and a dialog scene identity are the same, which may be the same as < a >, < b >, < c >, < d >, and < e >.
For example, when the identifiers of a group of sample parameter values and a dialog scene identifier are not the same, if the identifier of the sample parameter value is < a >, the dialog scene identifier may be < a1 >.
In practical application, according to the dialog scene identifier and the corresponding relationship, the target parameter value corresponding to the dialog scene identifier can be uniquely determined in a plurality of groups of sample parameter values.
S203: and processing the dialogue sentences by adopting a preset model of the target parameter values to determine reply sentences corresponding to the dialogue sentences.
Alternatively, when the dialogue sentence is an encoded sentence, the preset model may perform decoding processing on the encoded sentence, and determine a reply sentence according to the decoded sentence.
In the disclosed embodiment, it is exemplarily given that a reply sentence is determined according to a dialog sentence input by a user in different dialog scenarios.
For example, the dialog scenario is a single-turn dialog scenario, and the dialog sentence input by the user is "what name you call? "while, the reply sentence might be" I don't tell you! ".
For example, when the dialog scene is a multi-turn dialog scene, the dialog sentence input by the user is "what name you call? When the answer sentence is likely to be "I call little tweed! "where Xiaoxi is information of the occurrence of a process in the conversation history.
For example, the dialog scenario is based on tag control, and when the dialog sentence input by the user is "what you are doing", the answer sentence may be "haha, i are playing with games, and play is happy".
For example, the dialog scenario is based on personalization, and the dialog sentence input by the user is "what name you are? When, the reply sentence may be "I call bei yei. Wherein, the little north is the attribute information of the robot (namely the name of the robot)
For example, when the dialog scenario is a knowledge dialog scenario, and the dialog sentence input by the user is "you know someone in week", the reply sentence may be "certainly know so that his wife is a kunzhi", where "the wife in week is a kunzhi" is stored in the knowledge base of the robot.
S204: and returning a reply sentence corresponding to the dialogue sentence.
Alternatively, the sentence type of the reply sentence may be a text sentence, or a voice sentence. Note that the sentence type of the dialogue sentence may be the same as or different from the reply sentence type.
For example, when the dialogue sentence is a text sentence, the reply sentence may be a text sentence or a speech sentence.
In the prior art, different dialog generation systems are required to be adopted in different application scenes, and dialog generation methods of different dialog generation systems are different, so that one dialog generation system and one dialog generation method thereof cannot be applied to multiple dialog scenes, and the generality of the dialog generation system and the dialog generation method thereof is poor. Different from the prior art, in the embodiment of the present disclosure, the dialog generation system and the dialog generation method are applicable to a plurality of dialog scenarios, so that the versatility of the dialog generation system and the dialog generation method in the embodiment of the present disclosure is strong.
The dialog generating method provided by the embodiment of the disclosure comprises the following steps: and acquiring dialogue information which comprises dialogue sentences and dialogue scene marks. And acquiring a target parameter value corresponding to the conversation scene identification, wherein the target parameter value is a parameter value of a preset model, and the preset model is applicable to a plurality of conversation scenes. And processing the dialogue sentences by adopting a preset model of the target parameter values to determine reply sentences corresponding to the dialogue sentences. And returning a reply sentence corresponding to the dialogue sentence. In the process, the target parameter value corresponding to the dialog scene identifier is obtained, the dialog statement is processed by adopting the preset model of the target parameter value, and the reply statement corresponding to the dialog statement is determined, so that the situation that different dialog generation systems and dialog generation methods corresponding to the dialog generation systems need to be used in different dialog scenes in the prior art can be avoided, and the universality of the dialog generation systems and the dialog generation methods is improved.
On the basis of any one of the above embodiments, the following describes in detail the dialog generating method provided by the embodiment of the present disclosure with reference to the embodiment of fig. 3, specifically, please refer to fig. 3.
Fig. 3 is a flowchart illustrating a second dialog generation method according to an embodiment of the present disclosure. As shown in fig. 3, the dialog generation method includes:
s301: and pre-training the text corpus through a generative pre-training GPT model to determine an intermediate parameter value, wherein the intermediate parameter value is the parameter value of the GPT model.
In the embodiment of the present disclosure, a Generative Pre-Training model (GPT) is used as an OpenAI transmomer-based GPT model, and the GPT model has the following attributes: the number of layers of the model is 12, the dimensionality of the embedded vector is 768, the discarding rate (dropout) parameter is 0.1, and when the text corpus is subjected to pre-training processing, Chinese characters are taken as vocabulary units.
Alternatively, the text corpora are non-parallel corpora that are crawled from the internet by crawler technology, for example, the non-parallel corpora may be novel corpora. In practical application, because the language quality of the novel corpus is good and the length is moderate, the GPT model can be trained well, so that the GPT model can learn the statistical characteristics among the most basic language element characters to obtain more accurate intermediate parameter values.
Specifically, before the pre-training process, the method further includes initializing a GPT model, and copying parameters of the GPT model to preset parameter values.
It should be noted that the process of determining the intermediate parameter value in S301 is a process of performing rough training on a GPT model with a preset parameter value. In the course of rough training, GPT models trained by a large amount of text corpora have better language characteristics, and reply sentences generated by the dialogue generating system have better grammar logic.
In an alternative embodiment, the execution of S301 includes: SA1 to SA 4.
SA 1: and carrying out sentence division processing on the text corpus, and determining M groups of sentences, wherein each group of sentences comprises at least one sentence, and M is an integer greater than or equal to 1.
Alternatively, the text corpus may be subjected to sentence division processing according to a period, an exclamation point, a question mark, a combination of a quotation mark and a period, a combination of a quotation mark and an exclamation point, a combination of a quotation mark and a question mark, and the like.
SA 2: and pre-training the M sets of sentences for N times simultaneously through the GPT model to obtain N sets of loss values of the GPT model, wherein each set of loss values comprises M loss values, and each loss value corresponds to one set of sentences and candidate intermediate parameter values.
For example, M sets of statements include M1、M2、……、MMThen GPT model pair M1Therefore, the GPT model performs N times of pre-training processing on M groups of sentences simultaneously to obtain N groups of loss values, and each group of loss values comprises M loss values.
SA 3: and obtaining the average loss value corresponding to each group of loss values according to the M loss values included in each group of loss values.
The average loss value corresponding to each of the N sets of loss values is an average of M loss values in the set of loss values.
Optionally, after the N pre-training process is completed, an average loss value corresponding to each group of loss values may be obtained, or after one training process is completed, an average loss value of a group of loss values corresponding to the training process may be obtained.
SA 4: and determining an intermediate parameter value according to the average loss value corresponding to each group of loss values and the candidate intermediate parameter value, wherein N is an integer greater than or equal to 1.
It should be noted that, within a preset time length, if the average loss values corresponding to the N groups of loss values are converged, the pre-training process is stopped, that is, the GPT model does not perform the (N + 1) th pre-training process on the M groups of statements. After stopping the pre-training process, determining an intermediate parameter value according to the average loss value corresponding to each group of loss values and the candidate intermediate parameter value includes:
determining a minimum average loss value in the average loss values corresponding to each group of loss values; acquiring a target loss value group in which the minimum average loss value is positioned; determining a minimum loss value among the set of target loss values; and determining the candidate intermediate parameter value corresponding to the minimum loss value as the intermediate parameter value.
The minimum average loss value is the minimum average loss value in the average loss values corresponding to each group of loss values, and the minimum loss value is the minimum loss value in the target loss value group.
S302: the method comprises the steps of obtaining a sample dialogue in a sample dialogue scene, and determining the corresponding relation between the sample parameter value and the identifier of the sample dialogue scene according to the sample dialogue in the sample dialogue scene and a GPT model with intermediate parameter values, wherein the sample dialogue comprises a sample dialogue statement, a sample reply statement and a sample dialogue scene identifier.
Specifically, the sample dialog scene has an identifier, the identifier of the sample dialog scene is the same as the identifier of the sample dialog scene, and the identifier of the sample dialog scene may be any one of < a >, < b >, < c >, < d >, < e > in table 2 below.
It should be noted that the process of determining the sample parameter values in S302 is to perform fine tuning training on the GPT model with the intermediate parameter values, and the GPT model subjected to fine tuning training is applicable to various dialog scenarios.
In an alternative embodiment, the execution of S302 includes: SB1 to SB 4.
SB 1: and splicing the identifier of the sample conversation scene and the sample conversation in the sample conversation scene according to a preset splicing rule to obtain P groups of splicing conversations, wherein P is an integer greater than or equal to 1.
It should be noted that the preset splicing rule is a splicing sequence of the identifier of the sample dialog scene, the sample dialog sentence, the sample reply sentence, and the identifier of the sample dialog scene. For example, the concatenation sequence is sequentially an identifier of a sample conversation scene (sample conversation scene identifier), a sample conversation sentence, and a sample reply sentence.
Further, when the identifier of the sample dialog scene is the same as the identifier of the sample dialog scene, the embodiment of the present disclosure exemplarily shows a splicing dialog obtained according to a preset splicing rule in different sample dialog scenes, and please refer to fig. 2 on the basis of fig. 1.
TABLE 2 splicing dialogues under different dialog scenarios
SB 2: and simultaneously carrying out Q times of training treatment on the P groups of splicing dialogues through the GPT model with the intermediate parameter values to obtain Q groups of loss values of the GPT model with the intermediate parameter values, wherein each group of loss values comprises P loss values, the P loss values respectively correspond to one group of splicing dialogues and candidate sample parameter values, and each candidate sample parameter value has an identifier.
Specifically, the execution method of SB2 is similar to the execution method of SA2, and the execution process of SB2 is not described here again.
It should be noted that the identification of each candidate sample parameter value is associated with the identification of the sample dialog scenario. For example, if the sample dialog scene is identified as < a >, then the candidate sample parameter values are identified as < a-1>, … … < a-Q >
SB 3: and obtaining the average loss value corresponding to each Q group loss value according to the P loss values included in each Q group loss value.
Specifically, the execution method of SB3 is similar to that of SA3, and is not described here again.
SB 4: and determining the sample parameter value according to the average loss value corresponding to each of Q groups of loss values and each candidate sample parameter value, and determining the corresponding relation between the identifier of the sample parameter value and the identifier of the sample dialogue scene according to the identifier of the sample dialogue scene and the identifier of each candidate sample parameter value, wherein Q is an integer greater than or equal to 1.
Specifically, the execution process of SB4 includes:
determining a minimum average loss value in the average loss values corresponding to the Q groups of loss values;
acquiring a target loss value group where the minimum average loss value is located;
determining a minimum loss value among the set of target loss values;
determining the candidate sample parameter value corresponding to the minimum loss value as the sample parameter value, determining the identifier of the candidate sample parameter value corresponding to the minimum loss value as the identifier of the sample parameter value,
and storing the identifier of the sample parameter and the identifier of the sample conversation scene to obtain the corresponding relation between the identifier of the sample parameter value and the identifier of the sample conversation scene.
For example, the corresponding relationship may be that the identifier of the sample parameter value and the identifier of the sample dialog scene have the same identifier information, for example, the identifier of the sample dialog scene is < a >, and the identifier of the sample parameter value is < a-1>, and then the same identifier information is a.
S303: and updating the GPT model according to the sample parameter value to obtain a preset model.
S304: and after the preset model is obtained and the conversation statement is obtained, obtaining a current conversation scene identifier of the conversation generation system, wherein the current conversation scene identifier is a preset conversation scene identifier.
In the embodiment of the present disclosure, the preset dialog scene identifier is any one of a single-turn dialog scene identifier, a multi-turn dialog scene identifier, a dialog scene identifier based on tag control, a dialog scene identifier based on personalized background information, and a dialog scene identifier based on knowledge.
It should be noted that the current dialog scene identifier may be determined according to a dialog scene selection instruction, where the dialog scene selection instruction carries the current dialog scene identifier.
As shown in fig. 1, the robot is provided with a dialog generation system and a dialog scene list, the dialog scene list includes selection buttons corresponding to a single-turn dialog scene, a multi-turn dialog scene, a dialog scene based on tag control, a dialog scene based on personalization, and a dialog scene based on knowledge, and a user can press any one of the selection buttons, for example, the multi-turn dialog scene button.
S305: and acquiring the conversation information according to the conversation statement and the current conversation scene identifier.
Specifically, the implementation of S305 is similar to the implementation of S201, and here, the method of acquiring dialog information is described only by the following example.
For example, the current dialog scenario is identified as < c >, and the dialog statement is "today the weather is hot! ", it can be determined from table 1 that the obtained information is < s > < c > < p > and the weather is hot today |)! < p >.
S306: and acquiring a target parameter value corresponding to the conversation scene identifier in the conversation information, wherein the target parameter value is a parameter value of a preset model, and the preset model is applicable to a plurality of conversation scenes.
It should be noted that the dialog scene identifier is the same as the current dialog scene identifier. Specifically, the implementation manner of S306 is similar to that of S201, and is not described here again.
S307: and processing the dialogue sentences by adopting a preset model of the target parameter values. And determining a reply sentence corresponding to the conversation sentence.
Specifically, the implementation manner of S307 is similar to that of S202, and is not described herein again.
S308: and returning a reply sentence corresponding to the dialogue sentence.
Specifically, the implementation manner of S308 is similar to that of S203, and is not described here again.
Further, the dialog scene identifier is the same as the identifier of one dialog scene, and the dialog scene identifier corresponds to additional information of one dialog scene, where the additional information includes historical dialog information corresponding to multiple rounds of dialog scenes, tag control information corresponding to dialog scenes based on tag control, personalized information corresponding to dialog scenes based on personalized background information, and knowledge information corresponding to dialog scenes based on knowledge. Before determining the reply sentence corresponding to the dialog sentence, the dialog generating system may determine corresponding additional information according to the identifier of the dialog scene, so that the dialog generating system determines that the reply sentence better conforms to the dialog scene according to the additional information.
In the prior art, when the dialog generating system needs to generate a reply sentence according to the additional information, a model corresponding to the additional information needs to be designed according to prior knowledge, and the model and the dialog generating system are used, so that the dialog generating system can determine the reply sentence according to the additional information, and complexity of a process of determining the reply sentence according to the additional information by the dialog generating system is improved. Different from the prior art, in the embodiment of the present disclosure, the dialog generating system can determine the additional information according to the dialog scene identifier, and determine the reply sentence according to the additional information, thereby reducing the complexity of the process of determining the reply sentence according to the additional information by the dialog generating system.
The dialog generating method provided by the embodiment of the disclosure comprises the following steps: and pre-training the text corpus through a generative pre-training GPT model to determine an intermediate parameter value, wherein the intermediate parameter value is the parameter value of the GPT model. The method comprises the steps of obtaining a sample dialogue in a sample dialogue scene, and determining the corresponding relation between the sample parameter value and the identifier of the sample dialogue scene according to the sample dialogue in the sample dialogue scene and a GPT model with intermediate parameter values, wherein the sample dialogue comprises a sample dialogue statement, a sample reply statement and a sample dialogue scene identifier. And updating the GPT model according to the sample parameter value to obtain a preset model. And after the preset model is obtained and the conversation statement is obtained, obtaining a current conversation scene identifier of the conversation generation system, wherein the current conversation scene identifier is a preset conversation scene identifier. And acquiring the conversation information according to the conversation statement and the current conversation scene identifier. And acquiring a target parameter value corresponding to the conversation scene identifier in the conversation information, wherein the target parameter value is a parameter value of a preset model, and the preset model is applicable to a plurality of conversation scenes. And processing the dialogue sentences by adopting a preset model of the target parameter values. And determining a reply sentence corresponding to the conversation sentence. And returning a reply sentence corresponding to the conversation sentence. In the process, the current dialog scene identification of the dialog generation system is changed, so that the answer sentence corresponding to the dialog sentence can be returned in different dialog scenes, the specificity and complexity of the dialog generation method in various dialog generation systems in the prior art are avoided, and the universality of the dialog generation method is improved.
Fig. 4 is a schematic structural diagram of a dialog generating device according to an embodiment of the present disclosure. As shown in fig. 4, the dialog generating device 10 includes: a first obtaining module 101, a second obtaining module 102, a first determining module 103, and a feedback module 104, wherein,
the first obtaining module 101 is configured to obtain dialog information, where the dialog information includes a dialog statement and a dialog scene identifier;
the second obtaining module 102 is configured to obtain a target parameter value corresponding to the dialog scene identifier, where the target parameter value is a parameter value of a preset model, and the preset model is applicable to multiple dialog scenes;
the first determining module 103 is configured to process the dialogue statement by using a preset model of the target parameter value, and determine a reply statement corresponding to the dialogue statement;
the feedback module 104 is configured to return a reply sentence corresponding to the dialog sentence.
The dialog generating device provided in this embodiment may be configured to execute the technical solution of the method embodiment, and the implementation principle and the technical effect are similar, which are not described herein again.
Fig. 5 is a schematic structural diagram of a dialog generating device according to an embodiment of the present disclosure. On the basis of fig. 4, as shown in fig. 5, the dialog generating device 10 further includes: a second determination module 105, the second determination module 105 being configured to:
before the target parameter value matched with the conversation scene identification is obtained, pre-training processing is carried out on the text corpus through a generative pre-training GPT model, and an intermediate parameter value is determined, wherein the intermediate parameter value is the parameter value of the GPT model;
obtaining a sample dialogue in a sample dialogue scene, and determining a corresponding relation between a sample parameter value and an identifier of the sample dialogue scene according to the sample dialogue in the sample dialogue scene and a GPT (general packet transport) model with the intermediate parameter value, wherein the sample dialogue comprises a sample dialogue statement, a sample reply statement and a sample dialogue scene identifier;
and updating the GPT model according to the sample parameter value to obtain the preset model.
In an optional implementation manner, the second determining module 105 is specifically configured to:
performing sentence division processing on the text corpus to determine M groups of sentences, wherein each group of sentences comprises at least one sentence;
pre-training the M sets of sentences for N times simultaneously through the GPT model to obtain N sets of loss values of the GPT model, wherein each set of loss values comprises M loss values, and each loss value corresponds to one set of sentences and candidate intermediate parameter values;
obtaining an average loss value corresponding to each group of loss values according to the M loss values included in each group of loss values;
and determining the intermediate parameter value according to the average loss value corresponding to each group of loss values and the candidate intermediate parameter value, wherein M and N are integers greater than or equal to 1.
In an optional implementation manner, the second determining module 105 is specifically configured to:
determining a minimum average loss value in the average loss values corresponding to each group of loss values;
acquiring a target loss value group where the minimum average loss value is located;
determining a minimum loss value among the set of target loss values;
and determining the candidate intermediate parameter value corresponding to the minimum loss value as the intermediate parameter value.
In an optional implementation, the second determining module 105 is further configured to:
according to a preset splicing rule, carrying out splicing processing on the identifier of the sample conversation scene and the sample conversation in the sample conversation scene to obtain P groups of splicing conversations;
performing Q training treatments on the P groups of splicing dialogues simultaneously through the GPT model with the intermediate parameter values to obtain Q groups of loss values of the GPT model with the intermediate parameter values, wherein each group of loss values comprises P loss values, the P loss values respectively correspond to one group of splicing dialogues and candidate sample parameter values, and each candidate sample parameter value has an identifier;
obtaining average loss values corresponding to the Q groups of loss values according to the P loss values included in the Q groups of loss values;
and determining the sample parameter value according to the average loss value corresponding to each of the Q groups of loss values and each candidate sample parameter value, and determining the corresponding relation between the identifier of the sample parameter value and the identifier of the sample dialogue scene according to the identifier of the sample dialogue scene and the identifier of each candidate sample parameter value, wherein P and Q are integers greater than or equal to 1.
In an optional implementation, the second determining module 105 is further configured to:
determining a minimum average loss value in the average loss values corresponding to the Q groups of loss values;
acquiring a target loss value group where the minimum average loss value is located;
determining a minimum loss value among the set of target loss values;
determining the candidate sample parameter value corresponding to the minimum loss value as the sample parameter value, determining the identifier of the candidate sample parameter value corresponding to the minimum loss value as the identifier of the sample parameter value,
and storing the identifier of the sample parameter and the identifier of the sample conversation scene to obtain the corresponding relation between the identifier of the sample parameter value and the identifier of the sample conversation scene.
In an optional implementation manner, the first obtaining module 101 is specifically configured to:
after the dialog statement is obtained, obtaining a current dialog scene identifier of the system, wherein the current dialog scene identifier is a preset dialog scene identifier;
and acquiring the dialogue information according to the dialogue statement and the current dialogue scene identifier.
In an optional implementation manner, the preset dialog scene identifier is any one of a single-turn dialog scene identifier, a multi-turn dialog scene identifier, a dialog scene identifier based on tag control, a dialog scene identifier based on personalized background information, and a dialog scene identifier based on knowledge.
The dialog generating device provided in this embodiment may be configured to execute the technical solution of the foregoing method embodiment, and the implementation principle and the technical effect are similar, which are not described herein again.
Fig. 6 is a schematic diagram of a hardware structure of a dialog generating device according to an embodiment of the present disclosure. As shown in fig. 6, the dialog generating device 20 of the present embodiment includes: a processor 201 and a memory 202; wherein
A memory 202 for storing computer-executable instructions;
the processor 201 is configured to execute the computer-executable instructions stored in the memory to implement the steps performed by the dialog generation method in the above embodiments. Reference may be made in particular to the description relating to the method embodiments described above.
Alternatively, the memory 202 may be separate or integrated with the processor 201.
When the memory 202 is provided separately, the dialogue generating apparatus further includes a bus 203 for connecting the memory 202 and the processor 201.
The embodiment of the present disclosure also provides a computer-readable storage medium, in which computer-executable instructions are stored, and when a processor executes the computer-executable instructions, the dialog generating method executed by the dialog generating device is implemented.
In the several embodiments provided in the embodiments of the present disclosure, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of modules is only one logical division, and other divisions may be realized in practice, for example, a plurality of modules may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present application.
It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.
The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile storage NVM, such as at least one disk memory, and may also be a usb disk, a removable hard disk, a read-only memory, a magnetic or optical disk, etc.
The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. The buses in fig. 6 are not limited to only one bus or one type of bus for ease of illustration.
The storage medium may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (16)
1. A dialog generation method applied to a dialog generation system includes:
obtaining dialogue information, wherein the dialogue information comprises dialogue sentences and dialogue scene marks;
acquiring a target parameter value corresponding to the conversation scene identifier, wherein the target parameter value is a parameter value of a preset model, and the preset model is applicable to a plurality of conversation scenes;
processing the dialogue sentences by adopting a preset model of the target parameter values, and determining reply sentences corresponding to the dialogue sentences;
returning a reply sentence corresponding to the conversation sentence;
before the obtaining of the target parameter value matching the dialog scene identification, the method further includes:
pre-training the text corpus through a generative pre-training GPT model to determine an intermediate parameter value, wherein the intermediate parameter value is a parameter value of the GPT model;
obtaining a sample dialogue in a sample dialogue scene, and determining a corresponding relation between a sample parameter value and an identifier of the sample dialogue scene according to the sample dialogue in the sample dialogue scene and a GPT (general packet transport) model with the intermediate parameter value, wherein the sample dialogue comprises a sample dialogue statement, a sample reply statement and a sample dialogue scene identifier;
and updating the GPT model according to the sample parameter value to obtain the preset model.
2. The method of claim 1, wherein the pre-training the text corpus by the generative pre-trained GPT model to determine intermediate parameter values comprises:
performing sentence division processing on the text corpus to determine M groups of sentences, wherein each group of sentences comprises at least one sentence;
pre-training the M sets of sentences for N times simultaneously through the GPT model to obtain N sets of loss values of the GPT model, wherein each set of loss values comprises M loss values, and each loss value corresponds to one set of sentences and candidate intermediate parameter values;
obtaining an average loss value corresponding to each group of loss values according to the M loss values included in each group of loss values;
and determining the intermediate parameter value according to the average loss value corresponding to each group of loss values and the candidate intermediate parameter value, wherein M and N are integers greater than or equal to 1.
3. The method of claim 2, wherein determining the intermediate parameter value according to the average loss value corresponding to each group of loss values and the candidate intermediate parameter value comprises:
determining a minimum average loss value in the average loss values corresponding to each group of loss values;
acquiring a target loss value group where the minimum average loss value is located;
determining a minimum loss value among the set of target loss values;
and determining the candidate intermediate parameter value corresponding to the minimum loss value as the intermediate parameter value.
4. The method of claim 1, wherein obtaining a sample dialogue in a sample dialogue scene, determining a sample parameter value and a correspondence of an identification of the sample parameter value to an identification of the sample dialogue scene from the sample dialogue in the sample dialogue scene and a GPT model with the intermediate parameter value comprises:
according to a preset splicing rule, carrying out splicing processing on the identifier of the sample conversation scene and the sample conversation in the sample conversation scene to obtain P groups of splicing conversations;
performing Q training treatments on the P groups of splicing dialogues simultaneously through the GPT model with the intermediate parameter values to obtain Q groups of loss values of the GPT model with the intermediate parameter values, wherein each group of loss values comprises P loss values, the P loss values respectively correspond to one group of splicing dialogues and candidate sample parameter values, and each candidate sample parameter value has an identifier;
obtaining average loss values corresponding to the Q groups of loss values according to the P loss values included in the Q groups of loss values;
and determining the sample parameter value according to the average loss value corresponding to each of the Q groups of loss values and each candidate sample parameter value, and determining the corresponding relation between the identifier of the sample parameter value and the identifier of the sample dialogue scene according to the identifier of the sample dialogue scene and the identifier of each candidate sample parameter value, wherein P and Q are integers greater than or equal to 1.
5. The method of claim 4, wherein the determining the sample parameter value according to the average loss value corresponding to each of the Q sets of loss values and the each candidate sample parameter value, and the determining the correspondence between the identifier of the sample parameter value and the identifier of the sample dialog scenario according to the identifier of the sample dialog scenario and the identifier of each candidate sample parameter value comprises:
determining a minimum average loss value in the average loss values corresponding to the Q groups of loss values;
acquiring a target loss value group where the minimum average loss value is located;
determining a minimum loss value among the set of target loss values;
determining the candidate sample parameter value corresponding to the minimum loss value as the sample parameter value, determining the identifier of the candidate sample parameter value corresponding to the minimum loss value as the identifier of the sample parameter value,
and storing the identifier of the sample parameter and the identifier of the sample conversation scene to obtain the corresponding relation between the identifier of the sample parameter value and the identifier of the sample conversation scene.
6. The method according to any one of claims 1-5, wherein the obtaining the dialog information comprises:
after the dialog statement is obtained, obtaining a current dialog scene identifier of the system, wherein the current dialog scene identifier is a preset dialog scene identifier;
and acquiring the dialogue information according to the dialogue statement and the current dialogue scene identifier.
7. The method according to claim 6, wherein the preset dialog scene identifier is any one of a single-turn dialog scene identifier, a multi-turn dialog scene identifier, a dialog scene identifier based on tag control, a dialog scene identifier based on personalized background information, and a dialog scene identifier based on knowledge.
8. A dialog generation apparatus applied to a dialog generation system, the apparatus comprising: a first obtaining module, a second obtaining module, a first determining module and a feedback module, wherein,
the first acquisition module is used for acquiring dialogue information, and the dialogue information comprises dialogue sentences and dialogue scene marks;
the second obtaining module is configured to obtain a target parameter value corresponding to the dialog scene identifier, where the target parameter value is a parameter value of a preset model, and the preset model is applicable to multiple dialog scenes;
the first determining module is used for processing the dialogue sentences by adopting a preset model of the target parameter values and determining reply sentences corresponding to the dialogue sentences;
the feedback module is used for returning a reply sentence corresponding to the conversation sentence;
the device further comprises: a second determination module to:
before the target parameter value matched with the conversation scene identification is obtained, pre-training processing is carried out on the text corpus through a generative pre-training GPT model, and an intermediate parameter value is determined, wherein the intermediate parameter value is the parameter value of the GPT model;
obtaining a sample dialogue in a sample dialogue scene, and determining a corresponding relation between a sample parameter value and an identifier of the sample dialogue scene according to the sample dialogue in the sample dialogue scene and a GPT (general packet transport) model with the intermediate parameter value, wherein the sample dialogue comprises a sample dialogue statement, a sample reply statement and a sample dialogue scene identifier;
and updating the GPT model according to the sample parameter value to obtain the preset model.
9. The apparatus of claim 8, wherein the second determining module is specifically configured to:
performing sentence division processing on the text corpus to determine M groups of sentences, wherein each group of sentences comprises at least one sentence;
pre-training the M sets of sentences for N times simultaneously through the GPT model to obtain N sets of loss values of the GPT model, wherein each set of loss values comprises M loss values, and each loss value corresponds to one set of sentences and candidate intermediate parameter values;
obtaining an average loss value corresponding to each group of loss values according to the M loss values included in each group of loss values;
and determining the intermediate parameter value according to the average loss value corresponding to each group of loss values and the candidate intermediate parameter value, wherein M and N are integers greater than or equal to 1.
10. The apparatus of claim 9, wherein the second determining module is specifically configured to:
determining a minimum average loss value in the average loss values corresponding to each group of loss values;
acquiring a target loss value group where the minimum average loss value is located;
determining a minimum loss value among the set of target loss values;
and determining the candidate intermediate parameter value corresponding to the minimum loss value as the intermediate parameter value.
11. The apparatus of claim 8, wherein the second determining module is further configured to:
according to a preset splicing rule, carrying out splicing processing on the identifier of the sample conversation scene and the sample conversation in the sample conversation scene to obtain P groups of splicing conversations;
performing Q training treatments on the P groups of splicing dialogues simultaneously through the GPT model with the intermediate parameter values to obtain Q groups of loss values of the GPT model with the intermediate parameter values, wherein each group of loss values comprises P loss values, the P loss values respectively correspond to one group of splicing dialogues and candidate sample parameter values, and each candidate sample parameter value has an identifier;
obtaining average loss values corresponding to the Q groups of loss values according to the P loss values included in the Q groups of loss values;
and determining the sample parameter value according to the average loss value corresponding to each of the Q groups of loss values and each candidate sample parameter value, and determining the corresponding relation between the identifier of the sample parameter value and the identifier of the sample dialogue scene according to the identifier of the sample dialogue scene and the identifier of each candidate sample parameter value, wherein P and Q are integers greater than or equal to 1.
12. The apparatus of claim 11, wherein the second determining module is further configured to:
determining a minimum average loss value in the average loss values corresponding to the Q groups of loss values;
acquiring a target loss value group where the minimum average loss value is located;
determining a minimum loss value among the set of target loss values;
determining the candidate sample parameter value corresponding to the minimum loss value as the sample parameter value, determining the identifier of the candidate sample parameter value corresponding to the minimum loss value as the identifier of the sample parameter value,
and storing the identifier of the sample parameter and the identifier of the sample conversation scene to obtain the corresponding relation between the identifier of the sample parameter value and the identifier of the sample conversation scene.
13. The apparatus according to any one of claims 8 to 12, wherein the first obtaining module is specifically configured to:
after the dialog statement is obtained, obtaining a current dialog scene identifier of the system, wherein the current dialog scene identifier is a preset dialog scene identifier;
and acquiring the dialogue information according to the dialogue statement and the current dialogue scene identifier.
14. The apparatus of claim 13, wherein the preset dialog scene identifier is any one of a single-turn dialog scene identifier, a multi-turn dialog scene identifier, a dialog scene identifier based on tag control, a dialog scene identifier based on personalized background information, and a dialog scene identifier based on knowledge.
15. A dialog generation device, comprising:
a memory for storing a program;
a processor for executing the program stored by the memory, the processor being configured to perform the method of any of claims 1 to 7 when the program is executed.
16. A computer-readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910537494.XA CN110245224B (en) | 2019-06-20 | 2019-06-20 | Dialog generation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910537494.XA CN110245224B (en) | 2019-06-20 | 2019-06-20 | Dialog generation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110245224A CN110245224A (en) | 2019-09-17 |
| CN110245224B true CN110245224B (en) | 2021-08-10 |
Family
ID=67888558
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910537494.XA Active CN110245224B (en) | 2019-06-20 | 2019-06-20 | Dialog generation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110245224B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111339280A (en) * | 2020-03-23 | 2020-06-26 | 上海奔影网络科技有限公司 | Question and answer sentence processing method, device, equipment and storage medium |
| CN112860862B (en) * | 2021-02-01 | 2022-11-11 | 北京邮电大学 | Method and device for generating intelligent agent dialogue sentences in man-machine dialogue |
| CN114356860A (en) * | 2022-01-06 | 2022-04-15 | 支付宝(杭州)信息技术有限公司 | Dialog generation method and device |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20060110787A (en) * | 2005-04-20 | 2006-10-25 | 가부시키가이샤 소니 컴퓨터 엔터테인먼트 | Conversation aid device |
| US8374859B2 (en) * | 2008-08-20 | 2013-02-12 | Universal Entertainment Corporation | Automatic answering device, automatic answering system, conversation scenario editing device, conversation server, and automatic answering method |
| US20130325483A1 (en) * | 2012-05-29 | 2013-12-05 | GM Global Technology Operations LLC | Dialogue models for vehicle occupants |
| KR102159220B1 (en) * | 2017-05-11 | 2020-09-23 | 경희대학교 산학협력단 | Method for intent-context fusioning in healthcare systems for effective dialogue management |
| CN108874967B (en) * | 2018-06-07 | 2023-06-23 | 腾讯科技（深圳）有限公司 | Dialogue state determining method and device, dialogue system, terminal and storage medium |
| CN109033223B (en) * | 2018-06-29 | 2021-09-07 | 北京百度网讯科技有限公司 | Method, apparatus, device and computer-readable storage medium for cross-type conversation |
| CN109814831A (en) * | 2019-01-16 | 2019-05-28 | 平安普惠企业管理有限公司 | Intelligent dialogue method, electronic device and storage medium |
-
2019
- 2019-06-20 CN CN201910537494.XA patent/CN110245224B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110245224A (en) | 2019-09-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111625635B (en) | Question-answering processing method, device, equipment and storage medium | |
| US10515155B2 (en) | Conversational agent | |
| US10242667B2 (en) | Natural language generation in a spoken dialogue system | |
| CN107230475B (en) | Voice keyword recognition method and device, terminal and server | |
| CN110245224B (en) | Dialog generation method and device | |
| CN110543552B (en) | Conversation interaction method and device and electronic equipment | |
| CN109215630B (en) | Real-time voice recognition method, device, equipment and storage medium | |
| CN111695352A (en) | Grading method and device based on semantic analysis, terminal equipment and storage medium | |
| CN109284502B (en) | Text similarity calculation method and device, electronic equipment and storage medium | |
| US11113335B2 (en) | Dialogue system and computer program therefor | |
| CN111243571B (en) | Text processing method, device and equipment and computer readable storage medium | |
| CN110765270B (en) | Training method and system of text classification model for spoken language interaction | |
| US10579837B2 (en) | Method, device and electronic apparatus for testing capability of analyzing a two-dimensional code | |
| CN111291172A (en) | Method and device for processing text | |
| CN111767394A (en) | Abstract extraction method and device based on artificial intelligence expert system | |
| CN111344717A (en) | Interactive behavior prediction method, intelligent device and computer-readable storage medium | |
| TWI749349B (en) | Text restoration method, device, electronic equipment and computer readable storage medium | |
| CN107734123A (en) | A kind of contact sequencing method and device | |
| CN111339274B (en) | Dialogue generation model training method, dialogue generation method and device | |
| CN112579760A (en) | Man-machine conversation method and device, computer equipment and readable storage medium | |
| CN112735407A (en) | Conversation processing method and device | |
| CN116384412B (en) | Dialogue content generation method and device, computer readable storage medium and terminal | |
| CN112527967A (en) | Text matching method, device, terminal and storage medium | |
| CN117370512A (en) | Method, device, equipment and storage medium for replying to dialogue | |
| CN111046674A (en) | Semantic understanding method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |