CN117668171A

CN117668171A - Text generation method, training device, electronic equipment and storage medium

Info

Publication number: CN117668171A
Application number: CN202311338133.5A
Authority: CN
Inventors: 焦学武; 胡明清; 王礼鹏
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2023-10-16
Filing date: 2023-10-16
Publication date: 2024-03-08

Abstract

The disclosure provides a text generation method, a training device, electronic equipment and a storage medium, relates to the technical field of artificial intelligence, and particularly relates to the technical fields of deep learning, text processing and the like. The text generation method comprises the following specific implementation scheme: responding to the received question text aiming at the target scene, and processing the question text to obtain matching probabilities of a plurality of candidate words used for generating the reply text and the question text; determining target candidate words from a plurality of candidate words according to a preset word set and matching probability corresponding to the target scene, wherein the target candidate words represent candidate words which exist in the preset word set and have matching probability larger than a first preset threshold value; and generating a reply text according to the target candidate word.

Description

Text generation method, training device, electronic equipment and storage medium

Technical Field

The disclosure relates to the technical field of artificial intelligence, in particular to the technical fields of deep learning, text processing and the like, and specifically relates to a text generation method, a training device, electronic equipment and a storage medium.

Background

Along with the rapid development of a large-scale pre-training language model technology based on a transducer model structure, the training corpus of the large-scale pre-training language model is rich, and the learning effect of the language model is good, so that the pre-training language model is widely applied to a plurality of fields to adapt to different application scenes.

Disclosure of Invention

The disclosure provides a text generation method, a training device, electronic equipment and a storage medium.

According to an aspect of the present disclosure, there is provided a text generation method including: responding to the received question text aiming at the target scene, and processing the question text to obtain matching probabilities of a plurality of candidate words used for generating the reply text and the question text; determining target candidate words from a plurality of candidate words according to a preset word set and matching probability corresponding to the target scene, wherein the target candidate words represent candidate words which exist in the preset word set and have matching probability larger than a first preset threshold value; and generating a reply text according to the target candidate word.

According to another aspect of the present disclosure, there is provided a training method of a text generation model, the text generation model including a processing module, a selecting module, and a generating module, including: inputting a sample question text into a processing module to obtain sample matching probabilities of a plurality of sample candidate words for generating a sample reply text and the sample question text, wherein the sample question text is used for describing question information of a sample target scene; inputting a plurality of sample candidate words and sample matching probability into a selection module to obtain sample target candidate words; the sample candidate word characterization exists in a sample preset word set corresponding to a sample target scene, and the sample matching probability is larger than a first preset threshold value; inputting the sample target candidate words into a generating module to obtain sample reply texts; obtaining a loss value according to the sample reply text and the label text corresponding to the sample question text based on the target loss function; and adjusting parameters of the processing module based on the loss value to obtain a trained text generation model.

According to another aspect of the present disclosure, there is provided a text generating apparatus including: the device comprises a first processing module, a first selecting module and a first generating module. The first processing module is used for responding to the received question text aiming at the target scene, processing the question text and obtaining matching probabilities of a plurality of candidate words used for generating the reply text and the question text; the first selection module is used for determining target candidate words from a plurality of candidate words according to a preset word set and matching probability corresponding to a target scene, wherein the target candidate words represent candidate words which exist in the preset word set and have the matching probability larger than a first preset threshold value; and the first generation module is used for generating a reply text according to the target candidate word.

According to another aspect of the present disclosure, there is provided a training apparatus of a text generation model, including: the system comprises a second processing module, a second selecting module, a second generating module, a loss calculating module and a first adjusting module. The second processing module is used for inputting the sample question text into the processing module to obtain sample matching probabilities of a plurality of sample candidate words used for generating sample reply texts and the sample question text, wherein the sample question text is used for describing question information of a sample target scene; the second selection module is used for inputting a plurality of sample candidate words and sample matching probabilities into the selection module to obtain sample target candidate words; the sample candidate word characterization exists in a sample preset word set corresponding to a sample target scene, and the sample matching probability is larger than a first preset threshold value; the second generation module is used for inputting the sample target candidate words into the generation module to obtain sample reply texts; the loss calculation module is used for obtaining a loss value according to the sample reply text and the label text corresponding to the sample question text based on the target loss function; and the first adjusting module is used for adjusting parameters of the processing module based on the loss value to obtain a trained text generation model.

According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method as described above.

According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a method as described above.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 schematically illustrates an exemplary system architecture to which text generation methods and apparatus may be applied, according to embodiments of the present disclosure;

FIG. 2 schematically illustrates a flow chart of a text generation method according to an embodiment of the present disclosure;

FIG. 3 schematically illustrates a training phase and application phase schematic of a text generation large model applying a text generation method of an embodiment of the present disclosure;

FIG. 4A schematically illustrates a schematic diagram of a limited generation of reply text corresponding to a target scene in accordance with an embodiment of the disclosure;

FIG. 4B schematically illustrates a schematic diagram of a defined generation of reply text corresponding to a target scene according to another embodiment of the disclosure;

FIG. 5 schematically illustrates a schematic diagram of a defined generation of reply text corresponding to a target scene according to yet another embodiment of the disclosure;

FIG. 6 schematically illustrates a flow chart of a training method of a text generation model according to an embodiment of the disclosure;

fig. 7 schematically shows a block diagram of a text generating apparatus according to an embodiment of the present disclosure;

FIG. 8 schematically illustrates a block diagram of a training apparatus of a text generation model according to an embodiment of the present disclosure; and

fig. 9 schematically illustrates a block diagram of an electronic device adapted to implement a text generation method or a training method of a text generation model, according to an embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

The generation of results based on a large-scale pre-trained language model of a transducer model structure is relatively versatile. However, for a particular scenario, for example: the generated results may relate to specific game names or terms in specific scenes, the problem text in the specific scenes is directly processed by using the pre-training language model, and the generated results have low correlation degree and accuracy with the specific scenes.

In view of the above, the present disclosure provides a text generation method, by processing a question text, obtaining matching probabilities of a plurality of candidate words for generating a reply text and the question text; and determining target candidate words from the plurality of candidate words according to the preset word set corresponding to the target scene and the matching probability. Since the target candidate words used to generate the reply text are candidate words that exist in the predetermined word set and have a matching probability greater than the first predetermined threshold, the degree of correlation and accuracy of the reply text with the specific scene are improved while ensuring that the reply text matches the question text.

Fig. 1 schematically illustrates an exemplary system architecture to which text generation methods and apparatus may be applied, according to embodiments of the present disclosure.

It should be noted that fig. 1 is only an example of a system architecture to which embodiments of the present disclosure may be applied to assist those skilled in the art in understanding the technical content of the present disclosure, but does not mean that embodiments of the present disclosure may not be used in other devices, systems, environments, or scenarios. For example, in another embodiment, an exemplary system architecture to which the text generation method and apparatus may be applied may include a terminal device, but the terminal device may implement the text generation method and apparatus provided by the embodiments of the present disclosure without interacting with a server.

As shown in fig. 1, a system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and/or wireless communication links, and the like.

The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications may be installed on the terminal devices 101, 102, 103, such as a knowledge reading class application, a web browser application, a search class application, an instant messaging tool, a mailbox client and/or social platform software, etc. (as examples only).

The terminal devices 101, 102, 103 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.

The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for content browsed by the user using the terminal devices 101, 102, 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.

Note that, the text generation method provided by the embodiment of the present disclosure may be generally performed by the terminal device 101, 102, or 103. Accordingly, the text generating apparatus provided by the embodiments of the present disclosure may also be provided in the terminal device 101, 102, or 103.

Alternatively, the text generation method provided by the embodiments of the present disclosure may be generally performed by the server 105. Accordingly, the text generating apparatus provided by the embodiments of the present disclosure may be generally provided in the server 105. The text generation method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the text generating apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.

For example, the terminal devices 101, 102, 103 may acquire a question text sent by the user in the target scene, then send the acquired question text to the server 105, and the server 105 processes the question text to obtain matching probabilities of a plurality of candidate words for generating the reply text and the question text; determining target candidate words from the plurality of candidate words according to a preset word set and matching probability corresponding to the target scene; and generating a reply text according to the target candidate word. Or the server or server cluster capable of communicating with the terminal devices 101, 102, 103 and/or the server 105 processes the question text and finally enables the generation of a reply text corresponding to the target scene.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing, applying and the like of the personal information of the user all conform to the regulations of related laws and regulations, necessary security measures are adopted, and the public order harmony is not violated.

In the technical scheme of the disclosure, the authorization or consent of the user is obtained before the personal information of the user is obtained or acquired.

Fig. 2 schematically illustrates a flow chart of a text generation method according to an embodiment of the present disclosure.

As shown in fig. 2, the method 200 includes operations S210 to S230.

In operation S210, in response to the received question text for the target scene, the question text is processed, resulting in matching probabilities of a plurality of candidate words used to generate the reply text and the question text.

In operation S220, a target candidate word is determined from among the plurality of candidate words according to a predetermined word set corresponding to the target scene and the matching probability.

In operation S230, a reply text is generated according to the target candidate word.

According to embodiments of the present disclosure, the target scene may be various scenes in practical applications, such as: advertising, games, life scenes, etc. Words describing the category to which the target scene belongs may be included in the question text, for example: the question text may be "please recommend me a most popular game". The target scene may be determined to be a game scene. Words for describing the category to which the target scene belongs may not be included in the question text, and specific scene descriptors, for example: the question text may be "please recommend me a restaurant with a box in which 10 people can be accommodated". "restaurants" may be categorized as living scenes.

According to embodiments of the present disclosure, question text may be processed using a pre-trained language model based on a transducer model structure, resulting in a probability of matching a plurality of candidate words to the question text for generating reply text. For a pre-trained language model, complete reply text is typically generated through multiple iterations, each of which may generate multiple candidate words. The probability of matching the candidate word with the question text may represent the probability of matching the word with the question text in a corpus of the pre-training process.

For example: the question text may be "please recommend me a most popular game", and inputting the question text into the pre-trained language model, the obtained matching probability of the candidate word and the question text may sequentially include: "M" (match probability: 0.6), "P" (match probability: 0.4), "U" (match probability: 0.5).

In the related art, a target candidate word is generally determined from candidate words based on a matching probability, and thus, a reply text generated based on the matching probability may not be related to a target scene due to high versatility of a pre-trained language model. For example: there is no game in the game scene in which a name includes "M", but the answer text generated based on the matching probability is "Mxxx" game.

Accordingly, embodiments of the present disclosure filter candidate words based on a predetermined word set corresponding to a target scene, thereby enabling model-defined generation of reply text matching the target scene.

According to an embodiment of the present disclosure, the predetermined word set corresponding to the target scene may include words having a higher probability of occurrence in the target scene, for example: in a game scenario, the predetermined set of words may include various game names, game development mechanisms, and the like. In an advertising scenario, the predetermined set of words may include subject keywords or the like for describing a certain type of merchandise. The target candidate word characterizes candidate words that are present in the predetermined set of words and that have a matching probability that is greater than a first predetermined threshold.

For example: "U" and "V" may be included in a predetermined word set corresponding to a game scene. From the predetermined word set corresponding to the target scene, it may be determined that the candidate word "U" exists in the predetermined word set, and thus, it may be determined that the target candidate word is "U".

For example: "U" and "M" may be included in a predetermined word set corresponding to a game scene. From the predetermined word set corresponding to the target scene, it may be determined that both candidate words "U" and "M" exist in the predetermined word set. The predetermined threshold may be 0.5, and since only the candidate word "M" has a matching probability greater than 0.5, the target candidate word may be determined to be "M".

According to the embodiment of the disclosure, the selection operation is performed for each round of generated candidate words of the text generation model to determine the target candidate word generated for each round. It should be noted that, based on the characteristics of the generated model, the target candidate word generated in each round may be used together with the question text as the input text of the next round, and finally, the reply text matched with the question text is generated in a guided manner.

According to the embodiment of the disclosure, matching probabilities of a plurality of candidate words for generating the reply text and the question text are obtained by processing the question text; and determining target candidate words from the plurality of candidate words according to the preset word set corresponding to the target scene and the matching probability. Since the target candidate words used to generate the reply text are candidate words that exist in the predetermined word set and have a matching probability greater than the first predetermined threshold, the degree of correlation and accuracy of the reply text with the specific scene are improved while ensuring that the reply text matches the question text.

The method shown in fig. 2 is further described below with reference to fig. 3-5 in conjunction with the exemplary embodiment.

Fig. 3 schematically illustrates a training phase and an application phase schematic of a text generation large model to which the text generation method of the embodiment of the present disclosure is applied.

As shown in fig. 3, in embodiment 300, in a model training phase, an initial model is trained using a sample corpus 301, resulting in a text-generated generic large model 302. The text generation generic large model 302 may be a generic model that is applicable to various fields, and thus the generation results are more generic and broad.

In the model application phase, a predetermined vocabulary 303 corresponding to the target scene may be loaded before the text generation generic large model 302 is started. The question text 311 is then input into a defined text generation model 3021 loaded with a predetermined vocabulary 303 corresponding to the target scene.

Based on constraints on the candidate words by the predetermined set of words, a target candidate word 312 is determined from the plurality of candidate words generated for each round. Finally, reply text 313 is generated from the target candidate word 312.

Fig. 4A schematically illustrates a schematic diagram of a defined generation of reply text corresponding to a target scene according to an embodiment of the disclosure.

As shown in fig. 4A, in an embodiment 400A, question text 411 in a target scene is input into a defined text generation model. The defined text generation model may be constructed based on a transducer model structure. The text features of the question text are extracted, and then the text features are processed based on an attention mechanism, so that the matching probability of a plurality of candidate words and the question text is obtained.

According to an embodiment of the present disclosure, the predetermined word set is a dictionary tree composed of N layers of predetermined word nodes, N being an integer greater than 1. In embodiment 400A, N may be equal to 4. In the dictionary tree, the root node may be a preset fixed word, and may be determined according to a specific application scenario, where the first word representing all reply texts is the same, for example: may be an "answer".

According to the embodiment of the disclosure, the first candidate word is obtained by performing an mth round of processing on the question text by using a limited text generation model; the second candidate word is obtained by processing the question text and the first target candidate word in the m+1th round by using a limited text generation model, and m is an integer greater than or equal to 1. Determining a first target preset word which is the same as the first candidate word from a plurality of preset words in an N-th layer node of the dictionary tree, wherein N is an integer which is more than or equal to 1 and less than or equal to N; determining a first target candidate word from the first target predetermined words based on the first matching probability; determining a second target predetermined word identical to the second candidate word from a plurality of predetermined words in the n+1 level node associated with the first target predetermined word; and determining a second target candidate word from the second target predetermined words based on the second match probability.

For example: the match probability 421 for the first candidate word may be: a is that ₁ (0.7)、A ₃ (0.2) and M (0.8), in the dictionary tree of the target scene, the predetermined word A may be included in the layer 2 node associated with the root node ₁ And a predetermined word A2, a first target predetermined word identical to the first candidate word can be determined as the predetermined word A ₁ . Since only one predetermined word is matched based on the dictionary tree, the predetermined word A ₁ I.e., the first target candidate 422.

According to an embodiment of the present disclosure, the candidate words may be plural, and the target predetermined word selected based on the dictionary tree may be plural. For example: the matching probability 423 of the second candidate word may be: b (B) ₁ (0.7)、B ₂ (0.3), N (0.2) and B ₃ (0.8). In the layer 3 node of the dictionary tree, the first target predetermined word A ₁ The associated node may include the predetermined word B therein ₁ And predetermined word B ₂ . Since the second candidate word includes B ₁ And B ₂ Therefore, B with high matching probability can be obtained based on the second matching probability ₁ (0.7) as a second target candidate word 424. And generating a reply text A based on the first target candidate word and the second target candidate word ₁ B ₁ 431。

According to the embodiment of the disclosure, in the case that the number of target predetermined words is greater than 1, a threshold value of matching probability may be set based on the requirement of the actual application scene, and the predetermined word with the second matching probability greater than the threshold value of matching probability may be used as the second target candidate word. At this time, the second target candidate word may be multi-medium.

According to the embodiment of the disclosure, based on the association relation between the preset words in the dictionary tree and the preset words in the dictionary tree, the target candidate words matched with the target scene can be directionally selected from the candidate words generated in each round, so that the relevance of the reply text and the target scene and the accuracy of the reply text are improved.

For some more complex application scenes, the related preset words are more, the association relation is more complex, the association degree between different preset words can be additionally arranged in the dictionary tree, and based on the second matching probability and the association degree, the second target candidate words are determined from the second target preset words, so that the generation result of the model is limited.

Fig. 4B schematically illustrates a schematic diagram of a defined generation of reply text corresponding to a target scene according to another embodiment of the disclosure.

As shown in fig. 4B, in embodiment 400B, question text 411 in the target scene is input into a defined text generation model. The structure of the limited text generation model in this embodiment is the same as that in embodiment 400A, and will not be described here.

According to an embodiment of the present disclosure, a degree of association between each predetermined word in the n-th layer node and each predetermined word in the n+1-th layer node is included in the dictionary tree, for example: predetermined word A in layer 2 node ₁ The association with the root node may be 0.7, the predetermined word A in the layer 2 node ₂ The association with the root node may be 0.3.

The selection process of the first target candidate word 422 is the same as that in embodiment 400A according to the embodiment of the present disclosure, and will not be described here. The matching probability 423 of the second candidate word may include: b (B) ₁ (0.7)、B ₂ (0.3)、N(0.2)、B ₃ (0.8). In layer 3 nodes of the dictionary tree with the predetermined word A ₁ The associated node includes the predetermined word B ₁ And predetermined word B ₂ Since the second candidate word includes B ₁ And B ₂ It can be determined that the second target predetermined word is the predetermined word B ₁ And predetermined word B ₂ 。

According to an embodiment of the present disclosure, word A is reserved in a dictionary tree ₁ And predetermined word B ₁ The association degree between the words is 0.2, and the predetermined word A ₁ And predetermined word B ₂ The degree of association between the two is 0.8.

According to an embodiment of the present disclosure, a third target predetermined word may be determined from the second target predetermined words based on the degree of association. For example: the association threshold can be set to be 0.5, and the predetermined word A ₁ And predetermined word B ₂ The association degree between the target words is 0.8 and is larger than 0.5, and the third target preset word can be determined to be the preset word B ₂ 。

The number of the third target predetermined words determined from the second target predetermined words based on the association degree may be plural, and the second target candidate word may be determined from the third target predetermined words based on the second matching probability for the plural third target predetermined words. Since there is only one third target predetermined word in this embodiment 400B, the second target candidate word 425 may be determined to be B ₂ 。

According to the embodiment of the disclosure, the relevance of the second target predetermined word and the first target candidate word can be obtained according to the second matching probability and the relevance, and the second target candidate word is determined from the second target predetermined word based on the relevance.

For example: the second target predetermined word is the predetermined word B ₁ And predetermined word B ₂ Predefining word A in dictionary tree ₁ And predetermined word B ₁ The association degree between the words is 0.2, and the predetermined word A ₁ And predetermined word B ₂ The degree of association between the two is 0.8. B in the second candidate word ₁ Is 0.7, B ₂ The matching probability of (2) is 0.3.

According to the embodiment of the disclosure, the correlation degree of the second target predetermined word and the first target candidate word can be obtained based on the product of the second matching probability and the correlation degree. For example: predetermined word B ₁ With the first target candidate word A ₁ The correlation between may be 0.7×0.2=0.14. Predetermined word B ₂ With the first target candidate word A ₁ The correlation between may be 0.3×0.8=0.24. Since 0.24 is greater than 0.14, it can be determined that the second target candidate word is B ₂ 。

According to the embodiment of the disclosure, the relevance of the second target predetermined word and the first target candidate word may also be determined according to the relevance, and the relevance weight is determined first; and obtaining the correlation degree according to the second matching probability and the correlation weight.

According to embodiments of the present disclosure, the relevance weights of a plurality of predetermined words may be determined according to a relevance ratio between the plurality of predetermined words in the n+1 layer node associated with the predetermined word of the n layer node. For example: for example: predefining word A in dictionary tree ₁ And predetermined word B ₁ The association degree between the words is 0.2, and the predetermined word A ₁ And predetermined word B ₂ The degree of association between the two is 0.8,0.2:0.8 =1: 4. the predetermined word A can be determined ₁ And predetermined word B ₁ The correlation weight between the words is 1, and the predetermined word A ₁ And predetermined word B ₂ The correlation weight between them is 4. Obtaining a preset word B according to the second matching probability and the related weight ₁ With the first target candidate word A ₁ The correlation between them is 0.7×1=0.7; predetermined word B ₂ With the first target candidate word A ₁ The correlation between them is 0.3×4=1.2. Since 0.7 is less than 1.2, it can be determined that the second target candidate word is B ₂ 。

According to an embodiment of the present disclosure, based on the method described above, the third target candidate word 427 may be determined to be C from the third candidate word based on the matching probability 426 of the third candidate word and the dictionary tree ₂ . And finally generates reply text 428.

According to the embodiment of the disclosure, the association degree between different preset words can be determined based on the semantics between different words in the application scene, and also can be determined based on the probability that the different words in the application scene appear in the same sentence. Candidate words matched with the target scene are directionally selected based on the association degree between the preset words, so that the method is suitable for generating rare reply texts in more complex or special scenes, and the accuracy of the reply texts is improved.

According to an embodiment of the present disclosure, in the limited text generation model, the target candidate words selected based on the dictionary tree each time may be plural, and in order to improve the recommendation accuracy of the reply text, the target word may be determined from the target candidate words; and generating a reply text according to the target word.

According to the embodiment of the disclosure, for a plurality of target candidate words generated in the same round, the target candidate words can be ranked based on the matching probability of the target candidate words and the problem text, so as to obtain a ranking result; and determining a target word from the target candidate words based on the ranking result.

For example: the target candidate words obtained based on the matching probability threshold are S and T respectively, wherein the matching probability of S and the problem text is 0.8, and the matching probability of T and the problem text is 0.5, and the target word can be determined to be S.

According to the embodiment of the disclosure, the dictionary tree is adopted to improve the correlation degree between the candidate words and the target scene, and meanwhile, the matching probability between the candidate words and the problem text is combined, so that the pre-trained large model precision advantage can be fully invoked, and the accuracy of the generated result is improved.

According to the embodiment of the disclosure, since the predetermined word set includes a plurality of predetermined words and a degree of association between the plurality of predetermined words, a plurality of candidate reply texts can be generated according to the target candidate words; determining the matching degree of a plurality of candidate reply texts and the question text according to the association degree; and determining a reply text from the plurality of candidate reply texts based on the degree of matching.

Fig. 5 schematically illustrates a schematic diagram of a defined generation of reply text corresponding to a target scene according to still another embodiment of the disclosure.

As shown in fig. 5, in embodiment 500, a question text 511 in a target scene is input into a limited text generation model, candidate words are selected based on a dictionary tree of the target scene, and a first target candidate word a is obtained ₁ 521. Second target candidate word B ₂ 522 and a third target candidate word 523C ₂ And C ₃ 。

According to the embodiment of the disclosure, the arrangement order of the target candidate words in the candidate reply text can be determined according to the positions of the target candidate words in the dictionary tree; and generating a plurality of candidate reply texts according to the target candidate words based on the arrangement order.

According to an embodiment of the present disclosure, determining an arrangement order of a target candidate word in a candidate reply text according to a position of the target candidate word in a dictionary tree may include the operations of: and responding to the nth layer of the ith target candidate word and the jth target candidate word in the dictionary tree, determining the nth order of the ith target candidate word in the first candidate reply text, and the nth order of the jth target candidate word in the second candidate reply text, wherein N is an integer greater than or equal to 1 and less than or equal to N, and I and j are integers greater than or equal to 1 and less than or equal to I.

For example: third target candidate word C ₂ And C ₃ All at level 4 in the dictionary tree, therefore, C ₂ May be at the 4 th order, C, in the first candidate reply text ₃ And may be at the 4 th order in the second candidate reply text.

According to an embodiment of the disclosure, in response to an nth level of an ith target candidate word in the dictionary tree, a jth target candidate word in an n+1th level in the dictionary tree, an nth order of the ith target candidate word in the first candidate reply text is determined, and the jth target candidate word in the n+1th order in the first candidate reply text.

For example: first target candidate word A ₁ At layer 2 in the dictionary tree, a second target candidate word B ₂ At layer 3 in the dictionary tree, a first target candidate word A may be determined ₁ The rank in the first candidate reply text 531 is the 2 nd word; second target candidate word B ₂ At the first candidateThe order in reply text 531 is word 3. The root node may be a null character in this embodiment 500.

Thus, the first candidate reply text 531 generated in this embodiment 500 is a ₁ B ₂ C ₂ The method comprises the steps of carrying out a first treatment on the surface of the The second candidate reply text 532 is A ₁ B ₂ C ₃ 。

According to the embodiment of the disclosure, in an actual application scenario, a plurality of candidate reply texts may be output as a final reply text as a model. The degree of matching of the plurality of candidate answer texts with the question text may also be determined based on the degree of association, and the final answer text may be determined from the plurality of candidate answer texts based on the degree of matching.

For example: in the first candidate reply text 531, the root node is associated with A ₁ The degree of association between the two is 0.7, A ₁ And B is connected with ₂ The degree of association between the two is 0.8, B ₂ And C ₂ The degree of association between them is 0.4. The degree of matching between the first candidate reply text 531 and the question text 511 may be determined based on the average of the degrees of association, that is: 0.95.

for example: in the second candidate reply text 532, the root node is associated with A ₁ The degree of association between the two is 0.7, A ₁ And B is connected with ₂ The degree of association between the two is 0.8, B ₂ And C ₃ The degree of association between the two is 0.6. The degree of matching between the second candidate reply text 532 and the question text 511 may be determined from the average of the degrees of association, that is: 1.05.

since the degree of matching between the first candidate answer text 531 and the question text 511 is 0.95 smaller than the degree of matching between the second candidate answer text 532 and the question text 511 is 1.05, it can be determined that the answer text 533 is a ₁ B ₂ C ₃ 。

According to the embodiment of the disclosure, besides the average value according to the association degree, association weights can be configured for adjacent layer nodes in the dictionary tree, and the association degree is weighted based on the association weights to obtain the final matching degree.

According to the embodiment of the disclosure, candidate reply texts are screened based on the relevance between different preset words in the dictionary tree, so that the accuracy of the generated result is improved from the perspective of semantic relevance of the whole reply text.

The method provided by the embodiment of the disclosure is realized on the basis of the pre-trained language model, and for the language model with rich training corpus and good learning effect, the dictionary tree corresponding to the target scene can be directly loaded without secondary training, so that the reply text which is related to the target scene and matched with the problem text can be generated.

However, in the practical application scenario, the pre-trained language model may have the problems of less corpus and poor learning effect in the pre-training process. In this case, after loading the dictionary tree corresponding to the target scene, the accuracy of the output result can be improved through secondary training.

Fig. 6 schematically illustrates a flowchart of a training method of a text generation model according to an embodiment of the present disclosure.

As shown in fig. 6, the training method 600 may include operations S610 to S650.

In operation S610, the sample question text is input to the processing module, resulting in sample matching probabilities of a plurality of sample candidate words used to generate the sample reply text and the sample question text.

In operation S620, a plurality of sample candidate words and sample matching probabilities are input to the selection module to obtain sample target candidate words.

In operation S630, the sample target candidate word is input to the generation module, resulting in a sample reply text.

In operation S640, a loss value is obtained from the sample reply text and the tag text corresponding to the sample question text based on the target loss function.

In operation S650, parameters of the processing module are adjusted based on the loss value, resulting in a trained text generation model.

According to an embodiment of the present disclosure, sample question text is used to describe question information of a sample target scene. The sample target candidate word representation is present in a sample predetermined word set corresponding to the sample target scene and the sample match probability is greater than a first predetermined threshold. The sample question text, the sample preset word set and the sample target candidate word are the same as the definition ranges of the question text, the preset word set and the target candidate word in the text generation method described above, and are not repeated here.

According to embodiments of the present disclosure, the objective loss function may employ any loss function suitable for deep learning model training, such as: cross entropy loss function, etc., to which embodiments of the present disclosure are not particularly limited.

According to an embodiment of the present disclosure, the tag text characterizes standard reply text corresponding to the sample question text that is adapted to the target scene. Because the preset word set can enable the model to directionally generate the reply text related to the target scene, in the training process, the parameters of the pre-trained language model, namely the parameters of the processing module, can be mainly adjusted based on the loss value, so that the purpose of improving the model accuracy is achieved.

According to the embodiment of the disclosure, because the association relationship between different preset words can be preconfigured in the preset word set, the association relationship also affects the accuracy of the result, so that in the training process, parameters of the selection module, namely parameters such as association degree or association weight among different preset words, can be adjusted based on the loss value, and the efficiency of model training is improved.

Fig. 7 schematically shows a block diagram of a text generating apparatus according to an embodiment of the present disclosure.

As shown in fig. 7, the text generating apparatus 700 may include a first processing module 710, a first selecting module 720, and a first generating module 730.

The first processing module 710 is configured to process, in response to the received question text for the target scene, the question text, and obtain matching probabilities of the plurality of candidate words used for generating the reply text and the question text.

The first selection module 720 is configured to determine a target candidate word from the plurality of candidate words according to a predetermined word set and a matching probability corresponding to the target scene, where the target candidate word characterizes candidate words that exist in the predetermined word set and have a matching probability greater than a first predetermined threshold.

A first generating module 730, configured to generate a reply text according to the target candidate word.

According to an embodiment of the present disclosure, the predetermined word set is a dictionary tree composed of N layers of predetermined word nodes, N being an integer greater than 1; the plurality of candidate words includes a first candidate word and a second candidate word; the first candidate word is obtained by carrying out the mth round of processing on the question text; the second candidate word is obtained by carrying out m+1th round of processing on the question text and the first target candidate word, and m is an integer greater than or equal to 1. The first selection module includes: the first selection sub-module, the first determination sub-module, the second selection sub-module, and the second determination sub-module.

The first selecting sub-module is used for determining a first target preset word which is the same as the first candidate word from a plurality of preset words in the N-th layer node of the dictionary tree, wherein N is an integer which is more than or equal to 1 and less than or equal to N.

The first determining sub-module is used for determining a first target candidate word from first target preset words based on the first matching probability.

A second selection sub-module for determining a second target predetermined word that is identical to the second candidate word from a plurality of predetermined words in the n+1 level node associated with the first target predetermined word.

And a second determining sub-module for determining a second target candidate word from the second target predetermined word based on the second matching probability.

According to an embodiment of the present disclosure, a degree of association between each predetermined word in the n-th layer node and each predetermined word in the n+1-th layer node is included in the dictionary tree. The first selection module further includes: and a third determining sub-module. And a third determining sub-module for determining a second target candidate word from the second target predetermined words based on the second matching probability and the association degree.

According to an embodiment of the present disclosure, the third determining sub-module may include: a first determination unit and a second determination unit. And the first determining unit is used for determining a third target preset word from the second target preset words according to the association degree, wherein the association degree of the third target preset word and the first target preset word is larger than a second preset threshold value. And a second determining unit configured to determine a second target candidate word from the third target predetermined word based on the second matching probability.

According to an embodiment of the present disclosure, the third determination submodule includes: and a correlation calculation unit and a third determination unit. And the correlation calculation unit is used for obtaining the correlation between the second target preset word and the first target candidate word according to the second matching probability and the correlation. And a third determining unit configured to determine a second target candidate word from the second target predetermined words based on the degree of correlation.

According to an embodiment of the present disclosure, a correlation calculation unit includes: a weight calculation subunit and a correlation calculation subunit. And the weight calculating subunit is used for determining the related weight according to the association degree. And the correlation calculation subunit is used for obtaining the correlation according to the second matching probability and the correlation weight.

According to an embodiment of the present disclosure, the first generation module includes: a fourth determination sub-module and a first generation sub-module. And a fourth determination submodule, configured to determine a target word from the target candidate words. And the first generation sub-module is used for generating a reply text according to the target word.

According to an embodiment of the present disclosure, the fourth determination sub-module may include: a sorting unit and a fourth determining unit. And the ordering unit is used for ordering the target candidate words based on the matching probability of the target candidate words and the problem text, and obtaining an ordering result. And a fourth determining unit configured to determine a target word from the target candidate words based on the ranking result.

According to an embodiment of the present disclosure, the predetermined word set includes a plurality of predetermined words and a degree of association between the plurality of predetermined words. The first generation module includes: the system comprises a second generation submodule, a matching degree calculation submodule and a fifth determination submodule. And the second generation sub-module is used for generating a plurality of candidate reply texts according to the target candidate words. And the matching degree calculation submodule is used for determining the matching degree of the candidate answer texts and the question text according to the association degree. And a fifth determination sub-module for determining a reply text from the plurality of candidate reply texts based on the degree of matching.

According to an embodiment of the present disclosure, the predetermined word set is a dictionary tree composed of N layers of predetermined word nodes, N being an integer greater than 1; the second generation submodule includes: a bit sequence determining unit and a first generating unit. And the order determining unit is used for determining the arrangement order of the target candidate words in the candidate reply text according to the positions of the target candidate words in the dictionary tree. And the first generation unit is used for generating a plurality of candidate reply texts according to the target candidate words based on the arrangement order.

According to an embodiment of the present disclosure, a bit sequence determining unit includes: a first bit-order determination subunit and a second bit-order determination subunit. A first order determining subunit, configured to determine, in response to an nth layer of the ith target candidate word and the jth target candidate word in the dictionary tree, an nth order of the ith target candidate word in the first candidate reply text, and an nth order of the jth target candidate word in the second candidate reply text, where N is an integer greater than or equal to 1 and less than or equal to N, and I, j are integers greater than or equal to 1 and less than or equal to I. A second order determining subunit, configured to determine, in response to an nth level of the ith target candidate word in the dictionary tree, an n+1th level of the jth target candidate word in the dictionary tree, an nth order of the ith target candidate word in the first candidate reply text, and an n+1th order of the jth target candidate word in the first candidate reply text.

According to an embodiment of the present disclosure, a matching degree calculation submodule includes: a first matching degree calculating unit and a second matching degree calculating unit. And the first matching degree calculation unit is used for determining the matching degree of the first candidate reply text and the question text according to the association degree among a plurality of target candidate words used for generating the first candidate reply text. And a second matching degree calculation unit for determining the matching degree of the second candidate reply text and the question text according to the association degree among the plurality of target candidate words for generating the second candidate reply text.

According to an embodiment of the present disclosure, the first processing module includes: the feature extraction sub-module and the attention sub-module. And the feature extraction sub-module is used for extracting text features of the problem text. And the attention sub-module is used for processing the text characteristics based on an attention mechanism to obtain matching probabilities of a plurality of candidate words and the problem text.

Fig. 8 schematically illustrates a block diagram of a training apparatus of a text generation model according to an embodiment of the present disclosure.

As shown in fig. 8, the training apparatus 800 may include a second processing module 810, a second selection module 820, a second generation module 830, a loss calculation module 840, and a first adjustment module 850.

The second processing module 810 is configured to input a sample question text into the processing module, and obtain sample matching probabilities of a plurality of sample candidate words for generating a sample reply text and the sample question text, where the sample question text is used to describe question information of a sample target scene.

A second selection module 820, configured to input a plurality of sample candidate words and sample matching probabilities into the selection module, to obtain a sample target candidate word; the sample target candidate word characterizes sample candidate words which exist in a sample preset word set corresponding to the sample target scene and have a sample matching probability larger than a first preset threshold value.

The second generating module 830 is configured to input the sample target candidate word into the generating module, and obtain a sample reply text.

The loss calculation module 840 is configured to obtain a loss value based on the target loss function according to the sample reply text and the tag text corresponding to the sample question text.

A first adjustment module 850, configured to adjust parameters of the processing module based on the loss value, to obtain a trained text generation model.

According to an embodiment of the disclosure, the training device may further include a second adjustment module for adjusting parameters of the processing module and parameters of the selection module based on the loss value, resulting in a trained text generation module.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described previously.

According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method described above.

According to an embodiment of the present disclosure, a computer program product comprising a computer program which, when executed by a processor, implements the method described in the foregoing.

Fig. 9 shows a schematic block diagram of an example electronic device 900 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 9, the apparatus 900 includes a computing unit 901 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other by a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.

Various components in device 900 are connected to I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, or the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 908 such as a magnetic disk, an optical disk, or the like; and a communication unit 909 such as a network card, modem, wireless communication transceiver, or the like. The communication unit 909 allows the device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunications networks.

The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the respective methods and processes described above, for example, a text generation method or a training method of a text generation model. For example, in some embodiments, the text generation method or training method of the text generation model may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 900 via the ROM 902 and/or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the text generation method or the training method of the text generation model described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the text generation method or the training method of the text generation model in any other suitable way (e.g. by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A text generation method, comprising:

responding to a received question text aiming at a target scene, and processing the question text to obtain matching probabilities of a plurality of candidate words used for generating a reply text and the question text;

determining a target candidate word from the plurality of candidate words according to a preset word set corresponding to the target scene and the matching probability, wherein the target candidate word characterizes the candidate words which exist in the preset word set and have the matching probability larger than a first preset threshold value; and

And generating the reply text according to the target candidate word.

2. The method of claim 1, wherein the predetermined vocabulary is a dictionary tree consisting of N layers of predetermined vocabulary nodes, N being an integer greater than 1; the plurality of candidate words includes a first candidate word and a second candidate word; the first candidate word is obtained by carrying out mth round processing on the question text; the second candidate word is obtained by carrying out m+1th round of processing on the question text and the first target candidate word, wherein m is an integer greater than or equal to 1;

the determining a target candidate word from the plurality of candidate words according to a predetermined word set corresponding to the target scene and the matching probability comprises the following steps:

determining a first target preset word identical to the first candidate word from a plurality of preset words in an N-layer node of the dictionary tree, wherein N is an integer greater than or equal to 1 and less than or equal to N;

determining the first target candidate word from the first target predetermined word based on a first matching probability;

determining a second target predetermined word identical to the second candidate word from a plurality of predetermined words in an n+1 level node associated with the first target predetermined word; and

the second target candidate word is determined from the second target predetermined word based on a second match probability.

3. The method of claim 2, wherein the dictionary tree includes a degree of association between each predetermined word in the n-th level node and each predetermined word in the n+1-th level node; further comprises:

and determining the second target candidate word from the second target predetermined word based on the second matching probability and the association degree.

4. The method of claim 3, wherein the determining the second target candidate word from the second target predetermined word based on the second match probability and the degree of association comprises:

determining a third target preset word from the second target preset words according to the association degree, wherein the association degree of the third target preset word and the first target preset word is larger than a second preset threshold; and

and determining the second target candidate word from the third target predetermined word based on the second matching probability.

5. The method of claim 3, wherein the determining the second target candidate word from the second target predetermined word based on the second match probability and the degree of association comprises:

obtaining the correlation degree between the second target preset word and the first target candidate word according to the second matching probability and the correlation degree; and

And determining the second target candidate word from the second target predetermined words based on the relevance.

6. The method of claim 5, wherein the obtaining the relevance of the second target predetermined word and the first target candidate word according to the second matching probability and the relevance comprises:

according to the association degree, determining a correlation weight; and

and obtaining the correlation degree according to the second matching probability and the correlation weight.

7. The method of claim 1, wherein the generating the reply text from the target candidate word comprises:

determining a target word from the target candidate words; and

and generating the reply text according to the target word.

8. The method of claim 7, wherein the determining a target word from the target candidate words comprises:

sorting the target candidate words based on the matching probability of the target candidate words and the problem text to obtain a sorting result; and

and determining the target word from the target candidate words based on the sorting result.

9. The method of claim 1, the set of predetermined words comprising a plurality of predetermined words and a degree of association between the plurality of predetermined words; the generating the reply text according to the target candidate word comprises the following steps:

Generating a plurality of candidate reply texts according to the target candidate words;

determining the matching degree of the candidate reply texts and the question text according to the association degree; and

the answer text is determined from the plurality of candidate answer texts based on the degree of matching.

10. The method of claim 9, wherein the predetermined vocabulary is a dictionary tree consisting of N layers of predetermined vocabulary nodes, N being an integer greater than 1; the generating a plurality of candidate reply texts according to the target candidate words comprises the following steps:

determining the arrangement order of the target candidate words in the candidate reply text according to the positions of the target candidate words in the dictionary tree; and

and generating a plurality of candidate reply texts according to the target candidate words based on the arrangement order.

11. The method of claim 10, wherein the target candidate word comprises I, I being an integer greater than 1; the determining the arrangement sequence of the target candidate words in the candidate reply text according to the positions of the target candidate words in the dictionary tree comprises the following steps:

determining an nth order of the ith target candidate word in a first candidate reply text and an nth order of the jth target candidate word in a second candidate reply text in response to an nth layer of the ith target candidate word and the jth target candidate word in the dictionary tree, wherein N is an integer greater than or equal to 1 and less than or equal to N, and I and j are integers greater than or equal to 1 and less than or equal to I; and

And responding to the nth layer of the ith target candidate word in the dictionary tree, the (n+1) th layer of the jth target candidate word in the dictionary tree, and determining the nth order of the ith target candidate word in the first candidate reply text, wherein the (n+1) th order of the jth target candidate word in the first candidate reply text.

12. The method of claim 9, wherein the determining, according to the degree of association, a degree of matching of the plurality of candidate answer texts with the question text comprises:

determining the matching degree of the first candidate reply text and the question text according to the association degree among a plurality of target candidate words for generating the first candidate reply text; and

and determining the matching degree of the second candidate reply text and the question text according to the association degree among a plurality of target candidate words for generating the second candidate reply text.

13. The method of claim 1, wherein the processing the question text to obtain a probability of matching a plurality of candidate words used to generate a reply text with the question text comprises:

extracting text features of the question text; and

And processing the text characteristics based on an attention mechanism to obtain matching probabilities of a plurality of candidate words and the problem text.

14. A training method of a text generation model comprises a processing module, a selection module and a generation module; comprising the following steps:

inputting a sample question text into a processing module to obtain sample matching probabilities of a plurality of sample candidate words for generating a sample reply text and the sample question text, wherein the sample question text is used for describing question information of a sample target scene;

inputting the plurality of sample candidate words and the sample matching probability into a selection module to obtain sample target candidate words; wherein the sample target candidate word representation exists in a sample predetermined word set corresponding to the sample target scene and the sample matching probability is greater than a first predetermined threshold;

inputting the sample target candidate words into the generation module to obtain the sample reply text;

obtaining a loss value according to the sample reply text and the label text corresponding to the sample question text based on a target loss function; and

and adjusting parameters of the processing module based on the loss value to obtain a trained text generation model.

15. The method of claim 14, further comprising:

and adjusting parameters of the processing module and parameters of the selection module based on the loss value to obtain the trained text generation module.

16. A text generation apparatus comprising:

the first processing module is used for responding to the received question text aiming at the target scene, processing the question text and obtaining matching probabilities of a plurality of candidate words used for generating a reply text and the question text;

a first selection module, configured to determine a target candidate word from the plurality of candidate words according to a predetermined word set corresponding to the target scene and the matching probability, where the target candidate word characterizes a candidate word that exists in the predetermined word set and the matching probability is greater than a first predetermined threshold; and

and the first generation module is used for generating the reply text according to the target candidate word.

17. The apparatus of claim 16, wherein the predetermined vocabulary is a dictionary tree consisting of N layers of predetermined vocabulary nodes, N being an integer greater than 1; the plurality of candidate words includes a first candidate word and a second candidate word; the first candidate word is obtained by carrying out mth round processing on the question text; the second candidate word is obtained by carrying out m+1th round of processing on the question text and the first target candidate word, wherein m is an integer greater than or equal to 1; the first selection module includes:

A first selecting sub-module, configured to determine a first target predetermined word identical to the first candidate word from a plurality of predetermined words in an N-th layer node of the dictionary tree, where N is an integer greater than or equal to 1 and less than or equal to N;

a first determining sub-module for determining the first target candidate word from the first target predetermined word based on a first matching probability;

a second selection sub-module for determining a second target predetermined word that is identical to the second candidate word from a plurality of predetermined words in an n+1 level node associated with the first target predetermined word; and

and the second determining submodule is used for determining the second target candidate word from the second target preset word based on the second matching probability.

18. The apparatus of claim 17, wherein the dictionary tree includes a degree of association between each predetermined word in the n-th level node and each predetermined word in the n+1-th level node; the first selection module further includes:

and a third determining sub-module, configured to determine the second target candidate word from the second target predetermined word based on the second matching probability and the association degree.

19. The apparatus of claim 18, wherein the third determination submodule comprises:

A first determining unit, configured to determine a third target predetermined word from the second target predetermined words according to the association degree, where the association degree between the third target predetermined word and the first target predetermined word is greater than a second predetermined threshold; and

and a second determining unit configured to determine the second target candidate word from the third target predetermined word based on the second matching probability.

20. The apparatus of claim 18, wherein the third determination submodule comprises:

the correlation calculation unit is used for obtaining the correlation between the second target preset word and the first target candidate word according to the second matching probability and the correlation; and

and a third determining unit configured to determine the second target candidate word from the second target predetermined words based on the correlation degree.

21. The apparatus of claim 20, wherein the correlation calculation unit comprises:

a weight calculating subunit, configured to determine a correlation weight according to the correlation degree; and

and the correlation calculation subunit is used for obtaining the correlation according to the second matching probability and the correlation weight.

22. The apparatus of claim 16, wherein the first generation module comprises:

A fourth determining sub-module, configured to determine a target word from the target candidate words; and

and the first generation sub-module is used for generating the reply text according to the target word.

23. The apparatus of claim 22, wherein the fourth determination submodule comprises:

the ordering unit is used for ordering the target candidate words based on the matching probability of the target candidate words and the problem text to obtain an ordering result; and

and a fourth determining unit configured to determine the target word from the target candidate words based on the ranking result.

24. The apparatus of claim 16, the set of predetermined words comprising a plurality of predetermined words and a degree of association between the plurality of predetermined words; the first generation module includes:

the second generation sub-module is used for generating a plurality of candidate reply texts according to the target candidate words;

a matching degree calculation submodule, configured to determine matching degrees of the plurality of candidate reply texts and the question text according to the association degrees; and

and a fifth determination sub-module, configured to determine the reply text from the plurality of candidate reply texts based on the matching degree.

25. The apparatus of claim 24, wherein the predetermined vocabulary is a dictionary tree consisting of N layers of predetermined vocabulary nodes, N being an integer greater than 1; the second generating submodule includes:

A rank determining unit, configured to determine an arrangement rank of the target candidate word in the candidate reply text according to a position of the target candidate word in the dictionary tree; and

the first generation unit is used for generating a plurality of candidate reply texts according to the target candidate words based on the arrangement order.

26. The apparatus of claim 25, wherein the target candidate word comprises I, I being an integer greater than 1; the bit sequence determining unit includes:

a first order determining subunit, configured to determine, in response to an nth layer of the ith target candidate word and the jth target candidate word in the dictionary tree, an nth order of the ith target candidate word in a first candidate reply text, and an nth order of the jth target candidate word in a second candidate reply text, where N is an integer greater than or equal to 1 and less than or equal to N, and I, j are integers greater than or equal to 1 and less than or equal to I; and

a second order determining subunit, configured to determine, in response to an nth level of an ith target candidate word in the dictionary tree, an nth+1th level of a jth target candidate word in the dictionary tree, an nth order of the ith target candidate word in the first candidate reply text, and an nth+1th order of the jth target candidate word in the first candidate reply text.

27. The apparatus of claim 24, wherein the match computation submodule comprises:

a first matching degree calculation unit configured to determine a matching degree of the first candidate reply text and the question text according to a degree of association between a plurality of target candidate words used to generate the first candidate reply text; and

and a second matching degree calculation unit for determining the matching degree of the second candidate reply text and the question text according to the association degree among a plurality of target candidate words for generating the second candidate reply text.

28. The apparatus of claim 16, wherein the first processing module comprises:

the feature extraction sub-module is used for extracting text features of the problem text; and

and the attention sub-module is used for processing the text characteristics based on an attention mechanism to obtain matching probabilities of a plurality of candidate words and the problem text.

29. A training device of a text generation model comprises a processing module, a selection module and a generation module; comprising the following steps:

the second processing module is used for inputting the sample question text into the processing module, so as to obtain sample matching probabilities of a plurality of sample candidate words used for generating a sample reply text and the sample question text, wherein the sample question text is used for describing question information of a sample target scene;

The second selection module is used for inputting the plurality of sample candidate words and the sample matching probability into the selection module to obtain sample target candidate words; wherein the sample target candidate word representation exists in a sample predetermined word set corresponding to the sample target scene and the sample matching probability is greater than a first predetermined threshold;

the second generation module is used for inputting the sample target candidate words into the generation module to obtain the sample reply text;

the loss calculation module is used for obtaining a loss value based on a target loss function according to the sample reply text and the label text corresponding to the sample question text; and

and the first adjusting module is used for adjusting the parameters of the processing module based on the loss value to obtain a trained text generation model.

30. The apparatus of claim 29, further comprising:

and the second adjusting module is used for adjusting the parameters of the processing module and the parameters of the selecting module based on the loss value to obtain the trained text generating module.

31. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-15.

32. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-15.

33. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-15.