CN110851574A - Statement processing method, device and system - Google Patents
Statement processing method, device and system Download PDFInfo
- Publication number
- CN110851574A CN110851574A CN201810851351.1A CN201810851351A CN110851574A CN 110851574 A CN110851574 A CN 110851574A CN 201810851351 A CN201810851351 A CN 201810851351A CN 110851574 A CN110851574 A CN 110851574A
- Authority
- CN
- China
- Prior art keywords
- reply
- neural network
- vector
- keywords
- sentence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The present disclosure provides a statement processing method, including: acquiring a vector to be processed, wherein the vector to be processed comprises vector representation of an input statement; inputting a vector to be processed into a first recurrent neural network to generate one or more keywords; determining reply content corresponding to one or more keywords; and generating a reply sentence of the input sentence based on the one or more keywords and the reply content. The disclosure also provides a statement processing device and a system.
Description
Technical Field
The present disclosure relates to the field of electronic technologies, and in particular, to a method, an apparatus, and a system for processing a statement.
Background
With the development of electronic technology, man-machine conversation becomes an important research and application hotspot in the fields of speech recognition and natural language processing, and with the development of man-machine conversation systems and the gradual deepening of industrial applications, task-driven multi-turn conversation systems get more attention in the industry and also present more challenges to related research and development.
In the course of implementing the disclosed concept, the inventors found that there are at least the following problems in the prior art: the existing task-driven dialog systems mainly have a module type and an end-to-end type, wherein the module type dialog systems have the following limitations: the feedback of the user is difficult to be transmitted to the model, and the modules are mutually dependent, so that the user needs much manpower to modify when adapting to a new environment; the end-to-end type dialog system requires supervised learning of a large amount of data and a reliance tracker (belief tracker) to use a sentence occurring in the previous turn of dialog as a value of a middle state block (slot), thereby requiring much consumed parameter complexity.
Disclosure of Invention
In view of the above, the present disclosure provides a sentence processing method, apparatus and system capable of reducing complexity of a dialog system.
One aspect of the present disclosure provides a statement processing method, including: acquiring a vector to be processed, wherein the vector to be processed comprises vector representation of an input statement; inputting a vector to be processed into a first recurrent neural network to generate one or more keywords; determining reply content corresponding to one or more keywords; and generating a reply sentence of the input sentence based on the one or more keywords and the reply content.
According to an embodiment of the present disclosure, obtaining a vector to be processed includes: acquiring a current input sentence, a reply sentence generated last time and one or more keywords generated last time; converting words in a current input sentence, a reply sentence generated last time and one or more keywords generated last time into word vectors; and orderly inputting the word vectors into a second cyclic neural network, and acquiring vector representation of a hidden layer state of the second cyclic neural network as a vector to be processed, wherein under the condition that no reply sentence generated last time exists, the reply sentence generated last time and one or more keywords generated last time are default values.
According to the embodiment of the disclosure, inputting the vector to be processed into the first recurrent neural network, and generating the one or more keywords comprises the following operations executed circularly until the complete one or more keywords are generated: updating the hidden layer state of the first recurrent neural network by the input of the first recurrent neural network to obtain a plurality of current candidate keywords; and determining the current keywords according to the output probabilities of the current multiple candidate keywords, wherein the output probability of each candidate keyword comprises: and the sum of the probability generated by the first cyclic neural network and the probability obtained by copying the candidate keyword from the input statement, wherein in the cycle, a vector to be processed is used as the initial input of the first cyclic neural network, and the vector representation of the keyword obtained in the previous cycle and the vector representation of the hidden layer state are used as the input of the first cyclic neural network in the next cycle.
According to an embodiment of the present disclosure, generating a reply sentence of an input sentence based on one or more keywords and reply contents includes: inputting the vector representation of the one or more keywords, the vector representation of the hidden layer state of the first recurrent neural network and the vector representation of the answer type into a third recurrent neural network to generate one or more answer words; and generating a reply sentence of the input sentence according to the one or more reply words and the candidate reply value, wherein the reply content comprises a vector representation of the reply type and the candidate reply value, and the vector representation of the reply type is related to the number of the candidate reply value.
According to an embodiment of the present disclosure, generating one or more reply words includes looping the following operations performed until the complete one or more reply words are generated: updating the hidden layer state of the third recurrent neural network by the input of the third recurrent neural network to obtain a plurality of current candidate answer compound words; and determining the current reply word according to the output probability of the current multiple candidate reply words, wherein the output probability of each candidate reply word comprises the following steps: and the sum of the probability generated by the third recurrent neural network of the candidate reply word and the probability obtained by copying the candidate reply word from the input sentence, wherein in the cycle, the vector representation of one or more keywords, the vector representation of the final hidden layer state of the first recurrent neural network and the vector representation of the reply type are used as the initial input of the third recurrent neural network, and the vector representation of the reply word and the vector representation of the hidden layer state obtained in the previous cycle are used as the input of the third recurrent neural network in the next cycle.
Another aspect of the present disclosure provides a sentence processing apparatus, including a to-be-processed vector obtaining module, a keyword generation module, a reply content determination module, and a reply sentence generation module, where the to-be-processed vector obtaining module is configured to obtain a to-be-processed vector, and the to-be-processed vector includes a vector representation of an input sentence; the keyword generation module is used for inputting the vector to be processed into the first recurrent neural network to generate one or more keywords; the reply content determining module is used for determining reply content corresponding to the one or more keywords; the reply sentence generation module is used for generating a reply sentence of the input sentence based on the one or more keywords and the reply content.
According to the embodiment of the disclosure, the to-be-processed vector acquisition module comprises an acquisition submodule, a word vector generation submodule and a to-be-processed vector determination submodule, wherein the acquisition submodule is used for acquiring a current input sentence, a previously generated reply sentence and one or more previously generated keywords; the word vector generation submodule is used for converting words in the current input sentence, the reply sentence generated last time and one or more keywords generated last time into word vectors; the to-be-processed vector determining submodule is used for orderly inputting the word vectors to the second cyclic neural network, and obtaining the vector representation of the hidden layer state of the second cyclic neural network as the to-be-processed vector, wherein under the condition that no reply sentence generated last time exists, the reply sentence generated last time and one or more keywords generated last time are default values.
According to an embodiment of the present disclosure, the keyword generation module is configured to cyclically execute the following operations until the one or more complete keywords are generated: updating the hidden layer state of the first recurrent neural network by the input of the first recurrent neural network to obtain a plurality of current candidate keywords; and determining the current keywords according to the output probabilities of the current multiple candidate keywords, wherein the output probability of each candidate keyword comprises: and the sum of the probability generated by the first cyclic neural network and the probability obtained by copying the candidate keyword from the input statement, wherein in the above cycle, the vector to be processed is used as the initial input of the first cyclic neural network, and the vector representation of the keyword obtained in the previous cycle and the vector representation of the hidden layer state are used as the input of the first cyclic neural network in the next cycle.
According to an embodiment of the present disclosure, the reply sentence generation module includes a reply word generation sub-module and a reply sentence determination sub-module, and the reply word generation sub-module is configured to input vector representations of one or more keywords, a final hidden layer state of the first recurrent neural network, and a reply type into the third recurrent neural network to generate one or more reply words; the answer sentence determination submodule is used for generating an answer sentence of the input sentence according to the one or more answer words and the candidate answer values, wherein the answer content comprises a vector representation of the answer type and the candidate answer values, and the vector representation of the answer type is related to the number of the candidate answer values.
According to an embodiment of the present disclosure, the above-mentioned reply word generation submodule is configured to perform the following operations in a loop until the complete one or more reply words are generated: updating the hidden layer state of the third recurrent neural network by the input of the third recurrent neural network to obtain a plurality of current candidate answer compound words; and determining the current reply word according to the output probability of the current multiple candidate reply words, wherein the output probability of each candidate reply word comprises the following steps: and the sum of the probability generated by the third recurrent neural network of the candidate reply word and the probability obtained by copying the candidate reply word from the input sentence, wherein in the cycle, the vector representation of one or more keywords, the vector representation of the final hidden layer state of the first recurrent neural network and the vector representation of the reply type are used as the initial input of the third recurrent neural network, and the vector representation of the reply word and the vector representation of the hidden layer state obtained in the previous cycle are used as the input of the third recurrent neural network in the next cycle.
Another aspect of the present disclosure provides a sentence processing system including: one or more processors; a storage device for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the above-described method of determining an item to be placed.
Another aspect of the present disclosure provides a non-volatile storage medium storing computer-executable instructions for implementing the statement processing method as described above when executed.
Another aspect of the present disclosure provides a computer program comprising computer executable instructions for implementing the statement processing method as described above when executed.
According to the embodiment of the disclosure, the problem of complicated dialogue system caused by recording information about task completion in a dialogue by using a huge confidence tracker in the prior art can be at least partially avoided, and therefore, the information originally recorded by the confidence tracker can be recorded by using keywords, so that the effects of reducing the model complexity of the dialogue system and improving the statement processing efficiency can be achieved.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:
FIG. 1 is a diagram schematically illustrating an application scenario of a statement processing method, apparatus and system according to an embodiment of the present disclosure;
FIG. 2 schematically shows a flow diagram of a statement processing method according to an embodiment of the present disclosure;
FIG. 3 schematically shows a flow chart for obtaining a vector to be processed according to an embodiment of the present disclosure;
FIG. 4 schematically illustrates a flow diagram for generating one or more keywords according to an embodiment of the disclosure;
FIGS. 5A-5B schematically illustrate a flow diagram for generating a reply sentence according to an embodiment of the disclosure;
FIG. 6 schematically illustrates a schematic diagram of a statement processing method according to an embodiment of the present disclosure;
fig. 7 schematically shows a block diagram of a sentence processing apparatus according to an embodiment of the present disclosure;
fig. 8 schematically shows a block diagram of a pending vector acquisition module according to an embodiment of the present disclosure;
FIG. 9 schematically shows a structural diagram of a reply sentence generation module according to an embodiment of the present disclosure;
FIG. 10 schematically shows a block diagram of a statement processing system according to an embodiment of the disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a convention analogous to "A, B or at least one of C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B or C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase "a or B" should be understood to include the possibility of "a" or "B", or "a and B".
The embodiment of the disclosure provides a statement processing method, device and system. The method comprises the following steps: acquiring a vector to be processed, wherein the vector to be processed comprises vector representation of an input statement; inputting a vector to be processed into a first recurrent neural network to generate one or more keywords; determining reply content corresponding to one or more keywords; and generating a reply sentence of the input sentence based on the one or more keywords and the reply content.
Fig. 1 schematically illustrates an application scenario 100 of a statement processing method, apparatus and system according to an embodiment of the present disclosure. It should be noted that fig. 1 is only an example of an application scenario in which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios.
As shown in fig. 1, the application scenario 100 according to this embodiment includes a user 110 and terminal devices 121, 122, 123, and the user 110 may perform a man-machine conversation with the terminal devices 121, 122, 123 to enable the terminal devices 121, 122, 123 to complete an operation corresponding to the user's intention through the conversation.
Specifically, the user 110 may have the terminal devices 121, 122, 123 help the user complete actual specific tasks such as searching for goods sought by the user, reserving hotel restaurants, reserving airline tickets, and the like, through a dialog with the terminal devices 121, 122, 123, for example.
The terminal devices 121, 122, 123 may be installed with various client applications, such as a voice recognition-type application, a voice assistant-type application, a shopping-type application, a search-type application, social platform software, etc. (just examples), to process an input voice of a user and generate a reply sentence, implement a man-machine conversation, and perform an operation corresponding to the user's intention according to the input voice.
The terminal devices 121, 122, 123 may be, for example, electronic devices having a voice playing function, including but not limited to various electronic devices such as desktop computers, laptop portable computers, tablet computers, smart phones, portable multimedia players, cameras, and the like; or may also include, for example, various smart home devices such as televisions, audio equipment, refrigerators, air conditioners, vacuum cleaners, etc.; or may also include, for example, various medical devices (e.g., portable medical measurement devices), navigation devices, vehicle infotainment devices, marine electronics, industrial or home robots, automated teller machines for banks, and the like.
The electronic devices 121, 122, 123 of the embodiments of the present disclosure may also output a reply sentence corresponding to the input speech, for example, to request the user for further request for completing the task, or provide the user with the requirement information, by recognizing and processing the input sentence of the user, for example, for the input sentence of the user, "please help me reserve an airplane ticket flying to the guangzhou", and the reply sentence generated by the electronic device is "ask you for which day they want to reserve? ", to request further requirements from the user to complete the scheduled ticket task so that the final scheduled ticket meets the user requirements. It is to be understood that the above input sentence and the reply sentence are only examples to facilitate understanding of the present disclosure, and the present disclosure is not limited thereto.
It should be noted that the statement processing method provided by the embodiment of the present disclosure may be generally executed by the terminal devices 121, 122, 123. Accordingly, the sentence processing apparatus and system provided by the embodiments of the present disclosure may be generally disposed in the terminal devices 121, 122, 123.
It will be appreciated that the type of terminal device in figure 1 is merely illustrative. Any type of terminal equipment capable of man-machine interaction with the user can be provided according to implementation requirements.
FIG. 2 schematically shows a flow chart of a statement processing method according to an embodiment of the present disclosure.
As shown in fig. 2, the sentence processing method may include operations S210 to S240 for an input sentence of a single round of dialog among multiple rounds of dialogs of a human-machine dialog, and the sentence processing method may be performed, for example, by referring to the terminal devices 121, 122, 123 in fig. 1.
In operation S210, a to-be-processed vector is obtained, where the to-be-processed vector includes a vector representation of an input statement; in operation S220, inputting a vector to be processed to a first recurrent neural network, and generating one or more keywords; determining reply contents corresponding to one or more keywords in operation S230; in operation S240, a reply sentence of the input sentence is generated based on the one or more keywords and the reply content.
According to an embodiment of the present disclosure, the vector to be processed in operation S210 may be obtained based on input statement encoding, for example. The input sentence may be, for example, converted from a user's voice received by an audio receiver of the terminal device 121, 122, 123, and the encoding may be, for example, sequentially arranging words in the input sentence according to values corresponding to the vocabulary index table to form a value vector.
According to an embodiment of the present disclosure, the vector to be processed in operation S210 may also be obtained by: firstly converting input sentences into word vectors, then sequentially inputting the word vectors into a cyclic neural network to update the hidden layer state of the cyclic neural network, and finally representing the final hidden layer state vector of the cyclic neural network as a vector to be processed.
According to an embodiment of the present disclosure, converting an input sentence into a word vector may be, for example: recognizing the input sentence to obtain each word forming the input sentence, and sequentially arranging the words according to the sequence appearing in the input sentence to form a word vector.
Fig. 3 schematically shows a flowchart for obtaining a vector to be processed according to an embodiment of the present disclosure.
According to the embodiment of the present disclosure, the vector to be processed in operation S210 may also be obtained through operations S211 to S213 as shown in fig. 3, for example.
In operation S211, a current input sentence, a previously generated reply sentence, and one or more keywords generated previously are acquired.
According to the embodiment of the present disclosure, the current input sentence may be, for example, an input sentence in a current round of dialog, and accordingly, the previously generated reply sentence is a reply sentence obtained in a previous round of dialog through operation S240, and the one or more keywords generated in the previous round of dialog are one or more keywords obtained in the previous round of dialog through operation S220.
According to an embodiment of the present disclosure, in a case where the current round of dialog is the first round of dialog, there is no previous reply sentence, the previously generated reply sentence and the previously generated one or more keywords in operation S211 adopt default values, which may be null values, for example.
In operation S212, words in the current input sentence, the previously generated reply sentence, and the previously generated one or more keywords are converted into word vectors.
According to an embodiment of the present disclosure, the operation S212 may, for example, convert the input sentence into the word vector by using the above-mentioned method, and convert words in the current input sentence, the previously generated reply sentence, and the previously generated one or more keywords to obtain the word vector.
According to an embodiment of the present disclosure, the operation S212 may be, for example: converting the current input sentence to obtain a word vector, converting the reply sentence generated at the previous time to obtain a word vector, converting one or more keywords generated at the previous time to obtain a word vector, and finally splicing the three word vectors to obtain the final word vector.
In operation S213, the word vectors are sequentially input to the second recurrent neural network, and a vector representation of the hidden layer state of the second recurrent neural network is obtained as a to-be-processed vector.
According to an embodiment of the present disclosure, the operation S213 may specifically be: and the second cyclic neural network encodes according to the appearance sequence of each word in the word vector, the hidden layer information of the previous word is fed back to the hidden layer information of the current word, and the vector of the hidden layer state in the last cycle of the second cyclic neural network is represented as a vector to be processed until meeting the end character < eos > of the input word vector.
In the method for obtaining the to-be-processed vector in fig. 3, because the current input sentence, the reply sentence generated last time and the one or more keywords generated last time are jointly encoded, the obtained to-be-processed vector includes the information of the historical dialog, and the historical dialog can be considered when the reply sentence is obtained by decoding according to the to-be-processed vector, so that the reply sentence obtained by decoding has continuity with the historical dialog, and the reply content is more suitable for the actual requirement.
According to an embodiment of the present disclosure, the one or more keywords in operation S220 are information about task completion in the sentence processing method, which can be a basis for obtaining reply content from the knowledge base.
According to an embodiment of the present disclosure, the one or more keywords may include, for example, information class keywords (information slots) and request class keywords (request slots). The information key words are query conditions for querying from the knowledge base, and the request key words are words which can be queried. For example, a keyword indicating that a place name (e.g., the guangdong), a category (e.g., food, and beauty), and the like correspond to a plurality of results in the knowledge base is an information-based keyword, and a keyword indicating that an address, a telephone, and the like correspond to a unique result in the knowledge base under the restriction of the information-based keyword is a request-based keyword.
According to an embodiment of the present disclosure, the one or more keywords may include, for example, words in a currently input sentence, and may also include words in any one of multiple rounds of conversations before the current round of conversation, and the keywords processed by the sentences in the current round of conversation may be the same as or different from the keywords processed by the sentences in the previous round of conversation.
According to the embodiment of the present disclosure, the information class keyword in the one or more keywords obtained in operation S220 may be identified by < inf > < \ inf >, the request class keyword may be identified by < req > < \ req >, and two adjacent keywords are separated by a semicolon, so as to facilitate the search call of the terminal device in reference to fig. 1.
FIG. 4 schematically shows a flow diagram for generating one or more keywords according to an embodiment of the disclosure.
According to an embodiment of the present disclosure, the operation S220 of generating one or more keywords may specifically include the operations S221 to S222 of executing in a loop until a complete keyword is generated.
In operation S221, updating the hidden layer state of the first recurrent neural network with the input of the first recurrent neural network to obtain a plurality of current candidate keywords; in operation S222, a current keyword is determined according to the output probabilities of the current plurality of candidate keywords.
According to an embodiment of the present disclosure, in the first loop, the input of the first-loop neural network is the to-be-processed vector obtained in operation S210, that is, the to-be-processed vector is an initial input, so as to update the hidden layer state of the first-loop neural network, and the output is a keyword; in each subsequent cycle, the vector representation of the keyword and the updated vector representation of the hidden layer state obtained in the previous cycle are used as the input of the first cyclic neural network, so as to update the hidden layer state of the first cyclic neural network again, output a keyword, and sequentially circulate until an end character < eos > is output, thereby completing the generation of the one or more keywords.
According to an embodiment of the present disclosure, the output probability of each candidate keyword includes: the probability of the candidate keyword generated by the first recurrent neural network is, for example, the probability of the candidate keyword being obtained by using the same scoring function as the general encoder-decoder, and the probability of the candidate keyword being obtained by being copied from the input sentence is, for example: and selecting the probability of the candidate keyword from the word vector of the input sentence by using the hidden state represented in the vector to be processed.
According to the embodiment of the present disclosure, the operation of generating one or more keywords through the first recurrent neural network with the vector to be processed as an initial input may be regarded as a decoding process, that is, decoding the vector to be processed to obtain the one or more keywords.
According to an embodiment of the present disclosure, the above-mentioned operations S221 to S222 that are executed in a loop may also be executed by, for example, a copy mechanism neural network that is more inclined to generate words in the input sentence when generating the keyword. The reply sentence obtained based on the keyword obtained by the copy mechanism neural network can avoid the problem that the generated result and the input effective information cannot be fused in the traditional method, and therefore, the generation effect of the reply sentence can be effectively improved.
Considering that for some named entities (such as names of people, etc.), it is difficult to accurately generate words mentioned in the input sentence through the neural network, therefore, when determining the keywords, the probability that the candidate keywords are copied from the input sentence is also considered in the above embodiment, the keywords may be directly copied from the input sentence, so that the defect that the existing dialogue system cannot handle the problem of incomplete dictionary (out-of-vocabulary) can be solved to some extent.
According to an embodiment of the present disclosure, the operation S230 may specifically be to query the knowledge base to obtain the reply content corresponding to the one or more keywords according to the one or more keywords, and then the one or more keywords may be an index of the query knowledge base.
According to embodiments of the present disclosure, the reply content may include, for example, a vector representation of the reply type and a candidate reply value, where the vector representation of the reply type is related to the number of candidate reply values.
According to the embodiment of the present disclosure, the query-derived response content may specifically include two parts, one part is the candidate response value corresponding to the keyword queried in the knowledge base, the other part is the representation vector for representing the number of the candidate response values obtained by the query, the vector may be, for example, a three-dimensional vector for representing three types of query results (each dimension representing a case of a knowledge base query), such as a first value of the three-dimensional vector for representing one candidate answer value of the query (where one is a set of contents corresponding to a keyword), a second value for representing a plurality of candidate answer values of the query, a third value for representing no corresponding candidate answer value of the query, for example, if the three-dimensional vector is (0, 0, 1), it indicates that no candidate answer value is queried from the knowledge base according to the keyword.
According to the embodiment of the present disclosure, different response types may be corresponded according to different query results, for example, for a case where the candidate response value obtained by the query is one, there may be a first response type for providing the candidate response value obtained by the query to the user; for the above-mentioned case that the candidate answer value is multiple, there may be a second answer type for the user to provide further inquiry condition; for the above case where no candidate reply value is queried, there may be a third reply type to indicate to the user that no corresponding information is found. Therefore, the above-mentioned representation vector for representing the number of candidate answer values obtained by the query corresponds one-to-one to the answer type, and thus, the representation vector corresponds to the vector representation of the answer type.
Fig. 5A-5B schematically show a flow chart for generating a reply sentence according to an embodiment of the present disclosure.
According to an embodiment of the present disclosure, the operation S240 may be specifically a one-time decoding process, for example, decoding the input thereof through a third recurrent neural network to obtain a reply sentence of the input sentence.
According to an embodiment of the present disclosure, the reply sentence may also be generated, for example, by operations S241 to S242 described with reference to fig. 5A.
In operation S241, a vector representation of one or more keywords, a vector representation of a hidden layer state of the first recurrent neural network, and a vector representation of a reply type are input to the third recurrent neural network, generating one or more reply words.
According to an embodiment of the present disclosure, before the operation S241 is performed, for example, the vector representation of the reply type and the vector representation of the hidden layer state of the first recurrent neural network may be first spliced into one vector, and then the spliced vector is used as an input of the third recurrent neural network.
According to an embodiment of the present disclosure, the operation S241 may specifically be to cyclically execute operations S2411 to S2412 referring to fig. 5B, and in operation S2411, update the hidden layer state of the third recurrent neural network with the input of the third recurrent neural network to obtain a plurality of current candidate compound words; in operation S2412, a current compound word is determined according to the output probabilities of the current plurality of candidate compound words.
According to the embodiment of the disclosure, in the loop, the vector representation of the one or more keywords, the vector representation of the final hidden layer state of the first recurrent neural network and the vector representation of the answer type are used as the initial input of the third recurrent neural network, namely as the input of the third recurrent neural network in the first loop, so as to update the hidden layer state of the third recurrent neural network and output an answer word; in each subsequent cycle, the vector representation of the reply word obtained in the previous cycle and the updated vector representation of the hidden layer state are used as the input of the third cyclic neural network, so as to update the hidden layer state of the third cyclic neural network again and output a reply word, and the cycle is performed in sequence until the output end character < eos >, so as to finish the generation of the reply word or the reply words.
According to an embodiment of the present disclosure, the output probability of each candidate compound word includes: the probability of the candidate reply word generated by the third recurrent neural network is, for example, the probability of the candidate reply word obtained by using the same scoring function as the general encoder-decoder, and the probability of the candidate reply word obtained by copying from the input sentence is specifically: and finally, representing the probability of obtaining the candidate reply word by selecting the vector of the hidden layer state from the word vector of the input sentence through the vector of the first recurrent neural network, and/or representing the probability of obtaining the candidate reply word by copying the keyword from the vector of the keyword.
According to an embodiment of the present disclosure, the process of generating one or more reply words in operation S241 is a primary decoding process, and the one or more reply words are obtained by decoding the input vector.
According to an embodiment of the present disclosure, since the input of the third recurrent neural network includes a vector representation of a reply type, different reply words are obtained for the vector representations of different reply types to generate different reply sentences according to the different reply words.
According to an embodiment of the present disclosure, the operations S2411 to S2412 executed in a loop may also be executed by a copy mechanism neural network, for example, which is more inclined to generate words in the input sentence when generating the reply words. The problem that the generated result and the input effective information cannot be fused in the traditional method can be avoided based on the reply sentence obtained by the reply word obtained by the copy mechanism neural network, and therefore the generation effect of the reply sentence can be effectively improved.
Considering that for some named entities, such as names of people, it is difficult to accurately generate the input-mentioned words through a neural network, the probability that candidate reply words are copied from the input sentences and/or keywords is considered when determining the keywords in the above embodiment, and the reply words can be directly copied from the input sentences, so that the defect that the problem that the dictionary is not complete (out-of-vocabulary) in the existing dialogue system can be solved to some extent.
In operation S242, a reply sentence of the input sentence is generated according to the one or more reply words and the candidate reply value.
According to an embodiment of the present disclosure, the above-described generated reply sentence may be formed, for example, by combining the reply word generated in operation S241 with a candidate reply value obtained from the knowledge base query.
According to the embodiment of the present disclosure, when the reply sentences are generated according to the candidate reply values, if the number of the candidate reply values is different, the reply sentences obtained in operation S241 are different, and accordingly, the obtained reply sentences are also different, which is specifically as follows:
in the case that the number of candidate reply values is zero, the obtained reply word may be a default reply word such as "unknown" or "don't care", and the corresponding obtained reply sentence is "i don't know" or "do not find related information", etc.;
for the case where the number of candidate answer values is plural, the resultant answer words may be parameters related to the keywords such as "price", "color", "type", and the like, and the generated answer sentence may be, for example, "how much do you want the price range of the product? "or" what color you want ", etc., to request further query conditions from the user, in which case the generated reply sentence generally does not include the candidate reply value; then, according to an embodiment of the present disclosure, for example, the input of the third recurrent neural network further includes, for example, a candidate response value obtained by the query, and the generation of the response word may further be generated according to the plurality of candidate response values, so that the query condition obtained by the final response sentence request is not repeated with the previous query condition, so as to improve the efficiency of completing the user intention;
in the case where the number of candidate reply values is one, the obtained reply word may be a qualifier of the candidate reply word, such as "address", "telephone", etc., and the generated reply sentence may be a concatenation sentence of the reply word and the candidate reply value, such as "address is, etc.
According to the embodiment of the present disclosure, since the input of the third recurrent neural network includes the vector representation of the answer type when the answer sentence is generated, the answer sentence is generated according to the answer type, and thus, the efficiency of the dialog may be improved to some extent, so that the finally completed task conforms to the intention of the user.
FIG. 6 schematically illustrates a schematic diagram of a statement processing method according to an embodiment of the disclosure.
According to an embodiment of the present disclosure, the method described above with reference to fig. 2 to 5B may be implemented, for example, by using a neural network architecture from a sequence to a sequence model, using an input sentence as an input of the neural network architecture, and outputting a final reply sentence through one encoding and two decoding. Specifically, for each of the multiple rounds of dialog (as shown in the left diagram of fig. 6), the neural network framework first encodes the current input sentence (denoted by U3, e.g., "Tellme the address and the phone number multiplex"), the keyword (denoted by B2, e.g., "ltinf Italian"; "snap" < Req >/Req > ") in the previous round of dialog, and the reply sentence (denoted by R2, e.g.," NAME _ SLOT a left answering virtual tool ") in the previous round of dialog via a second recurrent neural network to obtain the vector to be processed in operation S210; then, a first decoding is performed through The first recurrent neural network to obtain one or more keywords (represented by B3, for example, "< Inf > Italian; snap [ < Req > ADDRESS; PHONE [ < Req >") of operation S220, then a Knowledge Base (Knowledge Base) is queried according to The obtained keywords to obtain The reply content in operation S230, and finally, a second decoding is performed through The third recurrent neural network to obtain a reply sentence of The input sentence (represented by R3, for example, "The ADDRESS is S SLOT and The PHONE number is PHONE SLOT", where "ADDRESS _ SLOT" and "PHONE SLOT _ SLOT" are obtained from The obtained reply content, according to The obtained reply content, The one or more keywords obtained by The first decoding, and The final hidden layer state of The first recurrent neural network implementing The first decoding).
According to an embodiment of the present disclosure, the sentence processing method of the present disclosure may be, for example, processing for an input sentence obtained by a task-oriented dialog system.
In summary, the sentence processing method according to the embodiment of the disclosure can obtain the keyword for determining the reply content only through the first recurrent neural network, and can effectively reduce the complexity of the model and thus improve the processing efficiency of the sentence, compared with a scheme in the prior art that a plurality of classifiers in a Belief tracker (Belief tracker) are required to respectively establish a model for each information domain and each request domain.
Fig. 7 schematically shows a block diagram of a sentence processing apparatus according to an embodiment of the present disclosure.
As shown in fig. 7, the sentence processing apparatus 700 includes a vector to be processed acquisition module 710, a keyword generation module 720, a reply content determination module 730, and a reply sentence generation module 740.
According to an embodiment of the present disclosure, the to-be-processed vector obtaining module 710 may, for example, perform operation S210 described with reference to fig. 2, which is not described herein again.
The keyword generation module 720 is configured to input the vector to be processed to the first recurrent neural network to generate one or more keywords, and according to an embodiment of the present disclosure, the keyword generation module 720 may, for example, perform operation S220 described with reference to fig. 2, which is not described herein again.
According to an embodiment of the present disclosure, the keyword generation module 720 may be configured to, for example, loop the following operations until a complete keyword or keywords are generated: updating the hidden layer state of the first recurrent neural network by the input of the first recurrent neural network to obtain a plurality of current candidate keywords; and determining the current keywords according to the output probabilities of the current multiple candidate keywords, wherein the output probability of each candidate keyword comprises: in the loop, a vector to be processed is used as the initial input of the first cyclic neural network, and the vector representation of the keyword obtained in the previous loop and the vector representation of the hidden layer state are used as the input of the first cyclic neural network in the next loop. According to an embodiment of the disclosure, the keyword generation module 720 may be configured to, for example, cyclically execute operations S221 to S222 described with reference to fig. 4, which are not described herein again.
The reply content determining module 730 is configured to determine the reply content corresponding to one or more keywords, and according to an embodiment of the present disclosure, the reply content determining module 730 may perform, for example, operation S230 described with reference to fig. 2, which is not described herein again.
The reply sentence generation module 740 is configured to generate a reply sentence of the input sentence based on the one or more keywords and the reply content, and according to the embodiment of the present disclosure, the reply sentence generation module 740 may, for example, perform operation S240 described with reference to fig. 2, which is not described herein again.
Fig. 8 schematically shows a structural block diagram of a to-be-processed vector acquisition module according to an embodiment of the present disclosure.
As shown in fig. 8, the to-be-processed vector obtaining module 710 may include, for example, a obtaining sub-module 711, a word vector generating sub-module 712, and a to-be-processed vector determining sub-module 713.
The obtaining sub-module 711 is configured to obtain a current input sentence, a previously generated reply sentence, and one or more keywords generated previously, where the previously generated reply sentence and the one or more keywords generated previously may take default values, for example, in a case that there is no previously generated reply sentence. According to an embodiment of the present disclosure, the obtaining sub-module 711 may be configured to perform operation S211 described with reference to fig. 3, for example, and is not described herein again.
The word vector generation sub-module 712 is configured to convert words in the current input sentence, the previously generated reply sentence, and the previously generated one or more keywords into word vectors. According to an embodiment of the present disclosure, the word vector generation sub-module 712 may be configured to perform operation S212 described with reference to fig. 3, for example, and is not described herein again.
The to-be-processed vector determining submodule 713 is configured to sequentially input the word vectors to the second recurrent neural network, and obtain a vector representation of a hidden layer state of the second recurrent neural network as the to-be-processed vector. According to an embodiment of the present disclosure, the to-be-processed vector determining submodule 713 may be configured to perform operation S213 described with reference to fig. 3, for example, and is not described herein again.
Fig. 9 schematically shows a structural diagram of a reply sentence generation module according to an embodiment of the present disclosure.
As shown in fig. 9, the above-mentioned reply sentence generation module 740 may include a reply word generation sub-module 741 and a reply sentence determination sub-module 742.
The reply word generation sub-module 741 is configured to input the vector representation of the one or more keywords, the vector representation of the final hidden layer state of the first recurrent neural network, and the vector representation of the reply type into the third recurrent neural network, and generate one or more reply words, where the reply contents include the vector representation of the reply type. According to an embodiment of the present disclosure, the reply word generation sub-module 741 may be configured to perform operation S241 described with reference to fig. 5A, for example, and is not described herein again.
According to an embodiment of the present disclosure, the reply word generation sub-module 741 may be specifically configured to loop the following operations until a complete one or more reply words are generated: updating the hidden layer state of the third recurrent neural network by the input of the third recurrent neural network to obtain a plurality of current candidate answer compound words; and determining the current reply word according to the output probability of the current multiple candidate reply words, wherein the output probability of each candidate reply word comprises the following steps: the sum of the probability of the candidate reply word generated by the third recurrent neural network and the probability of the candidate reply word copied from the input sentence. In the loop, vector representation of one or more keywords, vector representation of a final hidden layer state of the first cyclic neural network and vector representation of a reply type are used as initial input of a third cyclic neural network, and vector representation of a reply word and vector representation of a hidden layer state obtained in a previous loop are used as input of the third cyclic neural network in a next loop. According to an embodiment of the present disclosure, the reply word generation sub-module 741 may also be configured to, for example, cyclically execute operations S2411 to S2412 described with reference to fig. 5B, which are not described herein again.
The answer sentence determination sub-module 742 is configured to generate an answer sentence of the input sentence according to the one or more answer words and the candidate answer values, where the answer content includes the candidate answer values, and the vector representation of the answer type is related to the number of the candidate answer values. According to an embodiment of the present disclosure, the reply sentence determination sub-module 742 may be used to perform the operation S242 described with reference to fig. 5A, for example, and will not be described herein again.
Any number of modules, sub-modules, units, sub-units, or at least part of the functionality of any number thereof according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be implemented by being split into a plurality of modules. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in any other reasonable manner of hardware or firmware by integrating or packaging a circuit, or in any one of or a suitable combination of software, hardware, and firmware implementations. Alternatively, one or more of the modules, sub-modules, units, sub-units according to embodiments of the disclosure may be at least partially implemented as a computer program module, which when executed may perform the corresponding functions.
For example, any number of the to-be-processed vector acquisition module 710, the keyword generation module 720, the reply content determination module 730, the reply sentence generation module 740, the acquisition sub-module 711, the word vector generation sub-module 712, the to-be-processed vector determination sub-module 713, the reply word generation sub-module 741, and the reply sentence determination sub-module 742 may be combined and implemented in one module, or any one of them may be split into a plurality of modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to the embodiment of the present disclosure, at least one of the to-be-processed vector obtaining module 710, the keyword generating module 720, the reply content determining module 730, the reply sentence generating module 740, the obtaining sub-module 711, the word vector generating sub-module 712, the to-be-processed vector determining sub-module 713, the reply word generating sub-module 741, and the reply sentence determining sub-module 742 may be at least partially implemented as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or implemented by any one of three manners of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, at least one of the to-be-processed vector acquisition module 710, the keyword generation module 720, the reply content determination module 730, the reply sentence generation module 740, the acquisition sub-module 711, the word vector generation sub-module 712, the to-be-processed vector determination sub-module 713, the reply word generation sub-module 741, and the reply sentence determination sub-module 742 may be at least partially implemented as a computer program module that, when executed, may perform a corresponding function.
FIG. 10 schematically shows a block diagram of a statement processing system according to an embodiment of the disclosure. The computer system illustrated in FIG. 10 is only one example and should not impose any limitations on the scope of use or functionality of embodiments of the disclosure.
As shown in fig. 10, the sentence processing system 1000 according to the embodiment of the present disclosure includes a processor 1001 which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. Processor 1001 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 1001 may also include onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of a method flow according to embodiments of the present disclosure.
In the RAM 1003, various programs and data necessary for the operation of the system 1000 are stored. The processor 1001, ROM1002, and RAM 1003 are connected to each other by a bus 1004. The processor 1001 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM1002 and/or the RAM 1003. Note that the programs may also be stored in one or more memories other than the ROM1002 and the RAM 1003. The processor 1001 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to embodiments of the present disclosure, method flows according to embodiments of the present disclosure may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication part 1009 and/or installed from the removable medium 1011. The computer program performs the above-described functions defined in the system of the embodiment of the present disclosure when executed by the processor 1001. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
The present disclosure also provides a computer-readable medium, which may be embodied in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer readable medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to embodiments of the present disclosure, a computer readable medium may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer-readable signal medium may include a propagated data signal with computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical fiber cable, radio frequency signals, etc., or any suitable combination of the foregoing.
For example, according to embodiments of the present disclosure, a computer-readable medium may include ROM1002 and/or RAM 1003 and/or one or more memories other than ROM1002 and RAM 1003 as described above.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.
The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.
Claims (12)
1. A statement processing method, comprising:
acquiring a vector to be processed, wherein the vector to be processed comprises vector representation of an input statement;
inputting the vector to be processed into a first recurrent neural network to generate one or more keywords;
determining reply content corresponding to the one or more keywords; and
generating a reply sentence of the input sentence based on the one or more keywords and the reply content.
2. The method of claim 1, wherein obtaining a vector to be processed comprises:
acquiring the current input sentence, the reply sentence generated last time and one or more keywords generated last time;
converting words in the current input sentence, the reply sentence generated last time and one or more keywords generated last time into word vectors; and
orderly inputting the word vectors into a second recurrent neural network, acquiring vector representation of a hidden layer state of the second recurrent neural network as the vector to be processed,
wherein, in the case that there is no previously generated reply sentence, the previously generated reply sentence and the previously generated one or more keywords are default values.
3. The method of claim 1, wherein inputting the vector to be processed to a first recurrent neural network, generating one or more keywords comprises cyclically performing the following operations until the complete keyword or keywords are generated:
updating the hidden layer state of the first recurrent neural network by the input of the first recurrent neural network to obtain a plurality of current candidate keywords; and
determining the current keywords according to the output probabilities of the plurality of current candidate keywords, wherein the output probability of each candidate keyword comprises: the sum of the probability of the candidate keyword generated by the first recurrent neural network and the probability of the candidate keyword being copied from the input sentence,
in the above cycle, the vector to be processed is used as the initial input of the first recurrent neural network, and the vector representation of the keyword and the vector representation of the hidden layer state obtained in the previous cycle are used as the input of the first recurrent neural network in the next cycle.
4. The method of claim 1, wherein generating a reply sentence of the input sentence based on the one or more keywords and the reply content comprises:
inputting the vector representation of the one or more keywords, the vector representation of the hidden layer state of the first recurrent neural network and the vector representation of the answer type into a third recurrent neural network to generate one or more answer words; and
generating a reply sentence of the input sentence according to the one or more reply words and a candidate reply value,
wherein the reply content includes a vector representation of the reply type and the candidate reply values, the vector representation of the reply type being related to a number of the candidate reply values.
5. The method of claim 4, wherein generating one or more reply words comprises looping the following operations until the complete one or more reply words are generated:
updating the hidden layer state of the third recurrent neural network by using the input of the third recurrent neural network to obtain a plurality of current candidate answer compound words; and
determining the current reply word according to the current output probabilities of the multiple candidate reply words, wherein the output probability of each candidate reply word comprises the following steps: the sum of the probability of the candidate reply word generated by the third recurrent neural network and the probability of the candidate reply word being copied from the input sentence,
in the loop, vector representation of one or more keywords, vector representation of a final hidden layer state of the first loop neural network and vector representation of a reply type are used as initial input of the third loop neural network, and vector representation of a reply word and vector representation of a hidden layer state obtained in a previous loop are used as input of the third loop neural network in a next loop.
6. A sentence processing apparatus comprising:
the device comprises a to-be-processed vector acquisition module, a to-be-processed vector acquisition module and a to-be-processed vector processing module, wherein the to-be-processed vector acquisition module is used for acquiring a to-be-processed vector which comprises vector representation of an input statement;
the keyword generation module is used for inputting the vector to be processed into a first recurrent neural network to generate one or more keywords;
a reply content determination module for determining reply content corresponding to the one or more keywords; and
and the reply sentence generation module is used for generating the reply sentence of the input sentence based on the one or more keywords and the reply content.
7. The apparatus of claim 6, wherein the to-be-processed vector acquisition module comprises:
the obtaining submodule is used for obtaining the current input statement, the reply statement generated last time and one or more keywords generated last time;
the word vector generation submodule is used for converting words in the current input sentence, the reply sentence generated last time and one or more keywords generated last time into word vectors; and
a to-be-processed vector determination submodule for sequentially inputting the word vectors to a second recurrent neural network, acquiring vector representation of a hidden layer state of the second recurrent neural network as the to-be-processed vector,
wherein, in the case that there is no previously generated reply sentence, the previously generated reply sentence and the previously generated one or more keywords are default values.
8. The apparatus of claim 1, wherein the keyword generation module is configured to loop the following until the complete one or more keywords are generated:
updating the hidden layer state of the first recurrent neural network by the input of the first recurrent neural network to obtain a plurality of current candidate keywords; and
determining the current keywords according to the output probabilities of the plurality of current candidate keywords, wherein the output probability of each candidate keyword comprises: the sum of the probability of the candidate keyword generated by the first recurrent neural network and the probability of the candidate keyword being copied from the input sentence,
in the above cycle, the vector to be processed is used as the initial input of the first recurrent neural network, and the vector representation of the keyword and the vector representation of the hidden layer state obtained in the previous cycle are used as the input of the first recurrent neural network in the next cycle.
9. The apparatus of claim 6, wherein the reply sentence generation module comprises:
a reply word generation submodule, configured to input the vector representation of the one or more keywords, the vector representation of the final hidden layer state of the first recurrent neural network, and the vector representation of the reply type into a third recurrent neural network, and generate one or more reply words; and
a reply sentence determination submodule for generating a reply sentence of the input sentence according to the one or more reply words and the candidate reply value,
wherein the reply content includes a vector representation of the reply type and the candidate reply values, the vector representation of the reply type being related to a number of the candidate reply values.
10. The apparatus of claim 9, wherein the reply word generation submodule is configured to loop through the following operations until the complete one or more reply words are generated:
updating the hidden layer state of the third recurrent neural network by using the input of the third recurrent neural network to obtain a plurality of current candidate answer compound words; and
determining the current reply word according to the current output probabilities of the multiple candidate reply words, wherein the output probability of each candidate reply word comprises the following steps: the sum of the probability of the candidate reply word generated by the third recurrent neural network and the probability of the candidate reply word being copied from the input sentence,
in the loop, vector representation of one or more keywords, vector representation of a final hidden layer state of the first loop neural network and vector representation of a reply type are used as initial input of the third loop neural network, and vector representation of a reply word and vector representation of a hidden layer state obtained in a previous loop are used as input of the third loop neural network in a next loop.
11. A statement processing system comprising:
one or more processors;
a storage device for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-5.
12. A computer readable medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810851351.1A CN110851574A (en) | 2018-07-27 | 2018-07-27 | Statement processing method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810851351.1A CN110851574A (en) | 2018-07-27 | 2018-07-27 | Statement processing method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110851574A true CN110851574A (en) | 2020-02-28 |
Family
ID=69595538
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810851351.1A Pending CN110851574A (en) | 2018-07-27 | 2018-07-27 | Statement processing method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110851574A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111428014A (en) * | 2020-03-17 | 2020-07-17 | 北京香侬慧语科技有限责任公司 | Non-autoregressive conversational speech generation method and model based on maximum mutual information |
| WO2023137903A1 (en) * | 2022-01-22 | 2023-07-27 | 平安科技（深圳）有限公司 | Reply statement determination method and apparatus based on rough semantics, and electronic device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609092A (en) * | 2017-09-08 | 2018-01-19 | 北京百度网讯科技有限公司 | Intelligent response method and apparatus |
| CN107885756A (en) * | 2016-09-30 | 2018-04-06 | 华为技术有限公司 | Dialogue method, device and equipment based on deep learning |
| CN108038209A (en) * | 2017-12-18 | 2018-05-15 | 深圳前海微众银行股份有限公司 | Answer system of selection, device and computer-readable recording medium |
| CN108304364A (en) * | 2017-02-23 | 2018-07-20 | 腾讯科技（深圳）有限公司 | keyword extracting method and device |
-
2018
- 2018-07-27 CN CN201810851351.1A patent/CN110851574A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107885756A (en) * | 2016-09-30 | 2018-04-06 | 华为技术有限公司 | Dialogue method, device and equipment based on deep learning |
| CN108304364A (en) * | 2017-02-23 | 2018-07-20 | 腾讯科技（深圳）有限公司 | keyword extracting method and device |
| CN107609092A (en) * | 2017-09-08 | 2018-01-19 | 北京百度网讯科技有限公司 | Intelligent response method and apparatus |
| CN108038209A (en) * | 2017-12-18 | 2018-05-15 | 深圳前海微众银行股份有限公司 | Answer system of selection, device and computer-readable recording medium |
Non-Patent Citations (1)
| Title |
|---|
| WENQIANG LEI ET.AL: "Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures", 《PROCEEDINGS OF THE 56TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (LONG PAPERS)》, pages 1438 - 1442 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111428014A (en) * | 2020-03-17 | 2020-07-17 | 北京香侬慧语科技有限责任公司 | Non-autoregressive conversational speech generation method and model based on maximum mutual information |
| WO2023137903A1 (en) * | 2022-01-22 | 2023-07-27 | 平安科技（深圳）有限公司 | Reply statement determination method and apparatus based on rough semantics, and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10733983B2 (en) | Parameter collection and automatic dialog generation in dialog systems | |
| KR102532152B1 (en) | Multimodal content processing method, apparatus, device and storage medium | |
| US10854188B2 (en) | Synthesized voice selection for computational agents | |
| US10049667B2 (en) | Location-based conversational understanding | |
| CN112507099B (en) | Training method, device, equipment and storage medium of dialogue understanding model | |
| EP4109375A1 (en) | Multi computational agent performance of tasks | |
| JP2019503526A5 (en) | ||
| US10860289B2 (en) | Flexible voice-based information retrieval system for virtual assistant | |
| US11468881B2 (en) | Method and system for semantic intelligent task learning and adaptive execution | |
| WO2022095845A1 (en) | Dialogue understanding method and apparatus, readable medium, and electronic device | |
| CN115455161A (en) | Conversation processing method, conversation processing device, electronic equipment and storage medium | |
| CN109858045A (en) | Machine translation method and device | |
| US11403462B2 (en) | Streamlining dialog processing using integrated shared resources | |
| CN115908640A (en) | Method and device for generating image, readable medium and electronic equipment | |
| CN110851574A (en) | Statement processing method, device and system | |
| JP2021108095A (en) | Method for outputting information on analysis abnormality in speech comprehension | |
| CN114299955A (en) | Voice interaction method and device, electronic equipment and storage medium | |
| US11705113B2 (en) | Priority and context-based routing of speech processing | |
| US20230085458A1 (en) | Dialog data generating | |
| US11830497B2 (en) | Multi-domain intent handling with cross-domain contextual signals | |
| CN111046146A (en) | Method and apparatus for generating information | |
| US20230042838A1 (en) | Method for data processing, device, and storage medium | |
| US20230342553A1 (en) | Attribute and rating co-extraction | |
| CN117670275A (en) | Intelligent talent recommendation method, system, equipment and storage medium for enterprise | |
| CN117951257A (en) | Dialogue data processing method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |