WO2020155766A1

WO2020155766A1 - Method, device and apparatus for identification rejection in intention identification, and storage medium

Info

Publication number: WO2020155766A1
Application number: PCT/CN2019/118278
Authority: WO
Inventors: 许开河; 杨坤; 王少军
Original assignee: 平安科技（深圳）有限公司
Priority date: 2019-01-31
Filing date: 2019-11-14
Publication date: 2020-08-06
Also published as: CN109871446A; CN109871446B

Abstract

A method, device and apparatus for identification rejection in intention identification and a storage medium, relating to the technical field of artificial intelligence. The method comprises: obtaining input information to be identified (S1); inputting the input information into an intention identification model comprising a text classification model and a text similarity model, and obtaining by means of the intention identification model a classification category and a confidence score corresponding to the input information (S2); and determining whether the confidence score exceeds a preset threshold (S3); if yes, obtaining from a knowledge base knowledge point information corresponding to the classification category, and if not, rejecting the identification of the input information. The conditional probability obtained by means of the text classification model is corrected to obtain a confidence score, and the confidence score is used as the determination basis for rejecting the identification of the input information, thereby improving the accuracy of intention identification.

Description

Rejection method, device, equipment and storage medium in intention recognition

This application claims the priority rights of Chinese Patent Application No. 201910100204.5 filed on January 31, 2019, and the entire contents of the above cases are incorporated herein by reference.

Technical field

This application relates to the field of artificial intelligence technology, and in particular to a method, device, device, and storage medium for rejection in intention recognition.

Background technique

Intention recognition, that is, recognizing the intention of a behavior, is the most important part of a question answering robot. Intent recognition is often composed of two important directions. Intent recognition based on retrieval: Similar to a search engine, a robot retrieves its own knowledge base and returns the answer that best answers the user’s question. Intent recognition algorithm based on text classification: use knowledge points in the knowledge base to train a text classification model and use the text classification model to classify user questions to obtain knowledge points and return corresponding answers to the knowledge points. The text classification model based on the deep network tends to have higher accuracy than the retrieval model's question and answer, but the text classification model cannot correctly identify problems outside the knowledge base, and the user's classification model will force a classification for each problem. The final output layer of the existing text classification model often uses softmax to score the probability that the sample belongs to each category. First calculate the score that the sample belongs to each category, and then divide this score by the total score to get the probability of belonging to that category. The probability obtained in this way is actually a conditional probability: when the sample belongs to the knowledge base, the probability that it belongs to a certain class; when the sample does not belong to the knowledge base, this probability is completely random. Because the sample may be different from every knowledge point in the knowledge base, and the score belonging to each knowledge point is very low, softmax is equivalent to normalizing these small numbers to between 0-1. Therefore, it is entirely possible that a certain category is enlarged, and a relatively large probability is output, resulting in a lower classification accuracy of the text classification model, and a lower accuracy of intent recognition.

Summary of the invention

This application provides a method, device, equipment, and storage medium for rejecting intent recognition in order to solve the problem of low accuracy of intent recognition in the prior art.

In order to achieve the above objectives, the first aspect of this application is to provide a method of rejection in intention identification, including:

Obtain the input information to be recognized; input the input information into the trained intention recognition model, and obtain the classification category and the confidence score corresponding to the input information through the intention recognition model; determine whether the confidence score exceeds The preset threshold value, if it exceeds the preset threshold value, the knowledge point information corresponding to the classification category is obtained from the knowledge base, if the preset threshold value is not exceeded, the input information is rejected; wherein, the intention recognition model includes text A classification model and a text similarity model, the classification category corresponding to the input information and the conditional probability that the input information belongs to the classification category are obtained through the text classification model, and the text similarity model and the conditional probability are obtained The confidence score.

In order to achieve the above objective, the second aspect of the present invention is to provide a rejection device in intention recognition, including:

The input information obtaining module is used to obtain the input information to be recognized; the recognition module is used to input the input information into a trained intent recognition model for recognition, wherein the intent recognition model includes a text classification model and text similarity A model, obtaining a classification category corresponding to the input information and a conditional probability of the input information belonging to the classification category through the text classification model, and obtaining the confidence score through the text similarity model and the conditional probability; Confidence acquisition module, used to obtain the classification category and confidence score corresponding to the input information through the intention recognition model; judgment module, used to determine whether the confidence score exceeds a preset threshold, and if it exceeds the preset threshold , The knowledge point information corresponding to the classification category is obtained from the knowledge base, and if it does not exceed the preset threshold, the input information is rejected.

In order to achieve the above objective, the third aspect of the present application is to provide an electronic device, the electronic device includes a processor and a memory, the memory includes an intention recognition rejection program, the rejection program is The processor implements the rejection method in intention recognition as described above during execution.

In order to achieve the above objective, the fourth aspect of the present application is to provide a computer non-volatile readable storage medium, the computer non-volatile readable storage medium includes an intention recognition rejection program, the rejection When the recognition program is executed by the processor, the recognition rejection method in the intention recognition as described above is realized.

Compared with the prior art, this application has the following advantages and beneficial effects:

The intent recognition model of this application includes a text classification model and a text similarity model. The confidence score is obtained by modifying the conditional probability obtained by the text classification model, and the confidence score is used to determine whether to reject the input information, which improves the intention recognition Accuracy.

Description of the drawings

Fig. 1 is a schematic flow diagram of the rejection method in the intention recognition described in this application;

Figure 2 is a comparison diagram of the problem identification results in the knowledge base between the intention recognition model and the existing text classification model in this application;

Figure 3 is a comparison diagram of the recognition results of the intent recognition model in the application and the existing text classification model for problems outside the knowledge base;

Fig. 4 is a schematic diagram of modules of the rejection device in the intention recognition in this application.

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the drawings.

detailed description

The embodiments described in this application will be described below with reference to the drawings. Those of ordinary skill in the art may realize that, without departing from the spirit and scope of the present application, the described embodiments can be modified in various different ways or combinations thereof. Therefore, the drawings and description are illustrative in nature, and are only used to explain the application, rather than to limit the protection scope of the claims. In addition, in this specification, the drawings are not drawn to scale, and the same reference numerals denote the same parts.

The rejection method in the intention recognition described in this application is applied to a question answering robot. For a certain question of the user, the intention recognition model will output a classification result and a score. The classification result represents the corresponding knowledge point information in the knowledge base. The score represents the confidence score. For the case with a low confidence score, it can be rejected, so as to identify that the input question is outside the knowledge base. The knowledge base is composed of one or more knowledge point information, and each knowledge point information corresponds to a specific solution plan for the problem. After receiving the user’s question, the knowledge point information corresponding to the question can be fed back to the user, or the question Rejected.

Fig. 1 is a schematic diagram of the process of the rejection method in the intention recognition described in this application. As shown in Fig. 1, the rejection method includes:

Step S1: Obtain input information to be recognized;

Step S2: Input the input information into the trained intention recognition model, and obtain the classification category and the confidence score corresponding to the input information through the intention recognition model;

Step S3: Determine whether the confidence score exceeds a preset threshold, if it exceeds the preset threshold, obtain knowledge point information corresponding to the classification category from the knowledge base, and if it does not exceed the preset threshold, refuse to recognize the input information;

Wherein, the intent recognition model includes a text classification model and a text similarity model. The classification category corresponding to the input information and the conditional probability of the input information belonging to the classification category are obtained through the text classification model, and the text The similarity model and the conditional probability obtain the confidence score.

This application uses the text similarity model in the intention recognition model to modify the conditional probability obtained by the text classification model to obtain the confidence score, and use the confidence score as the basis for judgment to reject the input information, which improves the accuracy of the intention recognition .

In this application, the input information to be recognized can be directly input into the intent recognition model after processing. Further, the input information to be recognized can be directly input into the text classification model to obtain the classification category, or directly into the similarity model Get the similarity with the knowledge point information. Preferably, the step of obtaining the input information to be recognized includes:

Acquire voice information to be recognized; convert the acquired voice information into text information in a preset format; process the text information to obtain input information to be recognized. Wherein, obtaining the voice information to be recognized may be a user's voice command or chat voice. Further, processing the text information includes denoising processing and word segmentation processing on the text information, etc., through denoising processing, meaningless phrases can be removed without affecting the true meaning of the input information, and the text information is processed through word segmentation. Perform word segmentation, and further mark the part of speech of each phrase and identify named entities.

In this application, the input information to be recognized can be sentences or phrases, etc. The input information to be recognized includes the expression of the question that the user wants to consult, for example, the question expression is "How do I apply for a credit card with online banking?", corresponding knowledge The point information is "credit card application" and so on. Further, the input information includes user information. The user information includes but is not limited to the user’s age, gender, identity, occupation, region, hometown and other information, so as to facilitate the preference clustering of the user’s input information through the user information and identify the user The tendency of interest.

The present application obtains the confidence score according to the output result of the text similarity model. In an optional embodiment of the present application, the step of obtaining the confidence score through the text similarity model and the conditional probability includes: The input information and the knowledge point information in the knowledge base are input into the text similarity model; the similarity between the input information and the knowledge point information in the knowledge base is obtained through the text similarity model; The maximum similarity is selected among the multiple similarities; the maximum similarity is multiplied by the conditional probability to obtain the confidence score.

As shown in the following formula:

In the formula, x represents the input information; C _i represents the i-th type of knowledge point information in the knowledge base; C represents the knowledge base; Score (x∈C _i ) represents the confidence that the input information x belongs to the i-th type knowledge point in the knowledge base Score; P(x∈C _i ,x∈C) represents the probability that the input information x is within the scope of the knowledge base and belongs to the i-th type of knowledge point information;

Indicates the probability that the input information x is not within the scope of the knowledge base and belongs to the i-th type of knowledge point information, generally 0; P(x∈C _i |x∈C) means that the input information x is within the scope of the knowledge base and belongs to The conditional probability of the i-th type of knowledge point information is output by the text classification model, which can be calculated by Bayesian formula expansion joint probability; j represents the index of the knowledge point information category in the knowledge base; P(x∈C) represents the input information The probability of belonging to the knowledge base; sim(x, C _j ) represents the similarity between the input information x and the j-th type of knowledge point information in the knowledge base. If the input information x is very similar to any knowledge point information in the knowledge base, it is considered The input information x belongs to the knowledge base, so the maximum similarity is taken to calculate the confidence score.

In an embodiment of the present application, the preset threshold of confidence is set to a level, for example, a confidence score of 0.9 is set as a first-level threshold, a confidence score of 0.8 is set as a second-level threshold, and the confidence score is 0.6 is set as the three-level threshold, and the confidence score is set to 0.4 as the four-level threshold; when the intent recognition result is obtained based on the confidence score, one or more knowledge corresponding to the input information is obtained according to the threshold level to which the confidence score belongs Point information. Specifically, the input information and the knowledge point information in the knowledge base are input into the text similarity model, and the similarity between the input information and the knowledge point information in the knowledge base is obtained through the text similarity model. Degree, arrange the obtained similarities in descending order, select the top-ranked preset similarities, and obtain the corresponding preset confidence scores, according to actual needs , The knowledge point information corresponding to the confidence score exceeding a preset level threshold can be selected as the knowledge point information corresponding to the classification category and fed back to the user, and if the maximum value of the multiple confidence scores is lower than the set For example, the confidence scores obtained by the intent recognition model are 0.95, 0.85, and 0.5, respectively. If the first-level threshold is selected, only knowledge points with a confidence score of 0.95 will be fed back Information, if the secondary threshold is selected, the knowledge point information corresponding to the confidence scores of 0.95 and 0.8 can be fed back for the user's reference. If the confidence scores obtained by the intention recognition model are 0.38, 0.3, and 0.25, respectively, and the maximum confidence score is 0.38, which is lower than the set four-level threshold, the corresponding input information is rejected.

Assuming that the classification algorithm in the text classification model is trustworthy, if the input information x belongs to the knowledge base, the input information must be classified into the knowledge point information category most similar to the input information through the text classification model. Preferably, the step of obtaining the confidence score through the text similarity model and the conditional probability includes: inputting the input information and knowledge point information corresponding to the classification category in the knowledge base into the text similarity In the model; obtain the similarity between the input information and the knowledge point information corresponding to the classification category through the text similarity model; multiply the conditional probability and the similarity obtained by the text similarity model to obtain The confidence score.

As shown in the following formula:

In the formula, x represents the input information, C _i represents the i-th type of knowledge point information in the knowledge base, C represents the knowledge base, and Score (x∈C _i ) represents the confidence that the input information x belongs to the i-th type of knowledge point information in the knowledge base Score, P(x∈C _i ,x∈C) represents the probability that the input information x is within the scope of the knowledge base and belongs to the i-th type of knowledge point information;

Indicates the probability that the input information x is not within the scope of the knowledge base and belongs to the i-th type of knowledge point information, generally 0; P(x∈C _i |x∈C) means that the input information x is within the scope of the knowledge base and belongs to The conditional probability of the i-th type of knowledge point information is output through the text classification model, P(x∈C) represents the probability that the input information belongs to the knowledge base; sim(x,C _i ) represents the difference between the input information x and the i-th type of knowledge point information Similarity.

By first using the text classification model to obtain a classification result, obtain the classification category corresponding to the input information, and then use this classification result to calculate the text similarity, and obtain the confidence score, which greatly reduces the number of text similarity matches and improves the calculation efficiency. To determine whether an input information belongs to the knowledge base, it is no longer necessary to traversely calculate the similarity between the input information and each knowledge point information in the knowledge base.

The text classification model is used to classify input information (which can be a sentence or a phrase, etc.), and output classification categories and corresponding scores. Preferably, the text classification model includes: an input layer, an embedding layer, a convolutional layer, a pooling layer, a normalization layer, and an output layer. The input information is input to the input layer, and the input information is converted through the embedding layer Is a word vector matrix, convolution operation is performed through the convolution layer, pooling operation is performed through the pooling layer, and the score of the input information belonging to each category is normalized through the normalization layer, and through the output layer The classification category corresponding to the input information and the score of the input information belonging to the classification category are output. By obtaining the score of the input information belonging to each category, and then dividing this score by the total score to obtain the probability that the input information belongs to the category category, as shown in the following formula:

Where x is the input information, C _i is the i-th type of knowledge point information in the knowledge base, s is the score, P(x∈C _i ) is the probability that the input information x belongs to the i-th type of knowledge point information in the knowledge base, s (x∈C _i ) is the score of the input information x belonging to the i-th type of knowledge point information in the knowledge base, j is the index of the knowledge point information category in the knowledge base, and n is the total number of knowledge point information categories in the knowledge base.

In this application, the text classification model can use the cnn network structure model or the dnn network structure model.

In an embodiment of the present application, the text similarity model adopts a network model based on a twin network, which includes two parallel identical neural networks. The input information and the knowledge point information in the knowledge base are each input into a neural network. A neural network transforms the input information into a first vector, and transforms the knowledge point information into a second vector, and obtains the similarity between the input information and the knowledge point information by calculating the similarity between the first vector and the second vector. Output. Through the text similarity model, the similarity between the input information and the knowledge point information in the knowledge base can be obtained separately, or only the similarity between the input information and the knowledge point information corresponding to the classification category output by the text classification model can be obtained.

Further, the similarity between the first vector and the second vector is calculated by the following formula:

In the formula, Y ₁ is the first vector, Y ₂ is the second vector, and sim(Y ₁ , Y ₂ ) is the similarity between the first vector and the second vector.

By calculating the similarity between the first vector and the second vector to characterize the similarity between the input information and the knowledge point information, determine the possibility of the knowledge point information corresponding to the input information in the knowledge base

The parameters of the two neural networks in the text similarity model are the same. The neural network may be an RNN neural network, a CNN neural network, an LSTM neural network, etc. The application is preferably a bidirectional LSTM neural network.

The knowledge point information in the knowledge base is used as the training sample to train the text similarity model. Each training sample includes two knowledge point information, and labels the training sample. If the semantics of the two knowledge point information of the training sample are the same, the label is 1; if they are inconsistent, the label is 0. According to the similarity of the two knowledge points, the training samples are divided into positive samples and negative samples. A positive sample indicates that the information of the two knowledge points is similar, and the corresponding label is 1, and a negative sample indicates that the information of the two knowledge points is not similar, and the corresponding label Is 0. For example, among the multiple knowledge point information in the knowledge base, a standard question is matched with multiple extended questions. The matched standard question and the extended question are similar. The positive sample includes a standard question and the matched extended question. Negative samples include a standard question and an extended question that does not match it or another standard question. The accuracy of the text similarity model is improved by dividing the positive sample and the negative sample.

This application can use existing training methods to train the parameters of the twin network, which is not limited in this application.

Figure 2 is a comparison diagram of the problem identification results in the knowledge base between the intent recognition model in the application and the existing text classification model. As shown in Figure 2, the problem recognition in the knowledge base is treated by the intention recognition model in this application The score distribution of the input information obtained by the recognized input information processing that belongs to a certain knowledge point information is not much different from the score distribution obtained by the existing text classification model. Figure 3 is a comparison diagram of the intent recognition model in this application and the existing text classification model for the recognition of problems outside the knowledge base. As shown in Figure 3, the problem recognition outside the knowledge base is obtained through the existing text classification model The score is generally high, and the distribution of the score obtained through the intention recognition model of the present application is generally low, so as to reject the recognition based on the comparison of the score with the preset threshold, thereby improving the accuracy of the intention recognition. The abscissas in Figure 2 and Figure 3 both represent the score of the input information belonging to a certain classification category, and the ordinate represents the number of samples of the input model. The existing model in the figure refers to the text classification model used in the existing intention recognition .

The rejection method in the intention recognition described in this application is applied to electronic devices, which may be terminal devices such as televisions, smart phones, tablet computers, and computers.

The electronic device includes: a processor and a memory, the memory is used to store the rejection program in the intention recognition, and the processor executes the rejection program in the intention recognition to implement the following rejection method in the intention recognition:

The electronic device also includes a network interface, a communication bus, and the like. Among them, the network interface may include a standard wired interface and a wireless interface, and the communication bus is used to realize the connection and communication between various components.

The memory includes at least one type of readable storage medium, which can be a non-volatile storage medium such as a flash memory, a hard disk, an optical disc, or a plug-in hard disk, etc., and is not limited to this, and can be stored in a non-transitory manner Any device that provides instructions or software and any associated data files to the processor to enable the processor to execute the instructions or software program. In this application, the software program stored in the memory includes the rejection program in the intention recognition, and can provide the rejection program in the intention recognition to the processor, so that the processor can execute the rejection program in the intention recognition to realize the intention Rejection method in recognition.

The processor may be a central processing unit, a microprocessor, or other data processing chips, etc., and may run a program stored in the memory, for example, the recognition rejection program in the intention recognition in this application.

The electronic device may also include a display, which may also be called a display screen or a display unit. In some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an organic light-emitting diode (OLED) touch device, and the like. The display is used to display the information processed in the electronic device and to display the visual work interface, including input information and output information through the intent recognition model.

The electronic device may also include a user interface, and the user interface may include an input unit (such as a keyboard), a voice output device (such as a stereo, earphone), and the like.

It should be noted that the specific implementation of the electronic device of the present application is substantially the same as the specific implementation of the rejection method in the aforementioned intention recognition, and will not be repeated here.

Fig. 4 is a schematic diagram of the module of the recognition rejection device in the intention recognition in this application. As shown in Fig. 4, the recognition rejection device includes: an input information acquisition module 1 for acquiring input information to be recognized; an identification module 2 The input information is input into a trained intent recognition model for recognition, where the intent recognition model includes a text classification model and a text similarity model, and the classification category and classification corresponding to the input information are obtained through the text classification model. The input information belongs to the conditional probability of the classification category, and the confidence score is obtained through the text similarity model and the conditional probability; the confidence obtaining module 3 is configured to obtain the The classification category and the confidence score corresponding to the input information; the judgment module 4 is used to judge whether the confidence score exceeds a preset threshold, and if it exceeds the preset threshold, obtain knowledge point information corresponding to the classification category from the knowledge base If it does not exceed the preset threshold, refuse to recognize the input information.

In this application, the input information to be recognized can be directly input into the intent recognition model after processing. Further, the input information to be recognized can be directly input into the text classification model to obtain the classification category, or directly into the similarity model Get the similarity with the knowledge point information. Preferably, the recognition rejection device obtains the input information to be recognized through the following steps:

The recognition rejection device of the present application includes a confidence acquisition module, which acquires a confidence score according to the output result of the text similarity model. In an optional embodiment of the present application, the confidence acquisition module includes: a first information input unit , Input the input information and the knowledge point information in the knowledge base into the text similarity model; the first similarity obtaining unit obtains the input information and each of the knowledge points in the knowledge base through the text similarity model. The similarity of the knowledge point information; the selecting unit selects the maximum similarity from the obtained multiple similarities; the first confidence calculating unit multiplies the maximum similarity by the conditional probability to obtain the confidence score.

As shown in the following formula:

In the formula, x represents the input information; C _i represents the i-th type of knowledge point information in the knowledge base; C represents the knowledge base; Score(x∈C _i ) represents the confidence that the input information x belongs to the i-th type knowledge point information in the knowledge base Score; P(x∈C _i ,x∈C) represents the probability that the input information x is within the scope of the knowledge base and belongs to the i-th type of knowledge point information;

Assuming that the classification algorithm in the text classification model is trustworthy, if the input information x belongs to the knowledge base, the input information must be classified into the knowledge point information category most similar to the input information through the text classification model. Preferably, the confidence degree acquisition module includes: a second information input unit, which inputs the input information and knowledge point information corresponding to the classification category in the knowledge base into the text similarity model; and the second similarity degree acquisition Unit to obtain the similarity between the input information and the knowledge point information corresponding to the classification category through the text similarity model; a second confidence calculation unit to compare the conditional probability with the all obtained by the text similarity model The similarity is multiplied to obtain the confidence score.

As shown in the following formula:

In an embodiment of the present application, the text similarity model adopts a network model based on a twin network, which includes two parallel identical neural networks. The input information and the knowledge point information in the knowledge base are each input into a neural network. A neural network transforms the input information into a first vector, and transforms the knowledge point information into a second vector, and obtains the similarity between the input information and the knowledge point information by calculating the similarity between the first vector and the second vector. Output. Through the text similarity model, the similarity between the input information and the knowledge point information in the knowledge base can be obtained separately, or only the similarity between the input information and the knowledge point information corresponding to the classification category output by the text classification model can be obtained. Wherein, the method for obtaining the similarity between the first vector and the second vector is the same as the method for obtaining the similarity in the rejection method in the intention recognition, which will not be repeated here.

The knowledge point information in the knowledge base is used as the training sample to train the text similarity model. The training method of the text similarity model in the rejection method in intention recognition is roughly the same, so I won't repeat it here.

It should be noted that the rejection device in the intention recognition of this application is substantially the same as the rejection method and the specific implementation of the electronic device in the intention recognition mentioned above, and will not be repeated here.

In other embodiments, the rejection program in the intention recognition can also be divided into one or more modules, and the one or more modules are stored in the memory and executed by the processor to complete the application. The module referred to in this application refers to a series of computer program instruction segments that can complete specific functions. For example, the rejection procedure in the intention recognition can be divided into: the input information acquisition module 1, the recognition module 2, the confidence acquisition module 3, and the judgment module 4. The functions or operation steps implemented by the above modules are all similar to the above, and will not be detailed here.

In an embodiment of the present application, the computer non-volatile readable storage medium may be any tangible medium that contains or stores a program or instruction, the program can be executed, and the stored program instructs relevant hardware to implement corresponding functions. For example, the computer non-volatile readable storage medium may be a computer disk, hard disk, random access memory, read-only memory, etc. The application is not limited to this, and it can be any device that stores instructions or software and any related data files or data structures in a non-transitory manner and can be provided to the processor to enable the processor to execute the programs or instructions therein. The computer non-volatile readable storage medium includes a rejection program in intention recognition, and when the rejection program in intention recognition is executed by a processor, the aforementioned rejection method in intention recognition is implemented. This will not be repeated here.

The specific implementation of the computer non-volatile readable storage medium of the present application is substantially the same as the specific implementation of the rejection method, device and electronic device in the above-mentioned intention recognition, and will not be repeated here.

It should be noted that in this article, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements not only includes those elements, It also includes other elements not explicitly listed, or elements inherent to the process, device, article, or method. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, device, article, or method that includes the element.

The serial numbers of the foregoing embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. Through the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, hardware can also be used, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , Magnetic disk, optical disk), including several instructions to make a terminal device (can be a mobile phone, a computer, a server, or a network device, etc.) execute the method described in each embodiment of the present application.

Claims

A method of refusal in intention recognition, applied to electronic equipment, characterized in that it includes:

Obtain the input information to be recognized;

Inputting the input information into an intention recognition model obtained through training, and obtaining a classification category and a confidence score corresponding to the input information through the intention recognition model;

Determine whether the confidence score exceeds a preset threshold, if it exceeds the preset threshold, obtain knowledge point information corresponding to the classification category from the knowledge base, and if it does not exceed the preset threshold, refuse to recognize the input information;

Wherein, the intent recognition model includes a text classification model and a text similarity model. The classification category corresponding to the input information and the conditional probability of the input information belonging to the classification category are obtained through the text classification model, and the text The similarity model and the conditional probability obtain the confidence score.
The method of rejecting recognition in intention recognition according to claim 1, wherein the step of obtaining the confidence score through the text similarity model and the conditional probability comprises:

Input the input information and knowledge point information in the knowledge base into the text similarity model;

Obtaining the similarity between the input information and the knowledge point information in the knowledge base through the text similarity model;

Select the maximum similarity from the obtained multiple similarities;

The maximum similarity is multiplied by the conditional probability to obtain the confidence score.
The method of rejecting recognition in intention recognition according to claim 1, wherein the step of obtaining the confidence score through the text similarity model and the conditional probability comprises:

Inputting the input information and knowledge point information corresponding to the classification category in the knowledge base into the text similarity model;

Obtaining the similarity between the input information and the knowledge point information corresponding to the classification category through the text similarity model;

The conditional probability is multiplied by the similarity obtained by the text similarity model to obtain the confidence score.
The method for rejecting recognition in intention recognition according to claim 1, wherein obtaining knowledge point information corresponding to the classification category from the knowledge base comprises:

Set the level of the preset threshold of confidence;

Acquire one or more knowledge point information corresponding to the classification category according to the threshold level to which the confidence score belongs.
The method for rejecting recognition in intention recognition according to claim 4, wherein the step of obtaining the confidence score through the text similarity model and the conditional probability comprises:

Input the input information and knowledge point information in the knowledge base into the text similarity model;

Obtaining the similarity between the input information and the knowledge point information in the knowledge base through the text similarity model;

Arrange the acquired similarities in descending order, select the top preset similarities, and obtain the corresponding preset confidence scores;

The steps of obtaining knowledge point information corresponding to the classification category from the knowledge base include:

From a predetermined number of confidence scores, one or more knowledge point information corresponding to a confidence score exceeding a preset level threshold is determined as the knowledge point information corresponding to the classification category.
The method for rejecting recognition in intention recognition according to claim 1, wherein the text similarity model adopts a network model based on a twin network, including two parallel identical neural networks, which combine the input information and the knowledge in the knowledge base. Each point information is input into a neural network. The input information is converted into a first vector through two neural networks, and the knowledge point information is converted into a second vector. By calculating the first vector and the second vector, The similarity of the vector obtains and outputs the similarity of the input information and the knowledge point information.
The method for rejecting recognition in intention recognition according to claim 6, wherein the similarity between the first vector and the second vector is calculated by the following formula:

In the formula, Y 1 is the first vector, Y 2 is the second vector, and sim(Y 1 , Y 2 ) is the similarity between the first vector and the second vector.
The method for rejecting recognition in intention recognition according to claim 6, wherein the neural network is one of RNN neural network, CNN neural network, and LSTM neural network.
The method of rejecting recognition in intention recognition according to claim 1, wherein the step of obtaining the input information to be recognized comprises:

Obtain the voice information to be recognized;

Convert the acquired voice information into text information in a preset format;

The text information is processed to obtain input information to be recognized.
The method for rejecting recognition in intention recognition according to claim 9, wherein processing the text information comprises: denoising processing and word segmentation processing on the text information.
The method for rejecting intent recognition according to claim 1, wherein the text classification model includes: an input layer, an embedding layer, a convolutional layer, a pooling layer, a normalization layer, and an output layer. The input information is input to the input layer, the input information is converted into a word vector matrix through the embedding layer, the convolution operation is performed through the convolution layer, the pooling operation is performed through the pooling layer, and the input information belongs to the normalization layer. The score of each classification is normalized, and the classification category corresponding to the input information and the conditional probability that the input information belongs to the classification category are output through the output layer.
The method of rejecting recognition in intention recognition according to claim 11, wherein the conditional probability that the input information belongs to the classification category is obtained by the following formula:

Where x is the input information, C i is the i-th type of knowledge point information in the knowledge base, s is the score, P(x∈C i ) is the probability that the input information x belongs to the i-th type of knowledge point information in the knowledge base, s (x∈C i ) is the score of the input information x belonging to the i-th type of knowledge point information in the knowledge base, j is the index of the knowledge point information category in the knowledge base, and n is the total number of knowledge point information categories in the knowledge base.
A rejection device in intention recognition, which is characterized in that it includes:

The input information obtaining module is used to obtain the input information to be recognized;

The recognition module is used to input the input information into a trained intent recognition model for recognition, wherein the intent recognition model includes a text classification model and a text similarity model, and the corresponding input information is obtained through the text classification model The classification category of and the conditional probability that the input information belongs to the classification category, and the confidence score is obtained through the text similarity model and the conditional probability;

A confidence degree acquisition module, configured to acquire a classification category and a confidence score corresponding to the input information through the intention recognition model;

The judgment module is used to judge whether the confidence score exceeds a preset threshold, if it exceeds the preset threshold, obtain the knowledge point information corresponding to the classification category from the knowledge base, and if it does not exceed the preset threshold, refuse to identify the述input information.
The recognition rejection device in intention recognition according to claim 13, wherein the confidence level acquisition module comprises: a first information input unit, configured to input the input information and knowledge point information in the knowledge base into the place In the text similarity model; the first similarity acquisition unit is used to obtain the similarity between the input information and the knowledge point information in the knowledge base through the text similarity model; the selection unit is used to obtain The maximum similarity is selected among the multiple similarities; the first confidence calculation unit is configured to multiply the maximum similarity and the conditional probability to obtain the confidence score.
The device for rejecting recognition in intention recognition according to claim 13, wherein the confidence acquisition module comprises: a second information input unit, configured to correspond to the classification category in the input information and the knowledge base The knowledge point information of is input into the text similarity model; a second similarity obtaining unit is used to obtain the similarity between the input information and the knowledge point information corresponding to the classification category through the text similarity model; second The confidence calculation unit is configured to multiply the conditional probability and the similarity obtained by the text similarity model to obtain the confidence score.
The recognition rejection device in intention recognition according to claim 13, characterized in that the text similarity model adopts a network model based on a twin network, including two parallel identical neural networks, which combine the input information and the knowledge in the knowledge base. Each point information is input into a neural network. The input information is converted into a first vector through two neural networks, and the knowledge point information is converted into a second vector. By calculating the first vector and the second vector, The similarity of the vector obtains and outputs the similarity of the input information and the knowledge point information.
The recognition rejection device in intent recognition according to claim 13, wherein the text classification model comprises: an input layer, an embedding layer, a convolutional layer, a pooling layer, a normalization layer, and an output layer. The input information is input to the input layer, the input information is converted into a word vector matrix through the embedding layer, the convolution operation is performed through the convolution layer, the pooling operation is performed through the pooling layer, and the input information belongs to the normalization layer The score of each classification is normalized, and the classification category corresponding to the input information and the conditional probability that the input information belongs to the classification category are output through the output layer.
The method for rejecting recognition in intention recognition according to claim 17, wherein the conditional probability that the input information belongs to the classification category is obtained by the following formula:

Where x is the input information, C i is the i-th type of knowledge point information in the knowledge base, s is the score, P(x∈C i ) is the probability that the input information x belongs to the i-th type of knowledge point information in the knowledge base, s (x∈C i ) is the score of the input information x belonging to the i-th type of knowledge point information in the knowledge base, j is the index of the knowledge point information category in the knowledge base, and n is the total number of knowledge point information categories in the knowledge base.
An electronic device, characterized in that the electronic device comprises: a processor and a memory, the memory includes an intent recognition rejection program, and when the rejection program is executed by the processor, it realizes the following claims 1 to 12. The rejection method in intention recognition described in any one of 12.
A computer non-volatile readable storage medium, characterized in that the computer non-volatile readable storage medium includes an intent recognition rejection program, when the rejection program is executed by a processor, The method of rejecting recognition in intention recognition according to any one of claims 1 to 12.