CN112530415B - Negative reply recognition model acquisition and negative reply recognition method and device - Google Patents
Negative reply recognition model acquisition and negative reply recognition method and device Download PDFInfo
- Publication number
- CN112530415B CN112530415B CN202110181108.5A CN202110181108A CN112530415B CN 112530415 B CN112530415 B CN 112530415B CN 202110181108 A CN202110181108 A CN 202110181108A CN 112530415 B CN112530415 B CN 112530415B
- Authority
- CN
- China
- Prior art keywords
- negative
- reply
- recognition
- recognition model
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000012549 training Methods 0.000 claims abstract description 83
- 238000012545 processing Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 11
- 238000005070 sampling Methods 0.000 claims description 7
- 239000000126 substance Substances 0.000 claims description 2
- 238000013473 artificial intelligence Methods 0.000 abstract description 8
- 238000003058 natural language processing Methods 0.000 abstract description 5
- 238000013135 deep learning Methods 0.000 abstract description 4
- 238000005516 engineering process Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 13
- 238000004891 communication Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 8
- 238000002372 labelling Methods 0.000 description 8
- 238000013145 classification model Methods 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000007935 neutral effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/16—Speech classification or search using artificial neural networks
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/18—Speech classification or search using natural language modelling
Landscapes
- Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Machine Translation (AREA)
Abstract
The utility model discloses a negative reply recognition model acquisition and negative reply recognition method and device, relating to the artificial intelligence fields of natural language processing, deep learning, intelligent voice and the like, wherein the negative reply recognition model acquisition method can comprise the following steps: obtaining a semantic representation model through pre-training; training a semantic representation model by using the obtained weakly supervised corpus to obtain a first recognition model; training the first recognition model by using the obtained supervision corpus to obtain a second recognition model; and taking the second recognition model as a required negative reply recognition model so as to perform negative reply recognition on the reply to be processed by using the negative reply recognition model. By applying the scheme disclosed by the invention, the accuracy of the identification result can be improved, the recall rate of negative reply can be improved, and the like.
Description
Technical Field
The present disclosure relates to the field of artificial intelligence technologies, and in particular, to a negative-going answer recognition model acquisition and negative-going answer recognition method and device in the fields of natural language processing, deep learning, and intelligent speech.
Background
The negative reply recognition technology in chatting is a method for effectively intervening bad dialogue replies generated by an open domain dialogue system.
The current dialog system mainly has two types, retrieval and generation respectively. The retrieval formula needs to construct a large-scale dialogue corpus and is realized by using the ways of intention understanding, retrieval, recall and the like, and the generation formula mainly utilizes resources on the internet to train a neural network dialogue model. Due to the uneven quality of resources on the internet, a negative reply is inevitably generated by the dialog system. The negative replies have a great influence on products, especially in a chat scene facing children, so that the negative replies need to be identified, filtered correspondingly and the like.
Currently, the following negative reply recognition methods are generally adopted: a negative direction dictionary, or called a negative direction word set, is constructed in advance, and when the answer generated by the dialog system includes words in the negative direction dictionary, the answer can be regarded as a negative direction answer. However, the coverage of the negative direction dictionary is limited, and many negative direction replies do not have obvious extreme words, so that the accuracy of the recognition result is poor, and the like.
Disclosure of Invention
The disclosure provides a negative reply recognition model obtaining method and a negative reply recognition method and device.
A negative reply recognition model acquisition method comprises the following steps:
obtaining a semantic representation model through pre-training;
training the semantic representation model by using the obtained weakly supervised corpus to obtain a first recognition model;
training the first recognition model by using the obtained supervision corpus to obtain a second recognition model;
and taking the second recognition model as a required negative reply recognition model so as to perform negative reply recognition on the reply to be processed by using the negative reply recognition model.
A negative reply recognition method, comprising:
acquiring a reply to be processed, wherein the reply is a dialogue reply;
and performing negative reply recognition on the reply to be processed by using the negative reply recognition model obtained according to the method to obtain a recognition result.
A negative-going reply recognition model acquisition apparatus comprising: the training system comprises a first training module, a second training module and a third training module;
the first training module is used for acquiring a semantic representation model through pre-training;
the second training module is used for training the semantic representation model by using the obtained weakly supervised corpus to obtain a first recognition model;
the third training module is configured to train the first recognition model by using the obtained supervised corpus to obtain a second recognition model, and use the second recognition model as a required negative reply recognition model, so as to perform negative reply recognition on a to-be-processed reply by using the negative reply recognition model.
A negative reply recognition apparatus comprising: the device comprises an acquisition module and an identification module;
the acquisition module is used for acquiring a reply to be processed, wherein the reply is a dialogue reply;
and the identification module is used for carrying out negative reply identification on the reply to be processed by utilizing the negative reply identification model obtained according to the device to obtain an identification result.
An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method as described above.
A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method as described above.
A computer program product comprising a computer program which, when executed by a processor, implements a method as described above.
One embodiment in the above disclosure has the following advantages or benefits: the multi-stage model training mode based on pre-training can be adopted to train and obtain a high-performance negative reply recognition model, so that the negative reply recognition model can be used for carrying out negative reply recognition on the reply to be processed, the accuracy of a recognition result is improved, the recall rate of the negative reply is improved, and the like.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a flow chart of an embodiment of a negative reply recognition model acquisition method according to the present disclosure;
FIG. 2 is a schematic diagram illustrating an implementation of active learning according to the present disclosure;
FIG. 3 is a schematic diagram illustrating a training process of a negative-going reply recognition model according to the present disclosure;
FIG. 4 is a flow chart of an embodiment of a negative reply identification method according to the present disclosure;
FIG. 5 is a schematic diagram of an overall architecture corresponding to the method of the present disclosure;
fig. 6 is a schematic structural diagram illustrating a negative reply recognition model obtaining apparatus 600 according to an embodiment of the present disclosure;
FIG. 7 is a block diagram illustrating an exemplary negative reply recognition apparatus 700 according to the present disclosure;
FIG. 8 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In addition, it should be understood that the term "and/or" herein is merely one type of association relationship that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
Fig. 1 is a flowchart of an embodiment of a negative reply recognition model acquisition method according to the present disclosure. As shown in fig. 1, the following detailed implementation is included.
In step 101, a semantic representation model is obtained by pre-training.
In step 102, the obtained weakly supervised corpus is used to train the semantic representation model, so as to obtain a first recognition model.
In step 103, the first recognition model is trained by using the obtained supervised corpus to obtain a second recognition model.
In step 104, the second recognition model is used as the required negative-going reply recognition model, so as to perform negative-going reply recognition on the reply to be processed by using the negative-going reply recognition model.
It can be seen that, in the scheme of the embodiment of the method, a multi-stage model training mode based on pre-training can be adopted to train to obtain a high-performance negative reply recognition model, so that the negative reply recognition model can be used for performing negative reply recognition on the reply to be processed, the accuracy of a recognition result is further improved, the recall rate of the negative reply is correspondingly improved, and the like.
First, a semantic representation model is obtained by pre-training large-scale corpora (or called training corpora, training samples, etc.). In recent years, the pre-training technology has made great progress in the field of natural language processing, and various task effects can be remarkably improved by utilizing a large amount of unstructured text corpora to learn semantic representation. The method introduces the pre-training technology into the negative reply recognition, can obtain better semantic representation, and recognizes whether the negative reply is performed from a deeper dialogue semantic level. How to obtain a semantic representation model through pre-training is the prior art.
The key problem of the data-driven recognition model is how to obtain a large amount of high-confidence data, which can only be obtained by a manual labeling method, but the manual labeling has the problems of high cost, long period and the like, and a small amount of labeled data is difficult to obtain a good effect in a complex recognition task.
Therefore, a multi-stage task training mode, namely a multi-stage model training mode is provided in the disclosure, specifically, the semantic representation model can be trained by using the obtained weak supervised corpus to obtain a first recognition model, the first recognition model can be trained by using the obtained supervised corpus to obtain a second recognition model, and the second recognition model can be used as a required negative reply recognition model.
Wherein, positive and negative linguistic data can be mined from the internet as the obtained weakly supervised linguistic data.
For example, negative corpora may include: comment statements not revealed in the information flow (feed), and/or negative statements filtered from social platform conversational comment statements using a negative dictionary, the negative statements being statements that include words in the negative dictionary. Comment sentences which are not shown in the information flow are usually masked negative sentences, and the sentences can be used as negative linguistic data. In addition, negative sentences filtered from the social platform dialogue comment sentences by using the negative dictionary can also be used as negative corpora. The negative dictionary can be pre-constructed, and specific words can be included according to actual needs.
The forward corpus may include: sentences that do not include words in the negative dictionary. The number of such sentences may be very large, and portions thereof may be extracted as forward corpuses.
The weakly supervised corpus obtained in the above weakly supervised mode is characterized by a large number, but the label accuracy is general, and the weakly supervised corpus can be applied to the next stage of the pre-training model, i.e. the semantic representation model is trained by using the weakly supervised corpus, so as to obtain the first recognition model.
By the mode, a large amount of positive linguistic data and negative linguistic data can be obtained without manual labeling, semantic representation models can be further optimized by the aid of the linguistic data, model performance is improved, the problems of high cost, long period and the like caused by manual labeling are solved, cost is saved, processing efficiency is improved, and the like.
And aiming at the first recognition model, training the first recognition model by using the obtained supervision corpus to obtain a second recognition model. The supervised corpus may include manually labeled positive and negative corpora. The first recognition model can be further finely adjusted by using a small amount of high-quality manual labeling corpora so as to further improve the performance of the model and the like.
The negative reply recognition model is essentially a classification model, which can be a two-classification model and a multi-classification model, and specifically, the two-classification model or the multi-classification model can be determined according to actual needs. For a dichotomous model, the identification result is one of positive and negative, and for a multiclassified model, the identification result may be one of positive, neutral, insulting 35881, abuse, child intolerance, and political sensitivity, wherein the abuse, child intolerance, and political sensitivity may be considered negative.
In order to obtain the supervised corpora, part of corpora can be extracted from the existing dialogue corpus for manual labeling, so that the required positive corpora and negative corpora can be obtained. For example, the annotating personnel can make linguistic category judgment according to their own feelings and classify them as positive or negative, or specifically as foul 35881, abuse or child inadequacy, etc.
In addition, an Active Learning (Active Learning) mode can be adopted, the first recognition model is trained by using the supervised corpus, and then corpus batch iterative labeling, model effect continuous optimization and the like can be carried out.
Fig. 2 is a schematic diagram of an implementation manner of active learning according to the present disclosure. As shown in fig. 2, assuming that 1 million corpora exist (actually, it may be far greater than this, and this is merely an example), then, a part of the corpora may be labeled manually, and the labeled corpora may be used to train the first recognition model, and then, the corpora that are difficult to classify the model may be enriched and labeled, and the labeled corpora may be used to train the model again, for example, the first recognition model obtained by the latest training may be used to score the unlabeled corpora, the score may be a value between 0 and 1, the higher the negative direction is the closer to 1, the corpora scored between 0.3 and 0.7 may be selected, labeled manually, and the first recognition model may be trained again in combination with the corpora labeled corpora, and then the above process may be repeated.
By means of active learning, the model training effect can be improved, the model performance can be improved, and the like.
Generally speaking, the proportion of the negative corpora in the real corpus distribution is small, and the imbalance of the sample categories (i.e., corpus categories) has a large influence on the training effect of the classification model.
Accordingly, certain methods may be employed to optimize the problem. For example, when the first recognition model is trained using the supervised corpus, the sample class equalization process may also be performed.
Specifically, the sample class equalization process may include one or any combination of the following: expanding the negative linguistic data according to the negative linguistic data in the supervision linguistic data; sampling forward linguistic data in the supervision linguistic data; the loss function in the training process is modified.
When negative corpus is expanded according to the negative corpus in the supervised corpus, the negative corpus in the supervised corpus can be used as query (query), a reply is given through a dialog system, in this case, the reply probability is also negative, and the reply can be used as the expanded negative corpus.
Sampling the forward linguistic data in the supervision linguistic data, namely, sampling the forward linguistic data, so that the proportion of the positive and negative linguistic data is controlled within a reasonable range. In addition, in the case of the imbalance of the sample types, the loss function during training is biased to the one with more samples, so that the loss function during training is small, but the accuracy of the class identification with less samples is not high, and the problem can be optimized by modifying the loss function. The concrete implementation of sampling the forward corpus in the supervised corpus and modifying the loss function in the training process is the prior art.
Through sample class equalization processing, the model training effect is further improved, the model performance is improved, and the like.
Based on the above description, fig. 3 is a schematic diagram of a training process of the negative-direction reply recognition model according to the present disclosure. As shown in fig. 3, in the pre-training stage, a kNowledge-Enhanced semantic Representation model (ERNIE, Enhanced Representation from knoveld ensemble) 2.0 may be used, which uses a large amount of text corpora to perform text Representation pre-training, and has superior performance in a plurality of tasks, and in addition, in the classification task, a Convolutional Neural Network (CNN) model may be used, and the corresponding output of the pre-training model may be passed through a CNN Network structure, and under the guidance of a label, model training may be performed using an optimization algorithm of gradient descent, and the like. For specific implementation, refer to the foregoing related descriptions, which are not repeated.
And the actual negative reply recognition can be carried out by utilizing the trained negative reply recognition model.
Fig. 4 is a flowchart of an embodiment of a negative reply identification method according to the present disclosure. As shown in fig. 4, the following detailed implementation is included.
In step 401, a reply to be processed is obtained, where the reply is a dialog reply.
In step 402, a negative reply recognition model obtained by pre-training is used to perform negative reply recognition on the reply to be processed, so as to obtain a recognition result.
The number of the negative reply recognition models can be one or more. When the number of the replies is multiple, namely when M negative reply recognition models are obtained, and M is a positive integer greater than one, the recognition results corresponding to the negative reply recognition models can be respectively obtained for the replies to be processed, and the recognition results can be integrated to determine the final recognition result of the replies to be processed.
The M negative-direction reply recognition models can be obtained according to the negative-direction reply recognition model obtaining method disclosed in the present disclosure, and different negative-direction reply recognition models can correspond to different training corpora and the like, for example, can correspond to different supervision corpora and the like.
Therefore, M recognition results can be obtained aiming at the reply to be processed, and the M recognition results can be integrated to determine the final recognition result of the reply to be processed, so that the accuracy of the recognition result is further improved.
For example, when it is determined that the reply to be processed is identified as a negative reply in any one of the identification results, it is determined that the reply to be processed is a negative reply, and the negative reply is filtered accordingly, so that occurrence of the negative reply is avoided as much as possible, and the recall rate of the negative reply is further improved.
For example, when it is determined that the pending reply is identified as a negative reply in more than half of the M recognition results, the pending reply may be determined as the negative reply, and the specific implementation manner is not limited, but the foregoing manner is a more preferred manner.
With the above introduction in mind, fig. 5 is a schematic diagram of an overall architecture corresponding to the method of the present disclosure. As shown in fig. 5, the negative reply recognition model obtained according to the method of the present disclosure may be applied to various dialog systems, such as a UNIT (Understanding and Interaction Technology) platform, a smart speaker, and the like.
Taking a dialog system as an intelligent sound box as an example, for a reply generated by the intelligent sound box and directed to a user, a negative reply recognition model obtained according to the method disclosed by the disclosure can be utilized to perform negative reply recognition on the reply, so that a recognition result is obtained, if the recognition result shows that the reply is a negative reply, the reply can be filtered, and then how to handle the reply is not limited, for example, the reply can be generated again and can be fed back to the user when the regenerated reply is determined to be a non-negative reply, and if the recognition result shows that the reply is a non-negative reply, the reply can be directly fed back to the user, so that the occurrence of the negative reply in the dialog is avoided as much as possible, and the dialog performance and the like of the intelligent sound box are improved.
The method can effectively identify the negative replies occurring in the conversation process, improve the recall rate of the negative replies and the like, and after the method is applied to the product, the occurrence proportion of the negative replies can be reduced to an extremely low level, while the occurrence proportion of the extreme negative replies is basically zero, thereby improving the product performance and the like.
It is noted that while for simplicity of explanation, the foregoing method embodiments are described as a series of acts, those skilled in the art will appreciate that the present disclosure is not limited by the order of acts, as some steps may, in accordance with the present disclosure, occur in other orders and concurrently. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required for the disclosure.
The above is a description of embodiments of the method, and the embodiments of the apparatus are further described below.
Fig. 6 is a schematic structural diagram illustrating a negative reply recognition model obtaining apparatus 600 according to an embodiment of the present disclosure. As shown in fig. 6, includes: a first training module 601, a second training module 602, and a third training module 603.
The first training module 601 is configured to obtain a semantic representation model through pre-training.
The second training module 602 is configured to train the semantic representation model by using the obtained weakly supervised corpus to obtain a first recognition model.
The third training module 603 is configured to train the first recognition model with the obtained supervised corpus to obtain a second recognition model, and use the second recognition model as a required negative reply recognition model, so as to perform negative reply recognition on a to-be-processed reply by using the negative reply recognition model.
The first training module 601 may perform pre-training through large-scale corpora to obtain a semantic representation model. On this basis, a multi-stage task training mode may be adopted, that is, the second training module 602 may train the semantic representation model using the obtained weakly supervised corpus to obtain a first recognition model, and the third training module 603 may train the first recognition model using the obtained supervised corpus to obtain a second recognition model, so that the second recognition model may be used as a required negative reply recognition model.
Wherein, positive and negative linguistic data can be mined from the internet as the obtained weakly supervised linguistic data. For example, negative corpora may include: comment sentences not shown in the information flow, and/or negative sentences filtered from the social platform dialogue comment sentences using a negative dictionary, the negative sentences being sentences that include words in the negative dictionary. The forward corpus may include: sentences that do not include words in the negative dictionary.
The supervised corpus may include: manually labeled positive and negative corpora.
In addition, the third training module 603 may use an active learning mode to train the first recognition model by using supervised corpora, and may perform corpus batch iterative labeling and continuous optimization of model effect.
Furthermore, generally speaking, the proportion of the negative-going corpora in the distribution of the real corpora is small, and the imbalance of the sample categories has a great influence on the training effect of the classification model. For this reason, the third training module 603 may further perform sample class equalization processing when the first recognition model is trained using the supervised corpus.
The sample class equalization process may include one or any combination of the following: expanding the negative linguistic data according to the negative linguistic data in the supervision linguistic data; sampling forward linguistic data in the supervision linguistic data; the loss function in the training process is modified.
Fig. 7 is a schematic diagram illustrating a structure of a negative reply recognition apparatus 700 according to an embodiment of the disclosure. As shown in fig. 7, includes: an acquisition module 701 and an identification module 702.
An obtaining module 701, configured to obtain a reply to be processed, where the reply is a dialog reply.
The identification module 702 is configured to perform negative reply identification on the reply to be processed by using the negative reply identification model obtained in the apparatus, so as to obtain an identification result.
The number of the negative reply recognition models can be more than one. Accordingly, the recognition module 702 may obtain the recognition results corresponding to the negative reply recognition models respectively for the replies to be processed, and may combine the recognition results to determine the final recognition result of the replies to be processed.
For example, the recognition module 702 may determine that the pending reply is a negative reply when determining that the pending reply is recognized as a negative reply in any of the recognition results.
For a specific work flow of the apparatus embodiments shown in fig. 6 and fig. 7, reference is made to the related description in the foregoing method embodiments, and details are not repeated.
In a word, by adopting the scheme of the embodiment of the device disclosed by the invention, a multi-stage model training mode based on pre-training can be adopted to train and obtain a high-performance negative reply recognition model, so that the negative reply recognition model can be used for carrying out negative reply recognition on the reply to be processed, the accuracy of a recognition result is further improved, the recall rate of the negative reply is further improved, and the like.
The scheme disclosed by the invention can be applied to the field of artificial intelligence, in particular to the fields of natural language processing, deep learning, intelligent voice and the like. Artificial intelligence is a subject for studying a computer to simulate some thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning and the like) of a human, and has a hardware technology and a software technology, the artificial intelligence hardware technology generally comprises technologies such as a sensor, a special artificial intelligence chip, cloud computing, distributed storage, big data processing and the like, and the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, machine learning/deep learning, a big data processing technology, a knowledge graph technology and the like.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
FIG. 8 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and Virtual Private Server (VPS). The server may also be a server of a distributed system, or a server incorporating a blockchain. Cloud computing refers to accessing an elastically extensible shared physical or virtual resource pool through a network, resources can include servers, operating systems, networks, software, applications, storage devices and the like, a technical system for deploying and managing the resources in a self-service mode as required can be achieved, and efficient and powerful data processing capacity can be provided for technical applications and model training of artificial intelligence, block chains and the like through a cloud computing technology.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (22)
1. A negative reply recognition model acquisition method comprises the following steps:
obtaining a semantic representation model through pre-training;
training the semantic representation model by using the obtained weakly supervised corpus to obtain a first recognition model;
training the first recognition model by using the obtained supervision corpus to obtain a second recognition model;
and taking the second recognition model as a required negative reply recognition model so as to perform negative reply recognition on the reply to be processed by using the negative reply recognition model.
2. The method of claim 1, wherein the weakly supervised corpus comprises: and excavating positive linguistic data and negative linguistic data from the Internet.
3. The method of claim 2, wherein,
the negative corpus comprises: comment sentences not shown in the information flow and/or negative sentences filtered from the social platform dialogue comment sentences by using a negative dictionary, wherein the negative sentences are sentences comprising words in the negative dictionary;
the forward corpus comprises: sentences that do not include words in the negative dictionary.
4. The method of claim 1, wherein the supervisory corpus comprises: manually labeled positive and negative corpora.
5. The method of claim 1, wherein the training the first recognition model using the obtained supervised corpus comprises:
and training the first recognition model by using the supervision corpus in an active learning mode.
6. The method of claim 1, further comprising:
and when the first recognition model is trained by utilizing the supervision corpus, carrying out sample class equalization processing.
7. The method of claim 6, wherein,
the sample class equalization processing comprises one or any combination of the following steps: expanding negative linguistic data according to the negative linguistic data in the supervision linguistic data; sampling the forward linguistic data in the supervision linguistic data; the loss function in the training process is modified.
8. A negative reply recognition method, comprising:
acquiring a reply to be processed, wherein the reply is a dialogue reply;
and carrying out negative reply recognition on the reply to be processed by using a negative reply recognition model acquired according to the method of any one of claims 1 to 7 to obtain a recognition result.
9. The method of claim 8, wherein the negative reply recognition model has a number greater than one;
the method further comprises the following steps: and respectively acquiring the recognition results corresponding to the negative reply recognition models aiming at the replies to be processed, and integrating the recognition results to determine the final recognition result of the replies to be processed.
10. The method of claim 9, wherein the integrating the recognition results to determine a final recognition result of the reply to be processed comprises:
and when determining that the reply to be processed is identified as a negative reply in any identification result, determining that the reply to be processed is a negative reply.
11. A negative-going reply recognition model acquisition apparatus comprising: the training system comprises a first training module, a second training module and a third training module;
the first training module is used for acquiring a semantic representation model through pre-training;
the second training module is used for training the semantic representation model by using the obtained weakly supervised corpus to obtain a first recognition model;
the third training module is configured to train the first recognition model by using the obtained supervised corpus to obtain a second recognition model, and use the second recognition model as a required negative reply recognition model, so as to perform negative reply recognition on a to-be-processed reply by using the negative reply recognition model.
12. The apparatus of claim 11, wherein the weakly supervised corpus comprises: and excavating positive linguistic data and negative linguistic data from the Internet.
13. The apparatus of claim 12, wherein,
the negative corpus comprises: comment sentences not shown in the information flow and/or negative sentences filtered from the social platform dialogue comment sentences by using a negative dictionary, wherein the negative sentences are sentences comprising words in the negative dictionary;
the forward corpus comprises: sentences that do not include words in the negative dictionary.
14. The apparatus of claim 11, wherein the supervisory corpus comprises: manually labeled positive and negative corpora.
15. The apparatus of claim 11, wherein,
and the third training module trains the first recognition model by using the supervised corpus in an active learning mode.
16. The apparatus of claim 11, wherein,
the third training module is further configured to perform sample class balancing processing when the first recognition model is trained by using the supervised corpus.
17. The apparatus of claim 16, wherein,
the sample class equalization processing comprises one or any combination of the following steps: expanding negative linguistic data according to the negative linguistic data in the supervision linguistic data; sampling the forward linguistic data in the supervision linguistic data; the loss function in the training process is modified.
18. A negative reply recognition apparatus comprising: the device comprises an acquisition module and an identification module;
the acquisition module is used for acquiring a reply to be processed, wherein the reply is a dialogue reply;
the identification module is configured to perform negative reply identification on the reply to be processed by using a negative reply identification model acquired by the apparatus according to any one of claims 11 to 17, so as to obtain an identification result.
19. The apparatus of claim 18, wherein the number of negative-going reply recognition models is greater than one;
the identification module is further configured to, for the to-be-processed reply, respectively obtain an identification result corresponding to each negative reply identification model, and synthesize each identification result to determine a final identification result of the to-be-processed reply.
20. The apparatus of claim 19, wherein,
the identification module determines that the reply to be processed is a negative reply when determining that the reply to be processed is a negative reply in any one of the identification results.
21. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10.
22. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110181108.5A CN112530415B (en) | 2021-02-10 | 2021-02-10 | Negative reply recognition model acquisition and negative reply recognition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110181108.5A CN112530415B (en) | 2021-02-10 | 2021-02-10 | Negative reply recognition model acquisition and negative reply recognition method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112530415A CN112530415A (en) | 2021-03-19 |
| CN112530415B true CN112530415B (en) | 2021-07-16 |
Family
ID=74975736
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110181108.5A Active CN112530415B (en) | 2021-02-10 | 2021-02-10 | Negative reply recognition model acquisition and negative reply recognition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112530415B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113408300B (en) * | 2021-07-09 | 2024-02-20 | 北京百度网讯科技有限公司 | Model training method, brand word recognition device and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110807332A (en) * | 2019-10-30 | 2020-02-18 | 腾讯科技（深圳）有限公司 | Training method of semantic understanding model, semantic processing method, semantic processing device and storage medium |
| CN110909131A (en) * | 2019-11-26 | 2020-03-24 | 携程计算机技术（上海）有限公司 | Model generation method, emotion recognition method, system, device and storage medium |
| CN111241250A (en) * | 2020-01-22 | 2020-06-05 | 中国人民大学 | Emotional dialogue generation system and method |
| CN111695919A (en) * | 2019-03-11 | 2020-09-22 | 北京嘀嘀无限科技发展有限公司 | Evaluation data processing method, evaluation data processing device, electronic device, and storage medium |
| CN111951789A (en) * | 2020-08-14 | 2020-11-17 | 北京达佳互联信息技术有限公司 | Training of speech recognition model, speech recognition method, apparatus, device and medium |
| CN112259085A (en) * | 2020-09-28 | 2021-01-22 | 上海声瀚信息科技有限公司 | Two-stage voice awakening algorithm based on model fusion framework |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9875736B2 (en) * | 2015-02-19 | 2018-01-23 | Microsoft Technology Licensing, Llc | Pre-training and/or transfer learning for sequence taggers |
-
2021
- 2021-02-10 CN CN202110181108.5A patent/CN112530415B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111695919A (en) * | 2019-03-11 | 2020-09-22 | 北京嘀嘀无限科技发展有限公司 | Evaluation data processing method, evaluation data processing device, electronic device, and storage medium |
| CN110807332A (en) * | 2019-10-30 | 2020-02-18 | 腾讯科技（深圳）有限公司 | Training method of semantic understanding model, semantic processing method, semantic processing device and storage medium |
| CN110909131A (en) * | 2019-11-26 | 2020-03-24 | 携程计算机技术（上海）有限公司 | Model generation method, emotion recognition method, system, device and storage medium |
| CN111241250A (en) * | 2020-01-22 | 2020-06-05 | 中国人民大学 | Emotional dialogue generation system and method |
| CN111951789A (en) * | 2020-08-14 | 2020-11-17 | 北京达佳互联信息技术有限公司 | Training of speech recognition model, speech recognition method, apparatus, device and medium |
| CN112259085A (en) * | 2020-09-28 | 2021-01-22 | 上海声瀚信息科技有限公司 | Two-stage voice awakening algorithm based on model fusion framework |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112530415A (en) | 2021-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112487173B (en) | Man-machine conversation method, device and storage medium | |
| US20220358292A1 (en) | Method and apparatus for recognizing entity, electronic device and storage medium | |
| CN113344089B (en) | Model training method and device and electronic equipment | |
| CN112466289A (en) | Voice instruction recognition method and device, voice equipment and storage medium | |
| EP4057283A2 (en) | Method for detecting voice, method for training, apparatuses and smart speaker | |
| CN115309877A (en) | Dialog generation method, dialog model training method and device | |
| CN113360001A (en) | Input text processing method and device, electronic equipment and storage medium | |
| CN112530415B (en) | Negative reply recognition model acquisition and negative reply recognition method and device | |
| CN112541070B (en) | Mining method and device for slot updating corpus, electronic equipment and storage medium | |
| CN112560480B (en) | Task community discovery method, device, equipment and storage medium | |
| CN114186681A (en) | Method, apparatus and computer program product for generating model clusters | |
| CN114758649B (en) | Voice recognition method, device, equipment and medium | |
| CN113641724B (en) | Knowledge tag mining method and device, electronic equipment and storage medium | |
| CN113254578B (en) | Method, apparatus, device, medium and product for data clustering | |
| CN113886543A (en) | Method, apparatus, medium, and program product for generating an intent recognition model | |
| CN114118937A (en) | Information recommendation method and device based on task, electronic equipment and storage medium | |
| CN114969195A (en) | Dialogue content mining method and dialogue content evaluation model generation method | |
| CN114490967A (en) | Training method of dialogue model, dialogue method and device of dialogue robot and electronic equipment | |
| CN115312042A (en) | Method, apparatus, device and storage medium for processing audio | |
| CN113408269A (en) | Text emotion analysis method and device | |
| CN112559727A (en) | Method, apparatus, device, storage medium, and program for outputting information | |
| CN112632999A (en) | Named entity recognition model obtaining method, named entity recognition device and named entity recognition medium | |
| CN113593528B (en) | Training method and device of voice segmentation model, electronic equipment and storage medium | |
| CN116244413B (en) | New intention determining method, apparatus and storage medium | |
| CN112989797B (en) | Model training and text expansion methods, devices, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |