CN111708873A - Intelligent question answering method and device, computer equipment and storage medium - Google Patents
Intelligent question answering method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111708873A CN111708873A CN202010545399.7A CN202010545399A CN111708873A CN 111708873 A CN111708873 A CN 111708873A CN 202010545399 A CN202010545399 A CN 202010545399A CN 111708873 A CN111708873 A CN 111708873A
- Authority
- CN
- China
- Prior art keywords
- intention
- target
- question
- sub
- vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 239000013598 vector Substances 0.000 claims abstract description 300
- 238000012549 training Methods 0.000 claims description 67
- 230000015654 memory Effects 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 10
- 238000013473 artificial intelligence Methods 0.000 abstract description 13
- 206010035664 Pneumonia Diseases 0.000 description 32
- 238000010586 diagram Methods 0.000 description 20
- 238000005516 engineering process Methods 0.000 description 20
- 238000004422 calculation algorithm Methods 0.000 description 17
- 238000013136 deep learning model Methods 0.000 description 15
- 230000006870 function Effects 0.000 description 15
- 239000010410 layer Substances 0.000 description 12
- 238000013441 quality evaluation Methods 0.000 description 11
- 238000005065 mining Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000012795 verification Methods 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 7
- 201000010099 disease Diseases 0.000 description 7
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 7
- 238000001303 quality assessment method Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 6
- 238000003058 natural language processing Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 230000014509 gene expression Effects 0.000 description 5
- 238000012937 correction Methods 0.000 description 4
- 238000002372 labelling Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 238000004821 distillation Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 208000024891 symptom Diseases 0.000 description 3
- 241000711573 Coronaviridae Species 0.000 description 2
- 241000711902 Pneumovirus Species 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000009193 crawling Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 229940079593 drug Drugs 0.000 description 2
- 238000003064 k means clustering Methods 0.000 description 2
- 239000002346 layers by function Substances 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000002265 prevention Effects 0.000 description 2
- 230000001681 protective effect Effects 0.000 description 2
- 230000002829 reductive effect Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 230000006403 short-term memory Effects 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 208000015181 infectious disease Diseases 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000013516 source data verification Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000004659 sterilization and disinfection Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
- G16H40/20—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- General Business, Economics & Management (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Primary Health Care (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Epidemiology (AREA)
- Biomedical Technology (AREA)
- Human Computer Interaction (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Machine Translation (AREA)
Abstract
The application relates to an intelligent question answering method, an intelligent question answering device, computer equipment and a storage medium. The method comprises the steps of obtaining a target question to be answered; obtaining a target semantic intention related to the semantics of the target question; acquiring sub-intention representative vectors matched with target statement vectors corresponding to the target problems as target sub-intention representative vectors, wherein the sub-intention representative vectors are representative vectors corresponding to all sub-intention categories obtained by clustering problem vector sets corresponding to the target semantic intents; taking a sub-intention corresponding to the target sub-intention representative vector as a target sub-intention, and acquiring a problem matched with the target sub-intention as a reference problem; and obtaining a target answer of the target question according to the reference answer corresponding to the reference question. The target question can be subjected to semantic intention recognition based on an artificial intelligence model, and the accuracy of the obtained answer to the question can be improved by adopting the method.
Description
Technical Field
The present application relates to the field of question and answer technologies, and in particular, to an intelligent question and answer method, apparatus, computer device, and storage medium.
Background
With the rapid development of information technology, intelligent response to the user's question is required in many scenarios. For example, in the medical field, the intelligent question-answering robot can intelligently reply to questions related to the medical field, which are made by a user, based on an artificial intelligence model.
When a question of a user is received, the intelligent question-answering system can search answers based on the question to obtain answers to the question, however, the answer is not matched with the question, namely, the obtained answer has low accuracy.
Disclosure of Invention
In view of the above, it is necessary to provide an intelligent question answering method, apparatus, computer device and storage medium for solving the above technical problems.
A method of intelligent question answering, the method comprising: acquiring a target question to be answered; obtaining a target semantic intention related to the semantics of the target question; acquiring sub-intention representative vectors matched with target statement vectors corresponding to the target problems as target sub-intention representative vectors, wherein the sub-intention representative vectors are representative vectors corresponding to all sub-intention categories obtained by clustering problem vector sets corresponding to the target semantic intents; taking a sub-intention corresponding to the target sub-intention representative vector as a target sub-intention, and acquiring a problem matched with the target sub-intention as a reference problem; and obtaining a target answer of the target question according to the reference answer corresponding to the reference question.
An intelligent question answering device, characterized in that the device comprises: the target question acquisition module is used for acquiring a target question to be answered; a target semantic intention acquisition module for acquiring a target semantic intention related to the semantics of the target question; a target representative vector obtaining module, configured to obtain a sub-intention representative vector matched with a target statement vector corresponding to the target question, as a target sub-intention representative vector, where the sub-intention representative vector is a representative vector corresponding to each sub-intention category obtained by clustering a question vector set corresponding to the target semantic intention; a reference problem obtaining module, configured to use a sub-intention corresponding to the target sub-intention representative vector as a target sub-intention, and obtain a problem matching with the target sub-intention as a reference problem; and the target answer obtaining module is used for obtaining a target answer of the target question according to the reference answer corresponding to the reference question.
In some embodiments, the module for clustering the problem vector set corresponding to the target semantic intention to obtain the representative vector corresponding to each sub-intention category includes: a problem vector set acquisition unit, configured to acquire a problem set corresponding to the target semantic intention, acquire statement vectors corresponding to the problems in the problem set, and form a problem vector set; the clustering unit is used for clustering the problem vector set to obtain a clustering vector set corresponding to each sub-intention category of the target semantic intention; and the sub-intention representative vector acquisition unit is used for acquiring a representative vector of the clustering vector set corresponding to the sub-intention category as a sub-intention representative vector corresponding to the sub-intention category.
In some embodiments, the sub-intent representative vector acquisition unit is to: and acquiring a central point vector of the clustering vector set corresponding to the sub-intention category as a sub-intention representative vector corresponding to the sub-intention category.
In some embodiments, the target representative vector acquisition module is to: calculating the distance between the target statement vector and each sub-intention representative vector; and acquiring a sub-intention representative vector with the minimum distance from the target statement vector as a target sub-intention representative vector.
In some embodiments, the target semantic intent related to the semantics of the target question is identified by a trained first intent recognition model, the training module of which comprises: the first intention recognition result obtaining unit is used for carrying out intention recognition on the training problem based on a first intention recognition model to be trained to obtain a first intention recognition result; a second intention recognition result obtaining unit, configured to perform intention recognition on the training question based on a second intention recognition model to obtain a second intention recognition result; and the parameter adjusting unit is used for obtaining a model loss value of the first intention recognition model to be trained based on the difference between the first intention recognition result and the second intention recognition result, adjusting model parameters in the first intention recognition model to be trained according to the model loss value to obtain the trained first intention recognition model, and the model loss value and the difference are in positive correlation.
In some embodiments, the parameter adjustment unit is configured to: obtaining a first loss value according to the difference between the first intention identification result and the second intention identification result; obtaining a second loss value according to the difference between the first intention recognition result and a standard intention recognition result corresponding to the training problem; and obtaining a model loss value of the first intention recognition model to be trained according to the first loss value and the second loss value.
In some embodiments, the reference problem acquisition module is configured to include: acquiring a named entity set corresponding to the target problem, and acquiring a key condition intention corresponding to the target problem according to the named entity set; and acquiring a question matched with the key condition intention and the target sub-intention as a reference question.
In some embodiments, the apparatus further comprises: the target word acquisition module is used for acquiring words corresponding to the candidate new problems as target words; the first quantity obtaining module is used for obtaining a first quantity of questions including the target words in a first question set, and matching answers exist in the questions in the first question set; a first novelty obtaining module for determining a first novelty of the new candidate question in the first question set according to the first quantity, the first quantity having a negative correlation with the first novelty; a target new problem determination module to take the candidate new problem as a target new problem when the first novelty is greater than a first threshold.
In some embodiments, the apparatus further comprises: the second quantity obtaining module is used for obtaining a second quantity of questions including the target words in a second question set, wherein the questions in the second question set are questions without matching answers; a second novelty obtaining module for determining a second novelty of the candidate new question in the second question set according to the second quantity, the second quantity having a negative correlation with the second novelty; the target new problem determination module is to: when the first novelty is greater than a first threshold and the second novelty is less than a second threshold, the candidate new question is treated as a target new question.
In some embodiments, the first novelty acquisition module is to: calculating the inverse document frequency corresponding to the target words according to the first quantity; and calculating a first novelty of the candidate new question in the first question set according to the inverse document frequency corresponding to the target word.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program: acquiring a target question to be answered; obtaining a target semantic intention related to the semantics of the target question; acquiring sub-intention representative vectors matched with target statement vectors corresponding to the target problems as target sub-intention representative vectors, wherein the sub-intention representative vectors are representative vectors corresponding to all sub-intention categories obtained by clustering problem vector sets corresponding to the target semantic intents; taking a sub-intention corresponding to the target sub-intention representative vector as a target sub-intention, and acquiring a problem matched with the target sub-intention as a reference problem; and obtaining a target answer of the target question according to the reference answer corresponding to the reference question.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of: acquiring a target question to be answered; obtaining a target semantic intention related to the semantics of the target question; acquiring sub-intention representative vectors matched with target statement vectors corresponding to the target problems as target sub-intention representative vectors, wherein the sub-intention representative vectors are representative vectors corresponding to all sub-intention categories obtained by clustering problem vector sets corresponding to the target semantic intents; taking a sub-intention corresponding to the target sub-intention representative vector as a target sub-intention, and acquiring a problem matched with the target sub-intention as a reference problem; and obtaining a target answer of the target question according to the reference answer corresponding to the reference question.
According to the intelligent question-answering method, the intelligent question-answering device, the intelligent computer equipment and the intelligent question-answering storage medium, for the target question to be answered, the target semantic intention related to the semantics of the target question can be obtained, the sub-intention representative vector is obtained by clustering the question vector set corresponding to the target semantic intention, so that the sub-intention representative vector matched with the target statement vector is obtained to serve as the target sub-intention representative vector, the sub-intention corresponding to the target sub-intention representative vector serves as the target sub-intention, the target sub-intention which meets the semantics of the target question and has fine granularity can be accurately obtained, the target answer is obtained according to the answer corresponding to the question matched with the target sub-intention, and the obtained answer is high in accuracy.
Drawings
FIG. 1 is a diagram of an environment in which the intelligent question-answering method may be used in some embodiments;
FIG. 2 is a schematic flow chart of a smart question answering method in some embodiments;
FIG. 3 is a schematic diagram of a question and answer interface in some embodiments;
FIG. 4 is a schematic diagram of intent matching in some embodiments;
FIG. 5 is a schematic diagram of obtaining an input to a quality check that is index weighted in some embodiments;
FIG. 6 is a schematic flow chart diagram illustrating the training steps of the trained first intent recognition model in some embodiments;
FIG. 7A is a schematic diagram of a training principle for training a first intent recognition model in some embodiments;
FIG. 7B is a block diagram of a first intent recognition model in some embodiments;
FIG. 8 is a flow diagram illustrating the mining of new problems in some embodiments;
FIG. 9 is a schematic diagram of the processing of the intelligent question answering method in some embodiments;
FIG. 10 is a block diagram of a system in accordance with further embodiments;
FIG. 11 is a block diagram of an intelligent question and answer apparatus in some embodiments;
FIG. 12 is a diagram of the internal structure of a computer device in some embodiments.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The intelligent question answering method provided by the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. When a user needs to obtain answers to questions, the user can enter an intelligent question and answer interface through the terminal 102, the terminal 102 receives target questions to be answered input by the user through the intelligent question and answer interface, the server 104 obtains the target questions to be answered, the intelligent question and answer method provided by the embodiment of the application is executed, target answers corresponding to the target questions are obtained, the target answers are returned to the terminal 102, and the terminal 102 displays the target answers on the intelligent question and answer interface. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented by an independent server or a server cluster formed by a plurality of servers.
It should be noted that the terminal 102 may also locally execute the intelligent question-answering method in the embodiments of the present application.
The intelligent question answering means that answers can be automatically determined according to questions. According to the intelligent question-answering method in the embodiments of the application, the target semantic intention can be obtained based on artificial intelligence model recognition, and through the intelligent question-answering method in the embodiments of the application, a server or a terminal can automatically obtain answers of questions to realize intelligent human-computer interaction.
Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.
The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.
It can be understood that the information query method in the embodiments of the present application uses the natural language processing technology in the artificial intelligence technology to implement a human-computer interaction session, thereby automatically guiding the user to query and understand the user state of the user and the suggestion information corresponding to the user state
Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that enable efficient communication between humans and computers using natural language. Natural language processing is a science integrating linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language that people use everyday, so it is closely related to the research of linguistics. Natural language processing techniques typically include text processing, semantic understanding, machine translation, robotic question and answer, and knowledge-mapping techniques.
As shown in fig. 2, an intelligent question answering method is provided, which is described by taking the application of the method to the server in fig. 1 as an example, and includes the following steps:
step S202, a target question to be answered is acquired.
Specifically, the questions to be answered may be sent by the terminal, for example, when a user needs to obtain an answer to a certain question, the user may enter a question-answering interface through the operation terminal, the terminal receives the questions input by the user through the question-answering interface, sends an answer obtaining request to the server, and the server obtains the questions carried in the answer obtaining request as target questions to be answered.
In some embodiments, a plurality of candidate questions to be answered may be displayed on the question-answering interface, and the terminal may receive a question selection operation of the user, and take a question selected according to the selection operation as a target question. For example, as shown in fig. 3, which is a schematic diagram of a question-and-answer interface in some embodiments, the question-and-answer interface includes an input box 302, a candidate question presentation box 304, and a semantic intention presentation box 306, and the terminal may receive a question input by a user through the input box 302. The terminal may also receive a click operation of the user on a question on the candidate question display frame 304, and take a candidate question corresponding to a position clicked by the click operation as a question to be answered. The terminal can also determine the target semantic intention according to the selection operation of the semantic intention by the user. The server may return a question matching the target semantic intention to the terminal, the terminal may present the question returned by the server in the candidate question presentation box 304, and when the question "what clinical manifestation the coronary pneumonia patient has" input by the user is received, the server may obtain an answer to the question, return to the terminal, and the terminal displays the corresponding answer.
In some embodiments, the user may have spelling errors in the question input by keyboard or voice, and the server may automatically recognize and correct the question input by the user through the language model, i.e., perform language model error correction. The language model error correction is to obtain a candidate set of correct expressions by using the language model and sort the candidate set on the basis of realizing error identification, and the first ordered expression can be selected as correct input, namely as a target problem.
Step S204, a target semantic intention related to the semantics of the target question is acquired.
The semantics refers to the meaning of the language, and the target problem is expressed by the natural language, so the semantics of the target problem can be obtained by the sentence expressed by the natural language. For example, the target question may be a sentence in a language such as English, Chinese, or Japanese. Intent refers to the purpose of the question, intent representing the user's expectation, i.e. "what i want to be the answer to" the user embodies by the question presented. Semantic intent refers to an intent determined from the semantics of the target question. For example, the target question may be "what mask can block pneumovirus", and the target semantic intent may be "pneumonia protection".
Specifically, semantic analysis may be performed on the target problem to obtain a target semantic intention corresponding to the target problem. For example, semantic analysis can be performed on the target problem based on a pre-trained artificial intelligence model to obtain a semantic intention corresponding to the target problem. The artificial intelligence model may be, for example, a model based on BilSTM (Bi-directional Long Short-Term Memory), or a distillation learning model using BERT (Bidirectional encoder representation). The server can perform supervised training according to the problem with the semantic intention label to obtain an intention identification model, and the semantic intention of the target problem is identified based on the intention identification model. The semantic intention labels corresponding to the questions for model training can be manually labeled.
In some embodiments, the specific category of semantic intentions may be set according to practical situations, for example, for a pneumonia-related question and answer, 9 semantic intentions may be defined, including the categories of questions the user cares about during an epidemic situation, such as "pneumonia protection", "pneumonia treatment mode", and "pneumonia characteristics" semantic intentions. When training the semantic intention recognition model, artificially performing intention labeling on the problem based on the defined semantic intention, and performing model training on the semantic intention recognition model based on the problem labeled with the semantic intention, so that when acquiring the target problem, the semantic intention of the target problem can be recognized based on the trained semantic intention recognition model.
In some embodiments, the semantic intent of the target problem may be identified based on a manually set rule, a decision tree (dtn) and a CNN (Convolutional Neural Network) Neural Network classification model, so as to obtain the target semantic intent.
Step S206, acquiring the sub-intention representative vector matched with the target statement vector corresponding to the target question as a target sub-intention representative vector, wherein the sub-intention representative vector is a representative vector corresponding to each sub-intention category obtained by clustering the question vector set corresponding to the target semantic intention.
The target statement vector is obtained by vectorizing the statement of the target question. For example, the target problem may be segmented to obtain a word sequence, a word vector corresponding to each word in the word sequence is obtained to obtain a word vector sequence, and a text coding model is used to perform coding based on the word vector sequence to obtain a sentence vector representing sentence characteristics. The text coding model may be, for example, a BERT model, which may be a language coder, that converts an input sentence or paragraph into a feature vector.
The problem vector set comprises a plurality of problem vectors, wherein the plurality refers to at least two problem vectors. The problem vector is a vectorized representation of the problem statement. For example, the question may also be encoded by a text coding model, resulting in a sentence vector representing the sentence characteristics. In the question vector set corresponding to the target semantic intention, the semantic intention of the question corresponding to each question vector is the target semantic intention. That is, the question vector corresponding to the target semantic intent is a vectorized representation of the question corresponding to the target semantic intent, and for example, if the semantic intent is pneumonia protection, a plurality of questions whose semantic intent is pneumonia protection can be acquired, and sentence vectors of the questions can be acquired to form a question vector set corresponding to the "pneumonia protection" semantic intent.
The sub-intents of the target semantic intent are intents obtained by further intent classification of the target semantic intent, and one target semantic intent may include at least two sub-intent categories. For example, the semantic intent of "pneumonia protection" can be further divided into sub-intentions such as "pneumonia protection by mask" and "pneumonia protection by disinfection". The sub-intention representative vector refers to a vector representing the category of the sub-intention, and may be, for example, a cluster center of a set of cluster vectors corresponding to the sub-intention.
Clustering refers to the process of dividing a collection into multiple classes consisting of similar objects, and is an unsupervised data mining approach. The problem vector set of the semantic intentions is divided into a plurality of classes, each class corresponds to one sub-intention, the semantic intentions can be divided into intentions with finer granularity, and the problem vectors with similar distances, namely similar characteristics, can be divided into the same sub-intention category based on an unsupervised learning mode, so that the obtained sub-intentions are more accurate. The embodiment does not limit the specific form of the clustering algorithm. Taking the clustering algorithm that adopts a K-means algorithm or a K-centroids algorithm (K-means) as an example, the server may cluster vectors in the problem vector set according to the number K of the clustering categories to obtain K clustering vector sets, where one clustering vector set corresponds to one sub-intention, and K is a positive integer greater than or equal to 2, and may be set as needed, for example, 9.
The sub-intention representative vector matching the target sentence vector may be a sub-intention representative vector whose distance from the target sentence vector satisfies a distance condition, for example, the distance condition may be that a distance value is smaller than a preset distance. The distance condition may also be a distance minimum. For example, the distance between the target sentence vector and each sub-intention representative vector may be calculated, and the sub-intention representative vector having the smallest distance from the target sentence vector may be obtained as the sub-intention representative vector matching the target sentence vector, to obtain the target sub-intention representative vector. The distance calculation method may be set as required, and may be, for example, an euclidean distance calculation method, where, since the problem vector set corresponding to the target semantic intent is divided into a plurality of classes, one class may correspond to one sub-intent representative vector, that is, there are a plurality of sub-intent representative vectors. Therefore, by acquiring the sub-intention representative vector of the target statement vector of the target question, which is closest to the target statement vector, the sub-intention category to which the target statement vector belongs can be determined.
Specifically, the server may cluster the question vector sets corresponding to the target semantic intentions in advance to obtain the representative vectors corresponding to the respective sub-intention categories, or may cluster the question vector sets corresponding to the target semantic intentions after obtaining the target questions to be answered to obtain the representative vectors corresponding to the respective sub-intention categories. For example, questions and paired answers are stored in the question-answer pair database, semantic intentions corresponding to the questions in the question-answer pair database can be obtained, the question vectors corresponding to the questions belonging to the same semantic intention are clustered and divided into a plurality of cluster categories, and the questions corresponding to the question vectors of the same cluster category belong to the same sub-intention. In this way, when the target semantic intention of the target question to be answered is obtained, the sub-intention representative vectors of the sub-intention categories corresponding to the target semantic intention can be obtained, and the sub-intention representative vectors matched with the target statement vectors can be obtained from the sub-intention representative vectors. For example, assume that there are two semantic intents, one for pneumonia protection and one for pneumonia characteristics. The semantic intent of the question-answer to each question in the database may be obtained. And then clustering a question vector set corresponding to the pneumonia protection with the semantic intention in a question-answer pair database to obtain a plurality of sub-intention categories corresponding to the pneumonia protection, and clustering a question vector set corresponding to the pneumonia protection with the semantic intention in the question-answer pair database to obtain a plurality of sub-intentions corresponding to the pneumonia characteristics.
In some embodiments, when clustering the problem vector set, the clustering effect may be determined based on a perplexity (perplexity), which is inversely proportional to the performance of the clustering, and a smaller perplexity means a higher efficiency and a better classification effect.
In some embodiments, a keyword (keyword) with an importance degree greater than a preset importance degree or the importance degree is the largest in each clustering vector set may also be obtained as a name of a sub-intention category corresponding to the clustering vector set. The importance may be expressed in terms of Document-inverse Document Frequency (TF-IDF)). For example, assuming that the target semantic intention is "mask", the problem vector set corresponding to the problem belonging to the primary semantic intention of "mask" may be divided into 9 categories, that is, 9 secondary intentions, by means of k-means clustering, and the distribution of keywords in each category is statistically analyzed, and the keywords are used as the names of sub-intentions, for example, the sub-intentions may include mask selection, mask purchase, mask abandonment, and the like.
In some embodiments, the step of clustering the problem vector set corresponding to the target semantic intention to obtain the representative vector corresponding to each sub-intention category includes: acquiring a problem set corresponding to the target semantic intention, acquiring statement vectors corresponding to all problems in the problem set, and forming a problem vector set; clustering the problem vector sets to obtain clustering vector sets corresponding to all sub-intention categories of the target semantic intention; and acquiring a representative vector of the clustering vector set corresponding to the sub-intention category as a sub-intention representative vector corresponding to the sub-intention category.
The clustering vector set is a sub-vector set obtained by clustering and dividing the problem vector set. The problem vector set corresponding to one semantic intention can be divided into a plurality of clustering vector sets. The representative vector may be a vector in the cluster vector set, or may be obtained by counting the vectors in the cluster vector set, for example, the vectors in the cluster vector set may be added and then divided by the number of vectors in the cluster vector set, and the obtained vector may be used as the representative vector. Or a vector corresponding to the center of the cluster (i.e., a central point vector of the cluster vector set corresponding to the sub-intention category) may be obtained as a sub-intention representative vector corresponding to the sub-intention category. The central point vector is the vector with the minimum sum of distances from each vector in the cluster vector set. For example, assume there are 3 vectors in the cluster vector set: a1, a2, and A3. Assuming that the distance from a1 to a2 is 1, the distance from a2 to A3 is 2, and the distance from a1 to A3 is 1.5, the sum of the distances from a1 to the vectors in the cluster vector set is 1+1.5 — 2.5. The sum of the distances from a2 to each vector in the cluster vector set is 1+ 2-3, and the sum of the distances from A3 to each vector in the cluster vector set is 2+ 1.5-3.5. The sum of the distances of the vectors in the a1 and the corresponding clustering vector set is the minimum, so a1 can be taken as the representative vector in the clustering vector set.
Specifically, for the target semantic intention, a question corresponding to the target semantic intention in the question-answer pair database may be obtained, so as to obtain a question set. The method comprises the steps of obtaining problem vectors of all problems in a problem set, forming a problem vector set, clustering the problem vector set to obtain a plurality of clustering categories, namely the problem vector set can be divided into a plurality of sub-vector sets (clustering vector sets), and one sub-vector set corresponds to one sub-intention. The representative vector corresponding to the set of sub-vectors may be obtained as a vector representing the set of sub-vectors, and may also be referred to as a representative vector representing the sub-intention category corresponding to the set of vectors.
Step S208, the sub-intention corresponding to the target sub-intention representative vector is taken as the target sub-intention, and the question matched with the target sub-intention is obtained and taken as the reference question.
Specifically, the intent of the reference question may be obtained in advance. For example, the reference question may be a question in a question-answer pair database. The semantic intentions of questions in a database are determined in advance, and a question vector set with the same semantic intention is clustered to obtain a sub-intention corresponding to each question. Therefore, after obtaining the target sub-intention representative vector, the server may obtain the sub-intention corresponding to the target sub-intention representative vector as the target sub-intention, and obtain the question and answer corresponding to the target sub-intention in the database as the reference question. Namely, the question with the semantic intention as the target semantic intention and the sub-intention as the target sub-intention can be obtained from the question-answer pair database and used as the reference question.
In some embodiments, when the number of the questions matched with the target sub-intention is multiple, each question may be used as a reference question, and the questions matched with the target sub-intention may be further filtered. For example, if the target semantic intention is called a primary intention and the target sub-intention is called a secondary intention, the target semantic intention can be further divided into three levels of intentions, and intentions matching the primary intention, the secondary intention and the tertiary intention of the target question are obtained as a reference question. The method may further include obtaining a problem matching the primary intention, the secondary intention, and the tertiary intention of the target problem, and then performing further screening, for example, calculating a distance between vectors of the target problem and the problem matching the intentions, and using the problem with the smallest distance as a reference problem.
In some embodiments, the tertiary intent may be a key condition intent, and the problem matching the target sub-intent is obtained as a reference problem, including: acquiring a named entity set corresponding to the target problem, and acquiring a key condition intention corresponding to the target problem according to the named entity set; and acquiring the questions matched with the key condition intents and the target sub-intents from the candidate question set as reference questions.
The Named entity (Named-entity) refers to an entity having a specific meaning, and may include at least one of a name of a person, a name of a place, a name of an organization, a proper noun, or the like. The named entities in the set of named entities can be one or more. The named entities in the named entity set corresponding to the target question may be named entities included in the target question, or may be further obtained based on the named entities included in the target question. For example, at least one of a child named entity or a parent named entity corresponding to a named entity included in the target question may be acquired as the named entity corresponding to the target question. The child or parent named entity corresponding to the named entity in the target problem may be derived based on a knowledge graph. For example, if the target question includes a named entity "KN 90", and the parent named entity of "KN 90" can be obtained as a "mask" according to the top-bottom relationship of the named entity in the knowledge map, the "mask" can be used as the named entity corresponding to the target question. The corresponding named entity is obtained by linking the entity in the knowledge graph with the problem of the user, so that the obtained named entity is more accurate.
The key condition intention refers to an intention for indicating a key condition of the question, and the key condition intention can be obtained according to at least one of a word slot or a named entity, for example, can be obtained by combining at least one of the word slot or the named entity. The term slot may be a generic term for a named entity, which may also be referred to as a concept. A concept is the expression of entities having a common trait. A named entity refers to a specific thing to which a concept corresponds. For example, "mask" may be the word slot, which is a generic concept for the named entities "N95" and "KN 94," vehicle "may be the word slot, which is a generic concept for" cars "and" trains. The key condition intents can be obtained by combining word slots corresponding to the named entity set, combining named entities in the named entity set or combining the word slots and the named entities. For example, for the target question "how a mask worn on a train can protect against pneumovirus", the resulting set of named entities includes two named entities "train" and "mask", then the key condition intent may be "train and mask", or "vehicle and mask".
Specifically, the server may perform Named Entity Recognition (NER) on the target problem, resulting in a set of Named entities. After the key condition intents corresponding to the target questions are obtained according to the named entity set, questions with the sub-intents as target sub-intents and the condition intents as key condition intents corresponding to the target questions can be obtained from the question-answer database and used as reference questions.
In the embodiment of the application, the scene information required in the question answering process can be obtained through the named entity set, the reference problem is determined based on the scene information, the key condition intention of the reference problem can be consistent with that of the target problem, and therefore the more matched reference problem can be obtained.
Fig. 4 is a schematic diagram of the intent matching in some embodiments. For questions in the question-answer pair database, semantic intention recognition can be performed in advance by using a semantic intention recognition model to obtain semantic intentions (primary intentions) corresponding to the questions, and then problem vectors corresponding to the questions belonging to the same semantic intention can be clustered based on a K-means clustering method, wherein one cluster corresponds to one sub-intention. After clustering, a keyword mining method based on word frequency can be used for obtaining the keyword with the highest word frequency in the problem set corresponding to the sub-intention as the name of the sub-intention. Thereby obtaining the secondary intention of the question-answer pairs in the database. The problems in the same secondary intention can be identified by named entities based on the upper and lower relations of the knowledge graph, the obtained named entities are combined to obtain the key condition intention (tertiary intention) under the secondary intention, and the corresponding tertiary intentions are consistent for the problems corresponding to the same named entity combination. For example, if 2 problems in the secondary intention "pneumonia protection" are "N95 can protect against pneumonia on a vehicle" and "mask can protect against pneumonia on an automobile", and according to the upper-lower relationship of named entities of the knowledge graph, "automobile" belongs to "vehicle" and "N95" belongs to mask, then the named entities of the two problems can be both "vehicle" and "mask", and then the corresponding tertiary intentions of the two problems are "vehicle and mask".
For the target question, semantic intention recognition can be performed by using an intention recognition model to obtain a target semantic intention (primary intention) corresponding to the target question, and since the vectors of the question in the database are clustered in advance by the question and the sub-vector sets corresponding to the sub-intents are obtained, the clustering center corresponding to each sub-vector set under the target semantic intention can be obtained, the clustering center with the minimum distance of the target statement vector corresponding to the target question is determined, and the sub-intention category to which the clustering center belongs is taken as the target sub-intention. In addition, named entity extraction can also be performed on the target question, for example, assuming that the target question is "KN 94 mask can be used for pneumonia protection on the vehicle", which has been given its secondary intention of "pneumonia protection". Then KN94 belongs to "mask" according to the superior-inferior relation of the knowledge-graph, and the third level of intent of the target problem is "vehicle and mask". Therefore, "can protect against pneumonia on a vehicle" by N95 and "can protect against pneumonia on an automobile" by a mask can be obtained as problems matching the intention of the target problem. Further, a question whose question vector is the smallest in distance from the target sentence vector of the target question may be acquired as a reference question from questions matching the intention of the target question based on the distance between the vectors of the questions. For example, assuming that the distance between the sentence vector of "can protect against pneumonia on the vehicle" N95 and the sentence vector of "can protect against pneumonia on the vehicle" KN94 mask is the smallest, the question "can protect against pneumonia on the vehicle" N95 can be taken as a reference question.
Step S210, obtaining a target answer to the target question according to the reference answer corresponding to the reference question.
Specifically, the answer to the reference question is a reference answer, and the answer matching the reference question is predetermined, for example, stored in the question-answer pair database. The server may use the answer to the reference question as the answer to the target question, or may rewrite the reference answer to obtain the answer to the target question.
According to the intelligent question-answering method, the target semantic intention related to the semantics of the target question can be obtained for the target question to be answered, the sub-intention representative vector is obtained by clustering the question vector set corresponding to the target semantic intention, so that the sub-intention representative vector matched with the target statement vector is obtained to serve as the target sub-intention representative vector, the sub-intention corresponding to the target sub-intention representative vector serves as the target sub-intention, the target sub-intention which meets the semantics of the target question and has fine granularity can be accurately obtained, the target answer is obtained according to the answer corresponding to the reference question matched with the target sub-intention, the target answer is matched with the intention of the target question, and the obtained answer is high in accuracy.
In some embodiments, the questions in the question-answer pair database may be manually input, or may be automatically mined, for example, a new question may be automatically mined, and an answer corresponding to the new question may be obtained, so as to obtain a new question-answer pair, which is added to the question-answer pair database.
In some embodiments, the answers to the questions in the database by the questions and answers may be determined manually or automatically augmented by the server. For example, after obtaining a new question, a search may be performed using the new question, for example, on the internet, to obtain a candidate answer corresponding to the new question. And evaluating the quality of the candidate answer, acquiring the candidate answer with the quality meeting the conditions, taking the candidate answer as a standard answer of the new question, and adding the standard answer into a question-answer pair database. The quality satisfaction condition may be, for example, that the quality score is greater than a preset score, such as 0.95.
In some embodiments, a quality score corresponding to the candidate answer may be determined based on a quality assessment model, the quality score may represent a confidence of the answer, and the greater the quality score, the greater the confidence. For example, features corresponding to questions and features corresponding to answers may be extracted and input into a quality assessment model, which outputs corresponding quality scores. The quality evaluation model may be obtained by pre-training, and may include a plurality of quality evaluation models, for example, the quality evaluation model may include at least one of a bilst model or an xgboost (extreme Gradient boosting) model. When a plurality of quality evaluation models exist, the quality scores output by the quality evaluation models can be weighted to obtain the final quality scores, and the weights can be set according to needs. The quality assessment model may be pre-trained. For example, seed data may be obtained, the seed data including a question and an answer, the seed data may be considered data in which the question is paired with the answer, such as an answer from an authoritative website, which may be considered a standard answer, i.e. authentic. For example, for a medical question and answer system, the authoritative website may be a government website that provides health and wellness services. The corresponding quality score for the seed data may be the maximum quality score, e.g., 1, indicating a confidence of 1. And performing supervised model training based on the features of the seed data and the quality scores to obtain a quality evaluation model.
In some embodiments, the feature extraction may be performed by using a deep learning model, which may be trained in advance to obtain a deep learning model for quality evaluation of answers to questions, for example, a deep learning model based on a BERT structure. The target quality evaluation model for evaluating the quality of the answers to the questions may be a non-deep learning model, and after the deep learning model is obtained through training, the questions may be input into the deep learning model, and the features output by the feature extraction layer of the deep learning model are obtained and used as the features input into the non-deep learning model. Therefore, the accurate characteristics implicit in the data extracted by the deep learning model can be obtained, and the effect of the model can be explained by utilizing the strong interpretability of the non-deep learning model.
In some embodiments, the features input to the non-deep learning model may further include other features, for example, for a question and answer in the medical field, a plurality of features closely related to quality assessment may be induced manually according to a large amount of medical article data, and the entity features and the relationship features between entities of the questions and the corresponding answers may be extracted based on a knowledge graph in a relationship extraction manner, and the features extracted by the deep learning model are input to the non-deep learning model for quality assessment. The knowledge graph comprises entities and relations between the entities, the entities in the questions and the answers can be obtained based on the knowledge graph, and the relation characteristics between the entities are obtained through the relations between the entities in the knowledge graph, so that the characteristics extracted based on the knowledge graph have the advantage of strong interpretability.
FIG. 5 is a schematic diagram of obtaining features input to a quality assessment model in some embodiments. In fig. 5, the questions and answers may be obtained and input into a deep learning model based on the BERT network mechanism, and a plurality of features may be output by the deep learning model. The questions related to the question and answer of the disease are answered by answers, and the questions and the answers can be extracted based on the knowledge graph to obtain characteristics related to symptoms of the disease, characteristics related to treatment, disease names, and characteristics of subclass diseases or parent diseases corresponding to the disease, and the characteristics are input into a target quality evaluation model (such as a BilSTM model and an XGboost model) to obtain the confidence of the answers corresponding to the questions. In the BERT model, "CLS" represents a starting identifier of a text, "TOK N" represents the Nth word in the training problem, E represents a word vector, and C represents a classification result. T represents the feature vector obtained by the BERT model.
In some embodiments, the quality score may also be obtained based on a weighted multi-index check, for example, the quality score of the answer corresponding to the question may be obtained by combining the quality score of the answer obtained based on the quality evaluation model and the quality score of the answer corresponding to the question obtained by other methods. For example, as shown in fig. 5, the answers to the questions may be scored based on at least one of information sources, information original sources, keyword relevance, entity relevance, remote Supervision (distance Supervision), and a relationship network, and the quality scores obtained in each manner and the corresponding weights may be weighted and summed to obtain the quality score corresponding to the answers to the questions. The information source may refer to a website that crawls answers to questions. The original source of the information refers to a website for capturing an answer to a question to obtain an original source of the answer, for example, the website may be an article for transferring the answer from the website a, and the website a may be used as the original source of the information. The keyword relevance refers to the relevance of a search keyword for searching for an answer to a document in which the answer appears. The entity relevance refers to the relevance between the entity key words for searching the answers and the documents in which the answers appear. The closeness degree of the relationship can be obtained based on the relationship network, and the greater the closeness degree of the relationship, the higher the corresponding quality score. For example, the closer the relationship between the question and the entity in the answer, the higher the score obtained. For example, the answers to the questions may be subjected to information source reliability verification, information original source reliability verification, keyword-based relevance verification, entity-based relevance verification, remote supervision-based credibility verification, database-based relevance article weighting-based credibility verification, relation network inference-based confidence verification, and the like, and the quality scores corresponding to the answers to the questions may be obtained by combining the scores output by the quality assessment models, such as the BiLSTM model and the XGBoost model. The quality scores obtained through multiple modes are subjected to weighted calculation, which is equivalent to multiple answer verification, so that the server can find out high-quality answers related to new questions from the Internet, the scale of the question-answer pair database is automatically expanded, the knowledge of a question-answer system is guaranteed to be always dynamic following events, answers are timely provided for users, and the reliability of answers of the questions is guaranteed.
In some embodiments, a target semantic intent related to the semantics of the target question may be identified by a trained first intent recognition model. For example, semantic coding can be performed on the target problem based on a semantic coding model in a trained first intention recognition model to obtain a semantic coding vector; and performing intention decoding on the semantic coding vector based on a semantic decoding model in the trained first intention recognition model to obtain a target semantic intention. As shown in fig. 6, the training step of the trained first intention recognition model includes:
step S602, performing intention recognition on the training question based on the first intention recognition model to be trained to obtain a first intention recognition result.
The first intention recognition model is a semantic intention recognition model which is used for carrying out semantic analysis on the problem and obtaining semantic relation with the semantics of the problem. For example, the first intention recognition model may be a BilSTM (Bi-directional Long Short-Term Memory neural network) based model. The first intent recognition result may be a probability distribution of the intentions of the training problem as respective candidate semantic intentions. The training problem refers to a problem for model training. For example, the candidate semantic intent may include pneumonia protection and pneumonia characteristics, etc.
Specifically, the training problem may be segmented to obtain a word sequence, word vectors corresponding to words in the word sequence are obtained, the word vectors are input into a first intention recognition model to be trained, and the intention of the training problem is output as probability distribution of each candidate semantic intention.
Step S604, performing intention recognition on the training question based on the second intention recognition model to obtain a second intention recognition result.
The second intention recognition model may also be a semantic intention recognition model for performing semantic analysis on the problem to obtain semantic relations with the semantics of the problem. For example, the second intent recognition model may be a model based on the BERT network structure. The second intent recognition result may be a probability distribution of the intentions of the training problem as respective candidate semantic intentions. The second intention recognition model may be trained in advance, or may be trained together with the first intention recognition model.
Specifically, the training problem may be segmented to obtain a word sequence, a word vector corresponding to each word in the word sequence is obtained, the word vector is input into the second intention recognition model to be trained, and the intention of the training problem is output as the probability distribution of each candidate semantic intention. Namely, for the same training question, semantic intention analysis can be performed on the training question by using the first intention recognition model and the second intention recognition model.
Step S606, obtaining a model loss value of the first intention recognition model to be trained based on the difference between the first intention recognition result and the second intention recognition result, and adjusting model parameters in the first intention recognition model to be trained according to the model loss value to obtain the trained first intention recognition model.
Wherein the model loss value and the difference have a positive correlation relationship, that is, the difference between the first intention recognition result and the second intention recognition result positively affects the model loss value. If the other conditions are not changed, the difference between the first intention recognition result and the second intention recognition result becomes large, and the obtained model loss value becomes larger. The calculation method of the model loss value may be set as required, and the model loss value may be calculated according to a logarithmic loss function or a Square loss function (Square loss), for example. When adjusting the model parameters, the adjustment is performed in a direction to decrease the loss value, and the adjustment may be performed by using a random gradient decreasing method. It is to be understood that the first intention recognition model may be subjected to multiple parameter adjustments until the model converges, resulting in a trained first intention recognition model. The model convergence may be, for example, that the model loss value is less than a preset loss value.
In some embodiments, a first loss value may be derived from a difference of the first intent recognition result and the second intent recognition result; obtaining a second loss value according to the difference between the first intention recognition result and a standard intention recognition result corresponding to the training problem; and obtaining a model loss value of the first intention recognition model to be trained according to the first loss value and the second loss value.
The standard intention recognition result refers to a correct intention recognition result corresponding to the training problem. The standard intent recognition result may be, for example, a probability that the semantic intent of each training question corresponds to each candidate semantic intent, where the probability that the correct semantic intent (i.e., the intent tag) corresponds to the training question is 1.
Specifically, the first loss value and the difference between the first intention recognition result and the second intention recognition result are in a positive correlation. The second loss value is in positive correlation with the difference between the first intention recognition result and the standard recognition result corresponding to the training question. The model loss value may be obtained by performing weighted summation on the first loss value and the second loss value, and the weights respectively corresponding to the first loss value and the second loss value may be set according to a setting, and may be 0.5, for example. By making the first loss value and the difference between the first intention recognition result and the second intention recognition result in a positive correlation. The second loss value and the difference between the first intention recognition result and the difference between the standard recognition results corresponding to the training questions are in a positive correlation relationship, and the first intention recognition model can be adjusted in parameters in a direction in which the difference between the first intention recognition result and the second intention recognition result and the difference between the first intention recognition result and the difference between the standard recognition results corresponding to the training questions become smaller, so that the first intention recognition model can learn the learning ability of the second intention recognition model and the difference between the result obtained by the first intention recognition model and the standard intention recognition result is small.
In the embodiment of the application, the model loss value of the first intention recognition model to be trained is obtained based on the difference between the first intention recognition result and the second intention recognition result, the model loss value and the difference are in positive correlation, the learning capability of the first intention recognition model can be close to the learning capability of the second intention recognition model, and therefore the learning capability of the first intention recognition model can be improved towards the expected intention recognition capability.
In some embodiments, the second intent recognition model may be a learning-intensive model. The first intention recognition model can be a time-efficient model, and semantic intentions corresponding to the problems can be quickly recognized and obtained. I.e. the first intention recognition model has a higher recognition efficiency than the second intention recognition model, which has a higher learning ability, e.g. a higher recognition accuracy, than the second intention recognition. For example, the second intent recognition model may be a model based on the structure of a BERT network and the first intent recognition model may be a model based on the structure of a BiLSTM network. The model based on BERT can effectively learn the semantics in the characters, but the time consumption is very long, so that the requirement of the online service on the corresponding duration is difficult to meet. BilSTM is a more traditional network structure, which has inferior learning capabilities compared to BERT model, but this structure has the advantage of time efficiency. Therefore, in the training process, two intention recognition models are simultaneously used for intention recognition, the probability distribution of BERT output is simulated by using the BilSTM, and in the prediction process, the intention recognition is carried out by using the BilSTM, so that the learning capability of the BilSTM can be effectively improved by means of the learning capability of the BERT, and the time efficiency advantage of the BilSTM is utilized, so that the recognition efficiency is high.
In some embodiments, for the problems of too many categories of semantic intents and unbalanced data, a data enhancement mode can be adopted to solve the problem of unbalanced data. The training problem may be processed, for example, in at least one of synonym replacement, removal of stop words, random insertion, or random deletion of words, to obtain further training problems.
In some embodiments, when calculating the model loss value, the weight occupied by the class with better intention classification result can be reduced, and the weight occupied by the class difficult to classify can be increased, so that the class difficult to classify can be better trained to solve the class imbalance problem. For example, in model training, for the training problem with the intention label Y1, the probability p of the semantic intention Y1 output by the intention recognition model is assumed to betThen the weight corresponding to the probability can be obtained according to the difference between the semantic intention and 1, according to ptThe obtained loss value focal loss FL (p)t) The loss value can be expressed as formula (1), and can be used as the loss value of the model, or can be obtained by combining the loss values obtained by calculation of other training problems. γ is an adjustment factor and may be a value greater than 0. Thus, when p istThe closer to 1, the smaller the calculated loss value. The weight occupied by the class with better intention classification result can be reduced.
FL(pt)＝-(1-pt)γlog(pt) (1)
In some embodiments, more questions with intention labels can be obtained by using an active learning method, so as to increase the number of training questions and improve the effect of model training. For example, after the first intention recognition model is trained for multiple times based on a training question with artificially labeled semantic intents, an intention-unlabeled question may be input into the first intention recognition model, and if the probability corresponding to the semantic intention output by the first intention recognition model is greater than a preset probability, for example, 0.9, the semantic intention corresponding to the probability may be used as the intention of the question, so as to obtain an intention-labeled question as a training question for training the first intention recognition model. Therefore, the problems needing to be labeled can be automatically selected from the problems without labeling intentions and labeled automatically, and the number of training samples is expanded. It can be understood that the problem of automatic intention labeling can be detected by utilizing a manual extraction part so as to ensure the accuracy of the automatic labeling intention.
Fig. 7A is a schematic diagram of a training principle of training a first intention recognition model and a second intention recognition model in some embodiments, and fig. 7B is a schematic diagram of a structure of a BiLSTM model. The second intention recognition model is based on a model of the BERT network structure, and the first intention recognition model is based on a model of the BiLSTM network structure. After the training problem is obtained, the word sequence corresponding to the training problem can be input into a BilSTM model, the BilSTM carries out bidirectional coding (forward coding and backward coding) on the word vector of the word sequence, the coding vector is obtained, attention-based weight is obtained based on an attention mechanism, the vector obtained by coding is processed based on the attention weight to obtain a weight processing vector X, the weight processing vector is input into a full connection layer, and then normalization is carried out through a softmax layer to obtain probability distribution (first intention identification result), similarly, the training problem can be input into a model based on a BERT network structure to obtain probability distribution (second intention identification result), then a loss value of an initial model can be obtained based on the difference between the first intention identification result and the second intention identification result through a loss matching layer, and the loss value of the initial model is calculated based on a formula (1), and obtaining a final model loss value, and adjusting the model parameters of the first intention identification model based on the final model loss value.
In some embodiments, the server may also mine new questions to refine the question-answer pairs in the database to improve the accuracy of the answers. As shown in fig. 8, the step of performing new problem mining may include:
step S802, a target word corresponding to the candidate new question is obtained.
Wherein, the candidate new question is a question that the question has not yet determined whether it is a new question. A new question may refer to a question that lacks a matching accurate answer in the question-answer pair database. For example, a question that is not satisfied with the answer and fed back by the user may be used as a candidate new question, each obtained target question to be answered may be used as a candidate new question, or a hot question may be obtained as a candidate new question, and the hot question may be, for example, a question that is asked for more than a preset number of times.
The target words may be all words in the candidate new question, or words satisfying a predetermined condition in the candidate new question, such as adjectives or nouns. There may be one or more target words.
Specifically, after the candidate new question is obtained, the candidate new question may be segmented to obtain a target word corresponding to the candidate new question.
Step S804, a first number of questions including the target word in the first question set is obtained, and there are matching answers to the questions in the first question set.
Wherein the first set of questions may include a plurality of first questions. The questions in the first question set have matching answers, i.e. the questions in the first question set can be considered to have accurate answers. For example, questions in the question-answer pair database may be used as the first question to form the first question set.
Specifically, the first number may be obtained by counting the questions including the target word in the first question set. For example, assuming that the target word is "coronavirus", the number of questions including "coronavirus" in the first question set is acquired as the first number.
Step S806 determines a first novelty of the candidate new question in the first question set according to the first quantity, the first quantity having a negative correlation with the first novelty.
The novelty represents the degree of novelty, and the greater the novelty, the more novel the problem. A negative correlation indicates that the effect of the first quantity on the first novelty is negative, and under the same other conditions, the larger the first quantity, the smaller the first novelty, and the less novel the problem. I.e., the fewer the number of questions in the first question set in which the target word appears, the more novel the candidate new question is in the first question set.
Specifically, the server may use the first number as the first novelty, or may perform further calculation according to the first number, for example, calculate an inverse document frequency corresponding to the target term, and calculate the first novelty of the candidate new question in the first question set according to the inverse document frequency corresponding to the target term. The inverse document frequency may be taken as the first novelty, for example. Or calculating the word frequency-inverse document frequency corresponding to the target word as a first novelty, wherein the word frequency-inverse document frequency is the product of the word frequency and the inverse document frequency. The inverse document frequency may be derived from the total number of questions in the first question set and the number of questions (first number) that include the target term. The word frequency represents the frequency with which the target word appears in the candidate new question. For example, the first novelty may be expressed in terms of word frequency-inverse document frequency, as expressed by equations (2), (3), and (4). Wherein tf _ idf represents the word frequency of the target word-the inverse document frequency, and tf represents the word frequency of the target word. idf represents the inverse document frequency of the target term. n represents the number of occurrences of the target word in the candidate new question. N represents the number of words in the candidate new question. | D | represents the total number of questions in the first set of questions. p denotes the number of questions in the first question set, including the target word, i.e., the first number, and k may be any number, and may be 1, for example.
tf_idf＝tf*idf (2)
In some embodiments, when there are a plurality of target words in the candidate new question, the novelty corresponding to each target word may be calculated, and statistics may be performed to obtain a first novelty corresponding to the candidate new question. For example, the average or the maximum of the novelty degrees corresponding to the target word is used as the first novelty degree corresponding to the candidate new question.
And step S808, when the first novelty is larger than a first threshold value, taking the candidate new question as a target new question.
Specifically, the target new question refers to a question that is ultimately determined to be a new question. The first threshold value may be set as desired, for example, 0.5. Since the first quantity is inversely related to the first novelty, the smaller the first quantity, the greater the first novelty. The smaller the first number, the more novel the candidate new question is in the first set of questions, the less likely there is a matching answer. The candidate new problem may thus be targeted when the first novelty is greater than the first threshold.
After the new target question is obtained, answer mining may be performed on the new question, for example, searching is performed in the internet, a matching answer matched with the new target question is obtained, and the new target question and the matching answer are added to the question-answer pair database.
In some embodiments, a second number of questions in a second question set including the target word may also be obtained, where the questions in the second question set are questions for which no matching answer exists; determining a second novelty of the candidate new question in the second question set according to the second quantity, wherein the second quantity is in positive correlation with the second novelty;
wherein the second set of questions may include a plurality of second questions. The questions in the second set of questions are questions for which there are no matching answers. For example, the question fed back by the user that is not satisfied with the answer may be a question that is considered that there is no matching answer, and the question fed back by the user that is not satisfied with the answer may be added to the second set of questions. The method for determining the second novelty of the candidate new question in the second question set according to the second quantity may be the same as the method for determining the first novelty of the candidate new question in the first question set according to the first quantity, and is not repeated herein. The greater the second number of target words included in the second set of questions, the more common the words are in the second set of questions, and thus the second number is inversely related to the second novelty.
In some embodiments, when the first novelty is greater than the first threshold, the candidate new question is taken as a target new question, and obtaining a matching answer to the target new question comprises: and when the first novelty is greater than a first threshold and the second novelty is less than a second threshold, taking the candidate new question as a target new question, obtaining a matching answer of the target new question, wherein the second number and the question novelty degree of the candidate new question have positive correlation.
The second threshold may be set as desired, and may be, for example, 0.3. Since the questions in the second question set are questions for which no matching answer exists, the smaller the second novelty, the less novel the candidate new question is in the second question set, i.e., the more likely the candidate new question is a frequently occurring question for which no matching answer exists. Since the questions in the first question set are questions with matching answers, the larger the first novelty is, the more novel the candidate new questions are in the first question set, and it is difficult to find matching answers in the answer set corresponding to the first question set. Therefore, when the first novelty is greater than the first threshold and the second novelty is less than the second threshold, the candidate new question is used as a target new question, an accurate new question can be found, a matched answer is obtained based on the new question and is added into the question pair database, question-answer pairs in the question-answer database can be enriched, and answers to newly-appeared questions can be crawled in time.
The intelligent question answering method provided by the embodiment of the application can be applied to medical question answering, for example, to question answering of epidemic situations. Because of the highly spoken nature of a user's questions, it is often difficult to accurately identify what the user wants to ask, and therefore often times a literally close but significantly different question-and-answer pair is matched. The semantic expression of characters is complex, and the great change of the semantics is usually brought by the change of the sequence of the characters or the addition and deletion of a certain keyword. By the intelligent question-answering method provided by the embodiment of the application, the question sentences in the medical field can be divided into three levels of fine-grained intentions, and the questions matched with the intentions of the questions of the user are searched in the question-answering database based on intention matching, so that the ability of understanding the questions of the user is improved, and the performance of the question-answering robot in the epidemic situation field is improved.
Fig. 9 is a schematic diagram illustrating a processing principle of the intelligent question answering method in some embodiments, and includes 4 function information modules and 4 data resources. The 4 functional modules are respectively an authority information acquisition module, a hot spot problem mining module, an intelligent question-answering module and a personalized recommendation module. The 4 data resources are internet information data, knowledge map data, log data and question-answer pair databases respectively, wherein the internet information data are public data acquired on the internet, the question-answer pair databases store questions and answers which are subjected to structural processing, such as questions and answers related to an epidemic situation, and the knowledge map data comprise entities and relations between the entities, such as entities related to the epidemic situation and relations between the entities. For example, the entities may be drugs, protective tools, symptoms, and the like. Log Log data is the historical behavior record of the user in the epidemic situation question-answering system. The following describes, with reference to fig. 9, an intelligent question answering method provided in the embodiment of the present application, including the following steps:
1. and acquiring a new question, crawling information according to the new question, and acquiring an answer corresponding to the new question.
Specifically, the new question may be input by the user, or may be a question missing in the database of the current question-answer pair selected by an algorithm based on tf-idf. For example, the server may obtain a word corresponding to the candidate new question as a target word; acquiring a first number of questions including target words in a first question set, wherein the questions in the first question set have matching answers; determining a first novelty of the candidate new question in the first question set according to the first quantity, wherein the first quantity is in a negative correlation relation with the first novelty; and when the first novelty is larger than a first threshold value, taking the candidate new problem as a target new problem.
The server can acquire new questions through the authoritative information acquisition module, automatically acquire answers on authoritative websites related to the epidemic situation on the Internet in real time according to the new target questions, and then store the answers in the question-answer pair database for other functional modules to use, so that the timeliness and the reliability of the information of the intelligent question-answer technology are ensured. For example, the authority information collection module may include 6 functional units. Wherein the new problem discovery functional unit is used to discover new problems. The automated real-time crawling function unit can crawl information using a given-based multi-threaded crawler technology. And the multi-source data verification functional unit verifies the crawled data through a cross validation algorithm to remove the data with low credibility. And the data cleaning functional unit eliminates irrelevant information in the webpage data through the regular expression. The data formatting function may automatically tag the problem with an intent that is classified using a BERT based classification algorithm. The question-answer pair warehousing functional unit stores the new questions and the corresponding answers in a persistent mode by using the mongoDB-based database technology.
2. And storing the new question and the corresponding answer to a question-answer pair database.
3. And solving the target question to be answered, and outputting the answer of the question to the terminal.
Specifically, the server may obtain a target question to be answered, identify an intention of the target question, and obtain a target semantic intention, a target sub-intention, and a key condition intention. And acquiring questions with intentions of target semantic intentions, target sub-intentions and key condition intentions from the question-answer pair database as reference questions. For example, assuming that the semantic intent of the target is pneumonia prevention measure, the sub-intent of the target is mask protection, and the key condition intent is "vehicle and air", the question-answer pair database may be obtained, the semantic intent is also pneumonia prevention measure, the sub-intent is mask protection, and the key condition intent is "vehicle and air", as a reference question, and the answer corresponding to the reference question in the question-answer pair database is obtained as the answer corresponding to the target question to be answered.
Before answering the questions to be answered, the server can also acquire the intentions of the questions and answers to the questions in the database in advance. The intent corresponding to the question may include a semantic intent at one level, as well as sub-intents (secondary intents) and key condition intents (tertiary intents) under the semantic intent. For example, the server may identify semantic intentions corresponding to the questions in the database based on the first intention identification model to obtain the semantic intentions. And clustering the problem vector sets corresponding to the problems with the same semantic intention, wherein one clustering class corresponds to one sub-intention. One sub-intention can correspond to a plurality of key condition intentions, and a named entity set corresponding to each cluster can be obtained to obtain the key condition intentions corresponding to each question in the question and answer pair database.
For example, the intelligent question-answering module in fig. 9 may be used to obtain answers to questions to give corresponding answers to the questions input by the user. The intelligent question-answering module can comprise an establishing intention category system function unit, a spelling error correction function unit, a standardized rewriting function unit, an intention identification function unit, a fine-grained intention identification function unit and a difficult intention identification function unit. The functional unit for establishing the intention category system establishes a 3-level intention system to divide the intention types of the user problems through the analysis of the user problems. Since the input of the user may have spelling errors or the input of the voice has recognition errors, the language model can be obtained and stored in the spelling correction function unit, and the spelling errors in the problem of the input of the user can be automatically corrected by using the language model and the homophone replacement and the synonym replacement. The standardized rewrite function may unify entity names in the user-entered question by knowledge inference of alias relationships and superior-inferior relationships based on the medical knowledge map. The intention recognition functional unit carries out distillation learning on the BERT model to improve the time efficiency of the model, and combines the characteristics of epidemic situation data to determine the loss value of the model by using focalloss technology, so that the problem of data class imbalance is relieved, and meanwhile, the accuracy of the model is effectively improved by expanding training data by using an active learning mode. The fine-grained intention recognition functional unit is based on a symptom map, a drug map and a disease map in a knowledge map, accurately answers the user questions by using a template matching algorithm based on an entity slot position through an NER technology, and automatically learns and expands a matching template by using Bootstrap. Bootstrap is also called self-service. It is a uniform sampling with a drop-back from a given training set, i.e. whenever a sample is selected, it may be reselected and added again to the training set, etc. The difficult intention recognition functional unit approximately answers the question containing a plurality of different intentions by combining a text relevance algorithm, an entity relevance algorithm and an algorithm of source credibility scoring.
4. The questions and answers are stored to log data.
5. Personalized question recommendations are made based on questions input by the user.
For example, the personalized recommendation module may obtain a question input by the user and perform personalized recommendation based on the question input by the user. For example, the server acquires a question similar to a question posed by the user, such as a question with the same target semantic intention, based on the knowledge-graph data to perform personalized recommendation.
6. Storing the personalized recommended questions to log data.
7. And mining the hot problems based on the logs in the log data, and acquiring answers corresponding to the hot problems.
Specifically, the hot spot problem mining module is used for mining hot spot problems which are not covered in a current question-answer database through log data of an analysis system, automatically finding out answers of the problems from internet information data, and then storing the answers into a question-answer pair database. The intelligent question-answering module can contain 5 functional units. The new problem screening functional unit screens out candidate problems by combining a tf-idf algorithm and a word coverage rate evaluation standard, and then votes on the candidate problems by using a keyword algorithm based on word segmentation, word frequency and an irrelevant word list and analyzes the new problems. The search engine search function unit obtains candidate answers related to the new question by calling the search engine. The quality evaluation functional unit uses a lastwise-based Lambdamat algorithm to combine with the characteristics of epidemic situation problems to rank the candidate answers according to various ranking characteristics, and selects the answer with the highest quality score from the candidate answers, wherein the ranking characteristics comprise rule-based characteristics, statistical-based characteristics and deep learning-based semantic characteristics. The new question-answer pair construction functional unit constructs the new question and the answer into a structured question-answer pair through the corresponding rule. The question-answering uses the mongoDB-based database technology to store data persistently for the warehousing functional unit. The LambaMart algorithm based on listwise is a search ranking algorithm, the listwise method is to rank all documents related to a problem, the LambaMART model can be split into two parts, namely, Lamba and MART, and the MART (multiple addition Regression Tree) is used for representing the bottom-layer training model, Lamba is the gradient used in the MART solving process, and the physical meaning of the Lamba is the direction (upward or downward) and the strength of the ranking of the next iteration of a document to be ranked.
8. And storing the hot questions and the corresponding answers to a question-answer pair database.
A system structure diagram corresponding to the method provided by the embodiment of the present application may be as shown in fig. 10, and the system structure may be divided into 4 layers: data layer, technical layer, functional layer and application layer. The data layer comprises internet information data, knowledge graph data and log data. The internet information data refers to information on a network, is mainly in a webpage form and has the advantages of large data volume and high updating speed. The knowledge-graph data can be entities and relationships related to an epidemic. Log Log data refers to historical behavior data of a user in the epidemic situation question-answering system, a technical layer shows the technology used in the project, Learninggtorank is a sequencing learning algorithm, sequencing learning is a supervised machine learning process, LDT-BERT refers to a BERT model of distillation learning, and keyword extraction algorithm. In the functional layer, a plurality of independent functional units can be designed according to the characteristics of epidemic situation data and project targets. In the application layer, a question-answering system and a personalized recommendation system can be realized according to the requirements of users. In the question-answering system, the system intelligently answers questions related to the epidemic situation, such as hundreds of questions including protective measures, infection modes and national policies, which are made by the user. In the personalized recommendation system, the related information can be recommended in a personalized manner according to the behaviors of the user, so that the user can be helped to conveniently acquire the knowledge to be inquired.
It should be understood that, although the steps in the above-described flowcharts are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the above-mentioned flowcharts may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or the stages is not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a part of the steps or the stages in other steps.
In some embodiments, as shown in fig. 11, there is provided an intelligent question answering apparatus, which may be a part of a computer device using a software module or a hardware module, or a combination of the two, and specifically includes: a target question obtaining module 1102, a target semantic intention obtaining module 1104, a target representative vector obtaining module 1106, a reference question obtaining module 1108, and a target answer obtaining module 1110, wherein:
and an objective question obtaining module 1102, configured to obtain an objective question to be answered.
And a target semantic intent acquisition module 1104 for acquiring a target semantic intent related to the semantics of the target question.
A target representative vector obtaining module 1106, configured to obtain a sub-intention representative vector matched with a target statement vector corresponding to a target question as a target sub-intention representative vector, where the sub-intention representative vector is a representative vector corresponding to each sub-intention category obtained by clustering a question vector set corresponding to a target semantic intention.
A reference problem obtaining module 1108, configured to take a sub-intention corresponding to the target sub-intention representative vector as a target sub-intention, and obtain a problem matching the target sub-intention as a reference problem.
The target answer obtaining module 1110 is configured to obtain a target answer to the target question according to a reference answer corresponding to the reference question.
In some embodiments, the module for clustering the problem vector set corresponding to the target semantic intention to obtain the representative vector corresponding to each sub-intention category includes: the question vector set acquisition unit is used for acquiring a question set corresponding to the target semantic intention, acquiring statement vectors corresponding to all questions in the question set and forming a question vector set; the clustering unit is used for clustering the problem vector sets to obtain clustering vector sets corresponding to all the sub-intention categories of the target semantic intention; and the child intention representative vector acquisition unit is used for acquiring the representative vector of the clustering vector set corresponding to the child intention category as the child intention representative vector corresponding to the child intention category.
In some embodiments, the sub-intent representative vector acquisition unit is to: and acquiring a central point vector of the clustering vector set corresponding to the sub-intention category as a sub-intention representative vector corresponding to the sub-intention category.
In some embodiments, the target representative vector acquisition module is to: calculating the distance between the target statement vector and each sub-intention representative vector; and acquiring the child intention representative vector with the minimum distance from the target statement vector as the target child intention representative vector.
In some embodiments, the target semantic intent related to the semantics of the target question is identified by a trained first intent recognition model, the training module of the trained first intent recognition model comprising: the first intention recognition result obtaining unit is used for carrying out intention recognition on the training problem based on a first intention recognition model to be trained to obtain a first intention recognition result; a second intention recognition result obtaining unit, configured to perform intention recognition on the training question based on a second intention recognition model to obtain a second intention recognition result; and the parameter adjusting unit is used for obtaining a model loss value of the first intention recognition model to be trained based on the difference between the first intention recognition result and the second intention recognition result, adjusting model parameters in the first intention recognition model to be trained according to the model loss value to obtain the trained first intention recognition model, and the model loss value and the difference form a positive correlation relationship.
In some embodiments, the parameter adjustment unit is configured to: obtaining a first loss value according to the difference between the first intention identification result and the second intention identification result; obtaining a second loss value according to the difference between the first intention recognition result and a standard intention recognition result corresponding to the training problem; and obtaining a model loss value of the first intention recognition model to be trained according to the first loss value and the second loss value.
In some embodiments, the reference problem acquisition module is to include: acquiring a named entity set corresponding to the target problem, and acquiring a key condition intention corresponding to the target problem according to the named entity set; and acquiring a question matched with the key condition intention and the target sub-intention as a reference question.
In some embodiments, the apparatus further comprises: the target word acquisition module is used for acquiring words corresponding to the candidate new problems as target words; the first quantity acquisition module is used for acquiring a first quantity of questions including target words in a first question set, wherein the questions in the first question set have matching answers; the first novelty acquisition module is used for determining a first novelty of the candidate new question in the first question set according to the first quantity, and the first quantity and the first novelty have a negative correlation; and the target new problem determining module is used for taking the candidate new problem as the target new problem when the first novelty is larger than the first threshold.
In some embodiments, the apparatus further comprises: the second quantity obtaining module is used for obtaining a second quantity of the questions including the target words in a second question set, wherein the questions in the second question set are questions without matching answers; the second novelty acquisition module is used for determining second novelty of the candidate new question in the second question set according to a second quantity, and the second quantity and the second novelty are in a negative correlation relation; the target new problem determination module is to: when the first novelty is greater than a first threshold and the second novelty is less than a second threshold, the candidate new question is targeted as a new question.
In some embodiments, the first novelty acquisition module is to: calculating the inverse document frequency corresponding to the target words according to the first quantity; and calculating a first novelty of the candidate new question in the first question set according to the inverse document frequency corresponding to the target word.
For the specific limitations of the intelligent question-answering device, reference may be made to the above limitations of the intelligent question-answering method, which are not described herein again. The modules in the intelligent question answering device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In some embodiments, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 12. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing question-answer pairs. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an intelligent question-answering method.
Those skilled in the art will appreciate that the architecture shown in fig. 12 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (15)
1. An intelligent question-answering method, characterized in that the method comprises:
acquiring a target question to be answered;
obtaining a target semantic intention related to the semantics of the target question;
acquiring sub-intention representative vectors matched with target statement vectors corresponding to the target problems as target sub-intention representative vectors, wherein the sub-intention representative vectors are representative vectors corresponding to all sub-intention categories obtained by clustering problem vector sets corresponding to the target semantic intents;
taking a sub-intention corresponding to the target sub-intention representative vector as a target sub-intention, and acquiring a problem matched with the target sub-intention as a reference problem;
and obtaining a target answer of the target question according to the reference answer corresponding to the reference question.
2. The method according to claim 1, wherein the step of clustering the problem vector set corresponding to the target semantic intention to obtain the representative vector corresponding to each sub-intention category comprises:
acquiring a problem set corresponding to the target semantic intention, acquiring statement vectors corresponding to all problems in the problem set, and forming a problem vector set;
clustering the problem vector sets to obtain clustering vector sets corresponding to all sub-intention categories of the target semantic intention;
and acquiring a representative vector of the clustering vector set corresponding to the sub-intention category as a sub-intention representative vector corresponding to the sub-intention category.
3. The method according to claim 2, wherein the obtaining a representative vector of the problem vector set corresponding to the sub-intention category as a sub-intention representative vector corresponding to the sub-intention category comprises:
and acquiring a central point vector of the clustering vector set corresponding to the sub-intention category as a sub-intention representative vector corresponding to the sub-intention category.
4. The method according to claim 1, wherein the obtaining of the sub-intention representative vector matching the target statement vector corresponding to the target question as the target sub-intention representative vector comprises:
calculating the distance between the target statement vector and each sub-intention representative vector;
and acquiring a sub-intention representative vector with the minimum distance from the target statement vector as a target sub-intention representative vector.
5. The method of claim 1, wherein the target semantic intent related to the semantics of the target question is identified by a trained first intent recognition model, the training of the trained first intent recognition model comprising:
performing intention recognition on the training problem based on a first intention recognition model to be trained to obtain a first intention recognition result;
performing intention recognition on the training question based on a second intention recognition model to obtain a second intention recognition result;
obtaining a model loss value of the first intention recognition model to be trained based on the difference between the first intention recognition result and the second intention recognition result, adjusting model parameters in the first intention recognition model to be trained according to the model loss value to obtain the trained first intention recognition model, wherein the model loss value and the difference are in positive correlation.
6. The method of claim 5, wherein the deriving a model loss value for the first intent recognition model to be trained based on the difference between the first intent recognition result and the second intent recognition result comprises:
obtaining a first loss value according to the difference between the first intention identification result and the second intention identification result;
obtaining a second loss value according to the difference between the first intention recognition result and a standard intention recognition result corresponding to the training problem;
and obtaining a model loss value of the first intention recognition model to be trained according to the first loss value and the second loss value.
7. The method of claim 1, wherein the obtaining the question matching the target sub-intention as a reference question comprises:
acquiring a named entity set corresponding to the target problem, and acquiring a key condition intention corresponding to the target problem according to the named entity set;
and acquiring a question matched with the key condition intention and the target sub-intention as a reference question.
8. The method of claim 1, further comprising:
acquiring words corresponding to the candidate new problems as target words;
acquiring a first number of questions including the target words in a first question set, wherein the questions in the first question set have matching answers;
determining a first novelty of the new candidate question in the first set of questions in accordance with the first quantity, the first quantity being inversely related to the first novelty;
when the first novelty is greater than a first threshold, the candidate new question is treated as a target new question.
9. The method of claim 8, further comprising:
acquiring a second number of questions including the target words in a second question set, wherein the questions in the second question set are questions without matching answers;
determining a second novelty of the candidate new question in the second set of questions in accordance with the second quantity, the second quantity having a negative correlation with the second novelty;
the identifying the candidate new question as a target new question when the first novelty is greater than a first threshold comprises:
when the first novelty is greater than a first threshold and the second novelty is less than a second threshold, the candidate new question is treated as a target new question.
10. The method of claim 9, wherein said determining a first novelty of the new candidate question in the first set of questions in accordance with the first quantity comprises:
calculating the inverse document frequency corresponding to the target words according to the first quantity;
and calculating a first novelty of the candidate new question in the first question set according to the inverse document frequency corresponding to the target word.
11. An intelligent question answering device, characterized in that the device comprises:
the target question acquisition module is used for acquiring a target question to be answered;
a target semantic intention acquisition module for acquiring a target semantic intention related to the semantics of the target question;
a target representative vector obtaining module, configured to obtain a sub-intention representative vector matched with a target statement vector corresponding to the target question, as a target sub-intention representative vector, where the sub-intention representative vector is a representative vector corresponding to each sub-intention category obtained by clustering a question vector set corresponding to the target semantic intention;
a reference problem obtaining module, configured to use a sub-intention corresponding to the target sub-intention representative vector as a target sub-intention, and obtain a problem matching with the target sub-intention as a reference problem;
and the target answer obtaining module is used for obtaining a target answer of the target question according to the reference answer corresponding to the reference question.
12. The apparatus according to claim 11, wherein the means for clustering the problem vector set corresponding to the target semantic intent to obtain the representative vector corresponding to each sub-intent category comprises:
a problem vector set acquisition unit, configured to acquire a problem set corresponding to the target semantic intention, acquire statement vectors corresponding to the problems in the problem set, and form a problem vector set;
the clustering unit is used for clustering the problem vector set to obtain a clustering vector set corresponding to each sub-intention category of the target semantic intention;
and the sub-intention representative vector acquisition unit is used for acquiring a representative vector of the clustering vector set corresponding to the sub-intention category as a sub-intention representative vector corresponding to the sub-intention category.
13. The apparatus of claim 11, wherein a target semantic intent related to the semantics of the target question is identified by a trained first intent recognition model, and wherein a training module of the trained first intent recognition model comprises:
the first intention recognition result obtaining unit is used for carrying out intention recognition on the training problem based on a first intention recognition model to be trained to obtain a first intention recognition result;
a second intention recognition result obtaining unit, configured to perform intention recognition on the training question based on a second intention recognition model to obtain a second intention recognition result;
and the parameter adjusting unit is used for obtaining a model loss value of the first intention recognition model to be trained based on the difference between the first intention recognition result and the second intention recognition result, adjusting model parameters in the first intention recognition model to be trained according to the model loss value to obtain the trained first intention recognition model, and the model loss value and the difference are in positive correlation.
14. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor realizes the steps of the method of any one of claims 1 to 10 when executing the computer program.
15. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010545399.7A CN111708873B (en) | 2020-06-15 | 2020-06-15 | Intelligent question-answering method, intelligent question-answering device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010545399.7A CN111708873B (en) | 2020-06-15 | 2020-06-15 | Intelligent question-answering method, intelligent question-answering device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111708873A true CN111708873A (en) | 2020-09-25 |
| CN111708873B CN111708873B (en) | 2023-11-24 |
Family
ID=72540893
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010545399.7A Active CN111708873B (en) | 2020-06-15 | 2020-06-15 | Intelligent question-answering method, intelligent question-answering device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111708873B (en) |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112101044A (en) * | 2020-11-02 | 2020-12-18 | 北京淇瑀信息科技有限公司 | Intention identification method and device and electronic equipment |
| CN112182162A (en) * | 2020-09-30 | 2021-01-05 | 中国人民大学 | Personalized dialogue method and system based on memory neural network |
| CN112417096A (en) * | 2020-11-17 | 2021-02-26 | 平安科技（深圳）有限公司 | Question-answer pair matching method and device, electronic equipment and storage medium |
| CN112417127A (en) * | 2020-12-02 | 2021-02-26 | 网易（杭州）网络有限公司 | Method, device, equipment and medium for training conversation model and generating conversation |
| CN112632248A (en) * | 2020-12-22 | 2021-04-09 | 深圳追一科技有限公司 | Question answering method, device, computer equipment and storage medium |
| CN112667789A (en) * | 2020-12-17 | 2021-04-16 | 中国平安人寿保险股份有限公司 | User intention matching method and device, terminal equipment and storage medium |
| CN112749266A (en) * | 2021-01-19 | 2021-05-04 | 海尔数字科技(青岛)有限公司 | Industrial question and answer method, device, system, equipment and storage medium |
| CN112800190A (en) * | 2020-11-11 | 2021-05-14 | 重庆邮电大学 | Intent recognition and slot value filling joint prediction method based on Bert model |
| CN112989843A (en) * | 2021-03-17 | 2021-06-18 | 中国平安人寿保险股份有限公司 | Intention recognition method and device, computing equipment and storage medium |
| CN113064984A (en) * | 2021-04-25 | 2021-07-02 | 深圳壹账通智能科技有限公司 | Intention recognition method and device, electronic equipment and readable storage medium |
| CN113111160A (en) * | 2021-04-22 | 2021-07-13 | 中国平安人寿保险股份有限公司 | Synonym matching method, device, equipment and storage medium |
| CN113157890A (en) * | 2021-04-25 | 2021-07-23 | 深圳壹账通智能科技有限公司 | Intelligent question and answer method and device, electronic equipment and readable storage medium |
| CN113158680A (en) * | 2021-03-23 | 2021-07-23 | 北京新方通信技术有限公司 | Corpus processing and intention identification method and apparatus |
| CN113220896A (en) * | 2021-04-27 | 2021-08-06 | 北京大数医达科技有限公司 | Multi-source knowledge graph generation method and device and terminal equipment |
| CN113239146A (en) * | 2021-05-12 | 2021-08-10 | 平安科技（深圳）有限公司 | Response analysis method, device, equipment and storage medium |
| CN113377936A (en) * | 2021-05-25 | 2021-09-10 | 杭州搜车数据科技有限公司 | Intelligent question and answer method, device and equipment |
| CN113392197A (en) * | 2021-06-15 | 2021-09-14 | 吉林大学 | Question-answer reasoning method and device, storage medium and electronic equipment |
| CN113392631A (en) * | 2020-12-02 | 2021-09-14 | 腾讯科技（深圳）有限公司 | Corpus expansion method and related device |
| CN113435998A (en) * | 2021-06-23 | 2021-09-24 | 平安科技（深圳）有限公司 | Loan overdue prediction method and device, electronic equipment and storage medium |
| CN113486170A (en) * | 2021-08-02 | 2021-10-08 | 国泰新点软件股份有限公司 | Natural language processing method, device, equipment and medium based on man-machine interaction |
| CN113505207A (en) * | 2021-07-02 | 2021-10-15 | 中科苏州智能计算技术研究院 | Machine reading understanding method and system for financial public opinion research and report |
| CN113568929A (en) * | 2021-07-30 | 2021-10-29 | 咪咕文化科技有限公司 | Data storage method, data query method, data storage device, data query device and electronic equipment |
| CN113868415A (en) * | 2021-09-01 | 2021-12-31 | 阿里巴巴达摩院(杭州)科技有限公司 | Knowledge base generation method and device, storage medium and electronic equipment |
| CN115544236A (en) * | 2022-11-16 | 2022-12-30 | 北京红棉小冰科技有限公司 | Model evaluation method and device and electronic equipment |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170308531A1 (en) * | 2015-01-14 | 2017-10-26 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method, system and storage medium for implementing intelligent question answering |
| KR101851790B1 (en) * | 2017-12-22 | 2018-04-24 | 주식회사 마인드셋 | Question Data Set Extension and Method |
| CN108446286A (en) * | 2017-02-16 | 2018-08-24 | 阿里巴巴集团控股有限公司 | A kind of generation method, device and the server of the answer of natural language question sentence |
| WO2019024704A1 (en) * | 2017-08-03 | 2019-02-07 | 阿里巴巴集团控股有限公司 | Entity annotation method, intention recognition method and corresponding devices, and computer storage medium |
| CN109522393A (en) * | 2018-10-11 | 2019-03-26 | 平安科技（深圳）有限公司 | Intelligent answer method, apparatus, computer equipment and storage medium |
| CN109582933A (en) * | 2018-11-13 | 2019-04-05 | 北京合享智慧科技有限公司 | A kind of method and relevant apparatus of determining text novelty degree |
| CN110232114A (en) * | 2019-05-06 | 2019-09-13 | 平安科技（深圳）有限公司 | Sentence intension recognizing method, device and computer readable storage medium |
| US20190377793A1 (en) * | 2018-06-07 | 2019-12-12 | Alibaba Group Holding Limited | Method and apparatus for establishing a hierarchical intent system |
| CN110909165A (en) * | 2019-11-25 | 2020-03-24 | 杭州网易再顾科技有限公司 | Data processing method, device, medium and electronic equipment |
| US20200134380A1 (en) * | 2018-10-30 | 2020-04-30 | Beijing Horizon Robotics Technology Research And Development Co., Ltd. | Method for Updating Neural Network and Electronic Device |
| WO2020087774A1 (en) * | 2018-10-31 | 2020-05-07 | 平安科技（深圳）有限公司 | Concept-tree-based intention recognition method and apparatus, and computer device |
| CN111177186A (en) * | 2019-12-20 | 2020-05-19 | 北京淇瑀信息科技有限公司 | Question retrieval-based single sentence intention identification method, device and system |
| CN111243576A (en) * | 2020-01-16 | 2020-06-05 | 腾讯科技（深圳）有限公司 | Speech recognition and model training method, device, equipment and storage medium |
-
2020
- 2020-06-15 CN CN202010545399.7A patent/CN111708873B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170308531A1 (en) * | 2015-01-14 | 2017-10-26 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method, system and storage medium for implementing intelligent question answering |
| CN108446286A (en) * | 2017-02-16 | 2018-08-24 | 阿里巴巴集团控股有限公司 | A kind of generation method, device and the server of the answer of natural language question sentence |
| WO2019024704A1 (en) * | 2017-08-03 | 2019-02-07 | 阿里巴巴集团控股有限公司 | Entity annotation method, intention recognition method and corresponding devices, and computer storage medium |
| KR101851790B1 (en) * | 2017-12-22 | 2018-04-24 | 주식회사 마인드셋 | Question Data Set Extension and Method |
| US20190377793A1 (en) * | 2018-06-07 | 2019-12-12 | Alibaba Group Holding Limited | Method and apparatus for establishing a hierarchical intent system |
| CN109522393A (en) * | 2018-10-11 | 2019-03-26 | 平安科技（深圳）有限公司 | Intelligent answer method, apparatus, computer equipment and storage medium |
| US20200134380A1 (en) * | 2018-10-30 | 2020-04-30 | Beijing Horizon Robotics Technology Research And Development Co., Ltd. | Method for Updating Neural Network and Electronic Device |
| WO2020087774A1 (en) * | 2018-10-31 | 2020-05-07 | 平安科技（深圳）有限公司 | Concept-tree-based intention recognition method and apparatus, and computer device |
| CN109582933A (en) * | 2018-11-13 | 2019-04-05 | 北京合享智慧科技有限公司 | A kind of method and relevant apparatus of determining text novelty degree |
| CN110232114A (en) * | 2019-05-06 | 2019-09-13 | 平安科技（深圳）有限公司 | Sentence intension recognizing method, device and computer readable storage medium |
| CN110909165A (en) * | 2019-11-25 | 2020-03-24 | 杭州网易再顾科技有限公司 | Data processing method, device, medium and electronic equipment |
| CN111177186A (en) * | 2019-12-20 | 2020-05-19 | 北京淇瑀信息科技有限公司 | Question retrieval-based single sentence intention identification method, device and system |
| CN111243576A (en) * | 2020-01-16 | 2020-06-05 | 腾讯科技（深圳）有限公司 | Speech recognition and model training method, device, equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| 侯丽仙;李艳玲;林民;李成城;: "融合多约束条件的意图和语义槽填充联合识别", 计算机科学与探索, no. 09 * |
| 陈斯斯;董立平;许丹;郭继军;: "医学文献主题新颖性探测方法对比分析", 中华医学图书情报杂志, no. 02 * |
| 黄辉;刘永坚;解庆;: "基于用户-标签异构网络的社区问答专家发现方法", 计算机工程, no. 02 * |
Cited By (39)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112182162A (en) * | 2020-09-30 | 2021-01-05 | 中国人民大学 | Personalized dialogue method and system based on memory neural network |
| CN112182162B (en) * | 2020-09-30 | 2023-10-31 | 中国人民大学 | Personalized dialogue method and system based on memory neural network |
| CN112101044A (en) * | 2020-11-02 | 2020-12-18 | 北京淇瑀信息科技有限公司 | Intention identification method and device and electronic equipment |
| CN112101044B (en) * | 2020-11-02 | 2021-11-12 | 北京淇瑀信息科技有限公司 | Intention identification method and device and electronic equipment |
| CN112800190A (en) * | 2020-11-11 | 2021-05-14 | 重庆邮电大学 | Intent recognition and slot value filling joint prediction method based on Bert model |
| CN112417096A (en) * | 2020-11-17 | 2021-02-26 | 平安科技（深圳）有限公司 | Question-answer pair matching method and device, electronic equipment and storage medium |
| CN112417096B (en) * | 2020-11-17 | 2024-05-28 | 平安科技（深圳）有限公司 | Question-answer pair matching method, device, electronic equipment and storage medium |
| CN112417127A (en) * | 2020-12-02 | 2021-02-26 | 网易（杭州）网络有限公司 | Method, device, equipment and medium for training conversation model and generating conversation |
| CN113392631A (en) * | 2020-12-02 | 2021-09-14 | 腾讯科技（深圳）有限公司 | Corpus expansion method and related device |
| CN112417127B (en) * | 2020-12-02 | 2023-08-22 | 网易（杭州）网络有限公司 | Dialogue model training and dialogue generation methods, devices, equipment and media |
| CN113392631B (en) * | 2020-12-02 | 2024-04-26 | 腾讯科技（深圳）有限公司 | Corpus expansion method and related device |
| CN112667789A (en) * | 2020-12-17 | 2021-04-16 | 中国平安人寿保险股份有限公司 | User intention matching method and device, terminal equipment and storage medium |
| CN112632248A (en) * | 2020-12-22 | 2021-04-09 | 深圳追一科技有限公司 | Question answering method, device, computer equipment and storage medium |
| CN112749266A (en) * | 2021-01-19 | 2021-05-04 | 海尔数字科技(青岛)有限公司 | Industrial question and answer method, device, system, equipment and storage medium |
| CN112989843A (en) * | 2021-03-17 | 2021-06-18 | 中国平安人寿保险股份有限公司 | Intention recognition method and device, computing equipment and storage medium |
| CN112989843B (en) * | 2021-03-17 | 2023-07-25 | 中国平安人寿保险股份有限公司 | Intention recognition method, device, computing equipment and storage medium |
| CN113158680B (en) * | 2021-03-23 | 2024-05-07 | 北京新方通信技术有限公司 | Corpus processing and intention recognition method and device |
| CN113158680A (en) * | 2021-03-23 | 2021-07-23 | 北京新方通信技术有限公司 | Corpus processing and intention identification method and apparatus |
| CN113111160A (en) * | 2021-04-22 | 2021-07-13 | 中国平安人寿保险股份有限公司 | Synonym matching method, device, equipment and storage medium |
| CN113064984A (en) * | 2021-04-25 | 2021-07-02 | 深圳壹账通智能科技有限公司 | Intention recognition method and device, electronic equipment and readable storage medium |
| CN113157890B (en) * | 2021-04-25 | 2024-06-11 | 深圳壹账通智能科技有限公司 | Intelligent question-answering method and device, electronic equipment and readable storage medium |
| CN113157890A (en) * | 2021-04-25 | 2021-07-23 | 深圳壹账通智能科技有限公司 | Intelligent question and answer method and device, electronic equipment and readable storage medium |
| CN113220896A (en) * | 2021-04-27 | 2021-08-06 | 北京大数医达科技有限公司 | Multi-source knowledge graph generation method and device and terminal equipment |
| CN113220896B (en) * | 2021-04-27 | 2024-03-19 | 北京大数医达科技有限公司 | Multi-source knowledge graph generation method, device and terminal equipment |
| CN113239146B (en) * | 2021-05-12 | 2023-07-28 | 平安科技（深圳）有限公司 | Response analysis method, device, equipment and storage medium |
| CN113239146A (en) * | 2021-05-12 | 2021-08-10 | 平安科技（深圳）有限公司 | Response analysis method, device, equipment and storage medium |
| CN113377936B (en) * | 2021-05-25 | 2022-09-30 | 杭州搜车数据科技有限公司 | Intelligent question and answer method, device and equipment |
| CN113377936A (en) * | 2021-05-25 | 2021-09-10 | 杭州搜车数据科技有限公司 | Intelligent question and answer method, device and equipment |
| CN113392197B (en) * | 2021-06-15 | 2023-08-04 | 吉林大学 | Question-answering reasoning method and device, storage medium and electronic equipment |
| CN113392197A (en) * | 2021-06-15 | 2021-09-14 | 吉林大学 | Question-answer reasoning method and device, storage medium and electronic equipment |
| CN113435998A (en) * | 2021-06-23 | 2021-09-24 | 平安科技（深圳）有限公司 | Loan overdue prediction method and device, electronic equipment and storage medium |
| CN113435998B (en) * | 2021-06-23 | 2023-05-02 | 平安科技（深圳）有限公司 | Loan overdue prediction method and device, electronic equipment and storage medium |
| CN113505207B (en) * | 2021-07-02 | 2024-02-20 | 中科苏州智能计算技术研究院 | Machine reading understanding method and system for financial public opinion research report |
| CN113505207A (en) * | 2021-07-02 | 2021-10-15 | 中科苏州智能计算技术研究院 | Machine reading understanding method and system for financial public opinion research and report |
| CN113568929A (en) * | 2021-07-30 | 2021-10-29 | 咪咕文化科技有限公司 | Data storage method, data query method, data storage device, data query device and electronic equipment |
| CN113486170B (en) * | 2021-08-02 | 2023-12-15 | 国泰新点软件股份有限公司 | Natural language processing method, device, equipment and medium based on man-machine interaction |
| CN113486170A (en) * | 2021-08-02 | 2021-10-08 | 国泰新点软件股份有限公司 | Natural language processing method, device, equipment and medium based on man-machine interaction |
| CN113868415A (en) * | 2021-09-01 | 2021-12-31 | 阿里巴巴达摩院(杭州)科技有限公司 | Knowledge base generation method and device, storage medium and electronic equipment |
| CN115544236A (en) * | 2022-11-16 | 2022-12-30 | 北京红棉小冰科技有限公司 | Model evaluation method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111708873B (en) | 2023-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111708873B (en) | Intelligent question-answering method, intelligent question-answering device, computer equipment and storage medium | |
| US11537820B2 (en) | Method and system for generating and correcting classification models | |
| US9146987B2 (en) | Clustering based question set generation for training and testing of a question and answer system | |
| US9230009B2 (en) | Routing of questions to appropriately trained question and answer system pipelines using clustering | |
| CN109376222B (en) | Question-answer matching degree calculation method, question-answer automatic matching method and device | |
| US10140272B2 (en) | Dynamic context aware abbreviation detection and annotation | |
| CN114565104A (en) | Language model pre-training method, result recommendation method and related device | |
| US9183285B1 (en) | Data clustering system and methods | |
| CN111898374B (en) | Text recognition method, device, storage medium and electronic equipment | |
| WO2023029506A1 (en) | Illness state analysis method and apparatus, electronic device, and storage medium | |
| CN102663129A (en) | Medical field deep question and answer method and medical retrieval system | |
| US20230058194A1 (en) | Text classification method and apparatus, device, and computer-readable storage medium | |
| US20170169355A1 (en) | Ground Truth Improvement Via Machine Learned Similar Passage Detection | |
| CN111444344A (en) | Entity classification method, entity classification device, computer equipment and storage medium | |
| CN108536735B (en) | Multi-mode vocabulary representation method and system based on multi-channel self-encoder | |
| CN116561538A (en) | Question-answer scoring method, question-answer scoring device, electronic equipment and storage medium | |
| CN116992007B (en) | Limiting question-answering system based on question intention understanding | |
| CN112131345B (en) | Text quality recognition method, device, equipment and storage medium | |
| CN113011172A (en) | Text processing method and device, computer equipment and storage medium | |
| CN116541493A (en) | Interactive response method, device, equipment and storage medium based on intention recognition | |
| US20220164546A1 (en) | Machine Learning Systems and Methods for Many-Hop Fact Extraction and Claim Verification | |
| CN113821587B (en) | Text relevance determining method, model training method, device and storage medium | |
| CN114360715A (en) | Constitution identification method and device, electronic equipment and storage medium | |
| CN114416929A (en) | Sample generation method, device, equipment and storage medium of entity recall model | |
| US20170076223A1 (en) | Cognitive System Virtual Corpus Training and Utilization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40028907 Country of ref document: HK |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |