CN110019736B - Question-answer matching method, system, equipment and storage medium based on language model - Google Patents
Question-answer matching method, system, equipment and storage medium based on language model Download PDFInfo
- Publication number
- CN110019736B CN110019736B CN201711482842.5A CN201711482842A CN110019736B CN 110019736 B CN110019736 B CN 110019736B CN 201711482842 A CN201711482842 A CN 201711482842A CN 110019736 B CN110019736 B CN 110019736B
- Authority
- CN
- China
- Prior art keywords
- question
- answer
- language model
- model
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a question-answer matching method, a question-answer matching system, question-answer matching equipment and a storage medium based on a language model, wherein the method comprises the following steps: s1, after receiving the question, obtaining a target question matched with the question from a question database, and then obtaining each answer data corresponding to the target question in an answer database; s2, processing the answer data by adopting a language model to generate corresponding text characteristics and behavior characteristics, wherein the behavior characteristics are used for representing the state and the attribute of the answer data; and S3, calculating the text features and the behavior features by adopting a decision tree model, and predicting the sequencing result of the answer data according to the calculation result. According to the method and the system, the user requirements can be quickly and accurately positioned and answer data can be intelligently sequenced through the language model and the decision tree model, so that the most-thought answers can be screened out for the user, and the user experience is improved.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a question-answer matching method, a question-answer matching system, question-answer matching equipment and a storage medium based on a language model.
Background
In the information age of today, computers have become increasingly popular on a global scale as information transfer tools. With the development of artificial intelligence, the computer can be used for solving the language and valuable sequencing the conversation of the user. In the prior art, a question-answering system generally adopts the following two methods to match a question with an answer:
(1) rule-based matching of questions and answers
The method mainly comprises character string matching search, regular expressions and the like, keywords corresponding to each context are simulated to be matched and associated by compiling a complex regular expression rule template, and if a question does not have a corresponding matching rule, the sequence of the inaudible spectrum is obtained;
(2) corpus system based on search, calculating word vectors
The method forms word vectors after segmenting the words of the sentences, then sorts the answers according to the similarity, and because the algorithm does not really learn and utilize the internal logic relationship of the language, some possible answers copy the questions in the answers, but the similarity calculation based on the similarity can obtain high similarity, and obviously the user experience cannot be ensured.
Disclosure of Invention
The invention provides a question-answer matching method, system, equipment and storage medium based on a language model, aiming at overcoming the defects that in the prior art, a rule-based question-answer method needs a complex regular expression and answers may not be against a spectrum, a corpus system based on retrieval calculates word vector similarity between question-answer sentences, does not utilize the internal logical relationship of languages, and answers may not be against a spectrum.
The invention solves the technical problems through the following technical scheme:
the invention provides a question-answer matching method based on a language model, which is characterized by comprising the following steps:
s1, after receiving the question, obtaining a target question matched with the question from a question database, and then obtaining each answer data corresponding to the target question in an answer database;
and S2, processing the answer data by adopting a language model to generate corresponding text characteristics and behavior characteristics, wherein the behavior characteristics are used for representing the state and the attribute of the answer data, and the behavior characteristics can be at least one of the characteristics of answer state, client type of a user answering the question, whether the question is anonymous answer or not, answer type, time for creating the answer of the question to the present, time for modifying the answer to the present, the number of answered praise, the number of times the user receives the question message, the number of times the user clicks the question message, the number of times the user answers, the number of times the user is praised, the optimal number of times the user answers and the like.
And S3, calculating the text features and the behavior features by adopting a decision tree model, and predicting the sequencing result of the answer data according to the calculation result.
Preferably, the Decision Tree model comprises a GBDT (Gradient Boosting Decision Tree) model.
Preferably, the language model includes an N-Gram model (N-Gram statistical model), a neural network language model and a recurrent neural network, and the step S2 specifically includes:
generating corresponding answer word vectors from the answer data by adopting the N-Gram model;
training the answer word vector by adopting the neural network language model;
and training a result output by the neural network language model by adopting the circular neural network to obtain the text characteristic and the behavior characteristic.
Preferably, the N-Gram model comprises a Skip-Gram model (a language processing model that predicts context given an input term).
Preferably, after step S3, the question-answer matching method further includes: evaluating an AUC (Area under dark, which is an Area under a Roc (receiver operating characteristic) Curve) index of the ranking result and/or evaluating an exposure click rate of the ranking result.
Preferably, after step S3, the question-answer matching method further includes: and pushing a plurality of answer data ranked at the top in the ranking result to the user.
Preferably, before the step of pushing a plurality of answer data ranked at the top in the ranking result to the user, the question-answer matching method further includes: and carrying out ABtest (A/B test) on the sequencing result.
Preferably, after the step of pushing a plurality of answer data ranked in the top in the ranking result to the user, the question-answer matching method further includes:
receiving a user selection instruction;
selecting the best answer from the sorting result according to the selection instruction;
labeling the best answer with a label.
Preferably, step S1 specifically includes:
after receiving the question, processing the question by adopting a language model to generate a question word vector of the question;
calculating the similarity between the question word vector and the word vector corresponding to each question in a question database;
taking the question corresponding to the word vector with the maximum similarity with the question word vector in the question database as a target question;
and returning each answer data corresponding to the target question in the answer database.
The invention also provides a question-answer matching system based on the language model, which is characterized by comprising the following components: the system comprises an acquisition module, a language module and a decision tree module;
the obtaining module is used for obtaining a target question matched with the question from a question database after receiving the question, and is also used for obtaining each answer data corresponding to the target question from an answer database;
the language module is used for processing the answer data by adopting a language model to generate corresponding text characteristics and behavior characteristics, and the behavior characteristics are used for representing the state and the attribute of the answer data;
and the decision tree module is used for calculating the text characteristics and the behavior characteristics by adopting a decision tree model and predicting the sequencing result of the answer data according to the calculation result.
Preferably, the decision tree model comprises a GBDT model.
Preferably, the language model comprises an N-Gram model, a neural network language model and a recurrent neural network;
the N-Gram model is used for generating corresponding answer word vectors from the answer data;
the neural network language model is used for training the answer word vector;
the recurrent neural network is used for training the result output by the neural network language model to obtain the text characteristic and the behavior characteristic.
Preferably, the N-Gram model comprises a Skip-Gram model.
Preferably, the question-answer matching system based on the language model further comprises an evaluation module, and the evaluation module is used for evaluating an AUC indicator of the ranking result and/or evaluating an exposure click rate of the ranking result after the ranking result is predicted by the decision tree model.
Preferably, the question-answer matching system based on the language model further includes a pushing module, and the pushing module is configured to push a plurality of answer data ranked in the top in the ranking result to the user after the ranking result is predicted by the decision tree model.
Preferably, the question-answer matching system based on the language model further comprises an ABtest module, wherein the ABtest module is used for carrying out ABtest test on the sequencing result and then calling the pushing module.
Preferably, the question-answer matching system based on the language model further includes a tag module, and after the push module is called, the tag module is configured to receive a user selection instruction, select an optimal answer from the sorting result according to the selection instruction, and tag the optimal answer.
Preferably, the acquiring module specifically includes: the device comprises a word vector generating unit, a similarity calculating unit and a data returning unit;
the word vector generating unit is used for processing the question questions by adopting a language model after receiving the question questions to generate question word vectors of the question questions;
the similarity calculation unit is used for calculating the similarity between the question word vector and the word vector corresponding to each question in the question database;
the data returning unit is used for taking the question corresponding to the word vector with the maximum similarity with the question word vector in the question database as a target question and returning each answer data corresponding to the target question in the answer database.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, and is characterized in that the processor executes the program to realize the question-answer matching method based on the language model.
The present invention also provides a computer-readable storage medium, on which a computer program is stored, which is characterized in that the program, when executed by a processor, implements the steps of the above-mentioned question-answer matching method based on a language model.
The positive progress effects of the invention are as follows: the invention provides a question-answer matching method, system, equipment and storage medium based on a language model, which can quickly and accurately position user requirements by extracting features through the language model and performing prediction sorting according to the features by adopting a decision tree model, perform intelligent sorting on answer data, screen out the most-thought answers for users, and improve user experience.
Drawings
Fig. 1 is a flowchart of a question-answer matching method based on a language model according to embodiment 1 of the present invention.
Fig. 2 is a flowchart of step S101 of the question-answer matching method based on the language model according to embodiment 1 of the present invention.
Fig. 3 is a flowchart of step S102 of the question-answer matching method based on the language model according to embodiment 1 of the present invention.
Fig. 4 is a flowchart of a language model of the question-answer matching method based on the language model according to embodiment 1 of the present invention.
Fig. 5 is a flowchart of a question-answer matching method based on a language model according to embodiment 2 of the present invention.
Fig. 6 is a flowchart of a question-answer matching method based on a language model according to embodiment 3 of the present invention.
Fig. 7 is a schematic composition diagram of a question-answer matching system based on a language model according to embodiment 4 of the present invention.
Fig. 8 is a schematic composition diagram of a question-answer matching system based on a language model according to embodiment 5 of the present invention.
Fig. 9 is a schematic composition diagram of a question-answer matching system based on a language model according to embodiment 6 of the present invention.
Fig. 10 is a hardware configuration diagram of an electronic device according to embodiment 7 of the present invention.
Detailed Description
The invention is further illustrated by the following examples, which are not intended to limit the scope of the invention.
Example 1
As shown in fig. 1, the question-answer matching method based on the language model according to the present embodiment includes:
step S101, after receiving a question, obtaining a target question matched with the question from a question database, and then obtaining each answer data corresponding to the target question in an answer database.
In view of the development of information technology and artificial intelligence, the user requirements can be met through the database, for example, a user wants to buy a certain commodity on an e-commerce platform and knows some conditions of the commodity, the user can ask a question about the commodity, then the user who bought the commodity once receives the question and can answer the question, finally the answer is returned to the user who asks the question, and the user can make a decision on shopping according to the description of the answerer. As the question data and the answer data are increased, the question data and the answer data can be correspondingly established into a question database and an answer database, so that the question is formed into a question database content list, the answer data is formed into an answer database content list, some field contents related to the commodity are included in the question database content list, such as a user account number, a commodity number, the latest answer time of the question, the number of answers to the question, whether the question answers anonymously or not, and some field contents answered to the commodity are included in the answer database content list, such as a user commodity number, a user answer state, whether the user answers anonymously or not, a user answer type, whether the user answers are set to be optimal, the number of clicks answered by the user, and the like. As the database accumulates, answers associated with the user's question may be returned to the questioner.
In a specific implementation, as shown in fig. 2, step S101 may specifically include:
and S101-1, after receiving the question, processing the question by adopting a language model to generate a question word vector of the question.
Here, an N-Gram model such as a Skip-Gram model may be preferred as a language model to generate word vectors, because the N-Gram model is based on hidden markov's assumption, so that the inherent logical relationship of the language can be maintained, that is, assuming that the probability of the nth word appearing in a piece of text is only related to the preceding limited N-1 words, based on such assumption, a vector representing the content of the question can be generated for the question, so that the text content such as the question and the answer of the answer, etc. provided by the natural language used by the user, can be converted into a calculable word vector.
S101-2, calculating the similarity between the question word vector and the word vector corresponding to each question in a question database;
s101-3, taking a question corresponding to the word vector with the maximum similarity with the question word vector in the question database as a target question;
and S101-4, returning each answer data corresponding to the target question in the answer database.
Step S102, processing the answer data by adopting a language model, and generating corresponding text characteristics and behavior characteristics, wherein the behavior characteristics are used for representing the state and the attribute of the answer data, and the behavior characteristics can comprise at least one of the characteristics of answer state, client type of a user answering the question, whether the question is anonymous answer or not, answer type, time for creating the answer of the question to the present, time for modifying the answer to the present, answer praise number, number of times for receiving the question by the user, number of times for receiving the question message by the user, number of times for clicking the question message by the user, user answer number, number of times for praise of the user, and optimal answer number of times of the user. When a specific application scenario is implemented, the behavior features may be selectively set according to the application scenario, and the weight value of each behavior feature may also be determined according to the importance degree in the actual application.
In specific implementation, in order to obtain the degree of closeness of the association between the answer data and the target question, a language module is required to perform feature extraction on the answer data, such as generating text features, extracting behavior features included in the answer data, and then training the features to obtain features capable of better characterizing the answer data. At this time, as shown in fig. 3, step S102 may specifically include:
step S102-1, generating corresponding answer word vectors from the answer data by adopting the N-Gram model, wherein the N-Gram model can preferably select a Skip-Gram model;
s102-2, training the answer word vector by adopting the neural network language model;
and S102-3, training a result output by the neural network language model by adopting the recurrent neural network to obtain the text characteristic and the behavior characteristic.
Here, the processing flow diagram of the language model is shown in fig. 4, the words at the lowest layer are words obtained by splitting a text, the neural network at the upper layer of the words is word vectors calculated by a co-occurrence matrix, the neural network at the upper layer of the word vectors is nonlinearly transformed by an activation function such as a Sigmoid function (the Sigmoid function is often used as a threshold function of the neural network), and finally, the output values are normalized into probabilities by probability prediction by Softmax (Softmax is a function and is often used as a regression model), so that the model can be optimized by using a gradient descent algorithm like a general neural network, and an optimal parameter solution is obtained by training.
In specific implementation, the main parameters of the language model include:
(1) valid _ size: if the number of the similar meaning words of one word is calculated in the test set, the value is 16;
(2) batch _ size: when the neural network is trained in batch gradient, the number of samples to be trained in each batch is 128;
(3) imbedding _ size: the dimension of the vector in the word vector after the word conversion is 128;
(4) skip _ window: the number of context words around each word is 1;
(5) num _ skips: predicting the number of times of using the context words, and taking the value as 2;
(6) the learning rate of the neural network was set to 0.05.
S103, calculating the text characteristics and the behavior characteristics by adopting a decision tree model, and predicting the sequencing result of the answer data according to the calculation result.
In specific implementation, the GBDT model is preferably selected as the decision tree model, and the main parameters of the model include:
(1) the algorithm sorting logic is Classification;
(2) the learning rate of the optimizer is learngrate 0.05;
(3) the iteration number of the tree is NumIterants which is 50;
(4) the maximum depth of the tree is MaxDepth ═ 6;
(5) the number of bins for the consecutive features is: MaxBins 32.
In this way, multiple trees are constructed through the GBDT model, and answer data is predicted and ranked together.
In the embodiment, the text characteristics and the behavior characteristics are combined through the language model, and then the sequencing result is predicted through the GBDT model, so that the sequencing result is more suitable for the user to ask questions, the user can see the answer which the user wants to see most, and the user experience is improved.
Example 2
As shown in fig. 5, on the basis of embodiment 1, the question-answer matching method based on the language model according to this embodiment further includes:
and step S104, evaluating the AUC index of the sequencing result and/or evaluating the exposure click rate of the sequencing result.
On one hand, the effect of the question-answer matching method based on the language model can be obtained by performing algorithm evaluation on the sequencing result, namely performing AUC index evaluation, so that the question-answer matching method based on the language model can be conveniently subjected to parameter adjustment training. Through evaluation, the AUC can reach 0.836, which indicates that the question-answer matching method based on the language model achieves very good effect.
And on the other hand, the business evaluation can be carried out on the sequencing result, so that the exposure click rate of the user on the sequencing result can be obtained, the sequencing result output by the question-answer matching method based on the language model can be improved, and the user experience is improved.
Example 3
As shown in fig. 6, on the basis of embodiment 1, the question-answer matching method based on the language model according to this embodiment further includes:
step S105, performing an ABtest on the sequencing result;
s106, pushing a plurality of answer data ranked in the top in the ranking result to a user;
s107-1, receiving a user selection instruction;
s107-2, selecting the best answer from the sorting result according to the selection instruction;
and step S107-3, labeling the optimal answer with a label.
By carrying out the ABtest test and collecting the user use feedback, the question-answer matching method based on the language model is convenient to improve, a sequencing result with higher reference value can be provided for the user, the click conversion rate of answer data is improved, and the user use experience is improved.
Example 4
As shown in fig. 7, the question-answer matching system based on a language model according to this embodiment includes an obtaining module 1, a language module 2, and a decision tree module 3, where the obtaining module 1 is configured to obtain, after receiving a question, a target question matched with the question from a question database, and is further configured to obtain each answer data corresponding to the target question in an answer database, the language module 2 is configured to process the answer data by using the language model to generate corresponding text features and behavior features, the behavior features are used to represent states and attributes of the answer data, and the decision tree module 3 is configured to calculate the text features and the behavior features by using the decision tree model, and predict a ranking result of the answer data according to a calculation result.
In specific implementation, the obtaining module 1 includes a word vector generating unit 11, a similarity calculating unit 12, and a data returning unit 13, where the word vector generating unit 11 is configured to process the question questions by using a language model after receiving the question questions, and generate question word vectors of the question questions, the similarity calculating unit 12 is configured to calculate similarities between the question word vectors and word vectors corresponding to each question in a question database, and the data returning unit 13 is configured to use a question corresponding to a word vector in the question database with the highest similarity to the question word vectors as a target question, and also configured to return each answer data corresponding to the target question in an answer database.
In particular, the decision tree model in the decision tree module 3 includes a GBDT model.
In specific implementation, the language model in the language module 2 includes an N-Gram model, a neural network language model and a recurrent neural network, wherein the N-Gram model is used for generating the answer data into corresponding answer word vectors, and a Skip-Gram model can be preferably selected; the neural network language model is used for training the answer word vector; the recurrent neural network is used for training the result output by the neural network language model to obtain the text characteristic and the behavior characteristic.
Example 5
As shown in fig. 8, on the basis of embodiment 4, the question-answer matching system based on the language model according to this embodiment further includes an evaluation module 4, where the evaluation module 4 is configured to, after the decision tree model predicts the ranking result, evaluate an AUC indicator of the ranking result and/or evaluate an exposure click rate of the ranking result. On one hand, the effect of the question-answer matching method based on the language model can be obtained by performing algorithm evaluation on the sequencing result, namely performing AUC index evaluation, so that the question-answer matching method based on the language model can be conveniently subjected to parameter adjustment training. And on the other hand, the business evaluation can be carried out on the sequencing result, so that the exposure click rate of the user on the sequencing result can be obtained, the sequencing result output by the question-answer matching method based on the language model can be improved, and the user experience is improved.
Example 6
As shown in fig. 9, on the basis of embodiment 4, the question-answer matching system based on the language model according to this embodiment further includes an ABTest module 5, a pushing module 6, and a labeling module 7, where the ABTest module 5 is configured to perform an ABTest on the ranking results, the pushing module 6 is configured to push a plurality of answer data ranked in the ranking results in the top to a user, and the labeling module 7 is configured to receive a user selection instruction, select a best answer from the ranking results according to the selection instruction, and label the best answer. By carrying out the ABTest test, recommending answer data to the user and collecting the mark feedback condition of the user to the recommended answer data, the system can utilize the marks as the important behavior characteristics of the answer data, so that the answer data has more reference value, the question-answer matching system based on the language model is improved, the answer data with more reference value is provided for the user, the click conversion rate of the answer data is improved, and the use experience of the user is improved.
Example 7
The electronic device according to this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the question-answer matching method based on the language model described in embodiment 1, embodiment 2, or embodiment 3 when executing the computer program.
Fig. 10 is a schematic structural diagram of the electronic device according to the present embodiment. FIG. 10 illustrates a block diagram of an exemplary electronic device 50 suitable for use in implementing embodiments of the present invention. The electronic device 50 shown in fig. 10 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 10, the electronic device 50 may be embodied in the form of a general purpose computing device, which may be, for example, a server device. The components of the electronic device 50 may include, but are not limited to: the at least one processor 51, the at least one memory 52, and a bus 53 connecting the various system components (including the memory 52 and the processor 51).
The bus 53 includes a data bus, an address bus, and a control bus.
The memory 52 may include volatile memory, such as Random Access Memory (RAM)521 and/or cache memory 522, and may further include Read Only Memory (ROM) 523.
The processor 51 executes various functional applications and data processing, such as the question-answer matching method based on the language model provided in embodiment 1 of the present invention, by running the computer program stored in the memory 52.
The electronic device 50 may also communicate with one or more external devices 54 (e.g., a keyboard, a pointing device, etc.). Such communication may be through an input/output (I/O) interface 55. Further, the electronic device 50 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network such as the Internet) through a network adapter 56, the network adapter 56 communicating with other modules of the electronic device 50 through the bus 53. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 50, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.
It should be noted that although in the above detailed description several units/modules or sub-units/modules of the electronic device are mentioned, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module, according to embodiments of the application. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.
Example 8
The present embodiment relates to a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the steps of the language model-based question-answer matching method described in embodiment 1, embodiment 2, or embodiment 3.
More specific examples, among others, that the readable storage medium may employ may include, but are not limited to: a portable disk, a hard disk, random access memory, read only memory, erasable programmable read only memory, optical storage device, magnetic storage device, or any suitable combination of the foregoing.
In a possible implementation manner, the present invention can also be implemented in the form of a program product, which includes program code for causing a terminal device to execute steps of implementing the question-answer matching method based on a language model described in embodiment 1, embodiment 2 or embodiment 3 when the program product runs on the terminal device.
Where program code for carrying out the invention is written in any combination of one or more programming languages, the program code may be executed entirely on the user device, partly on the user device, as a stand-alone software package, partly on the user device and partly on a remote device or entirely on the remote device.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.
Claims (20)
1. A question-answer matching method based on a language model is characterized by comprising the following steps:
s1, after receiving the question, obtaining a target question matched with the question from a question database, and then obtaining each answer data corresponding to the target question in an answer database;
s2, processing the answer data by adopting a language model to generate corresponding text characteristics and behavior characteristics, wherein the behavior characteristics are used for representing the state and the attribute of the answer data;
and S3, calculating the text features and the behavior features by adopting a decision tree model, and predicting the sequencing result of the answer data according to the calculation result.
2. The language model-based question-answer matching method of claim 1, wherein the decision tree model comprises a GBDT model.
3. The method according to claim 1, wherein the language model includes an N-Gram model, a neural network language model and a recurrent neural network, and the step S2 specifically includes:
generating corresponding answer word vectors from the answer data by adopting the N-Gram model;
training the answer word vector by adopting the neural network language model;
and training a result output by the neural network language model by adopting the circular neural network to obtain the text characteristic and the behavior characteristic.
4. The language model-based question-answer matching method of claim 3, wherein the N-Gram model comprises a Skip-Gram model.
5. The language model-based question-answer matching method according to claim 1, wherein after step S3, said question-answer matching method further comprises: evaluating an AUC indicator of the ranking result and/or evaluating an exposed click rate of the ranking result.
6. The language model-based question-answer matching method according to claim 1, wherein after step S3, said question-answer matching method further comprises: and pushing a plurality of answer data ranked at the top in the ranking result to the user.
7. The language model-based question-answer matching method according to claim 6, wherein before the step of pushing a plurality of answer data ranked in the top in the ranking result to the user, the question-answer matching method further comprises: and carrying out ABtest on the sequencing result.
8. The language model-based question-answer matching method according to claim 6, wherein after the step of pushing a plurality of answer data ranked in the top in the ranking result to the user, the question-answer matching method further comprises:
receiving a user selection instruction;
selecting the best answer from the sorting result according to the selection instruction;
labeling the best answer with a label.
9. The language model-based question-answer matching method according to claim 1, wherein the step S1 specifically includes:
after receiving the question, processing the question by adopting a language model to generate a question word vector of the question;
calculating the similarity between the question word vector and the word vector corresponding to each question in a question database;
taking the question corresponding to the word vector with the maximum similarity with the question word vector in the question database as a target question;
and returning each answer data corresponding to the target question in the answer database.
10. A question-answer matching system based on a language model, comprising: the system comprises an acquisition module, a language module and a decision tree module;
the obtaining module is used for obtaining a target question matched with the question from a question database after receiving the question, and is also used for obtaining each answer data corresponding to the target question from an answer database;
the language module is used for processing the answer data by adopting a language model to generate corresponding text characteristics and behavior characteristics, and the behavior characteristics are used for representing the state and the attribute of the answer data;
and the decision tree module is used for calculating the text characteristics and the behavior characteristics by adopting a decision tree model and predicting the sequencing result of the answer data according to the calculation result.
11. The language model-based question-answer matching system of claim 10 wherein the decision tree model comprises a GBDT model.
12. The language model-based question-answer matching system of claim 10 wherein the language models include an N-Gram model, a neural network language model and a recurrent neural network;
the N-Gram model is used for generating corresponding answer word vectors from the answer data;
the neural network language model is used for training the answer word vector;
the recurrent neural network is used for training the result output by the neural network language model to obtain the text characteristic and the behavior characteristic.
13. The language-model-based question-answer matching system of claim 12, wherein the N-Gram model comprises a Skip-Gram model.
14. The language model-based question-answer matching system of claim 10, wherein the language model-based question-answer matching system further comprises an evaluation module for evaluating AUC measures of the ranked results and/or evaluating exposed click-through rates of the ranked results after the ranked results are predicted by the decision tree model.
15. The language model-based question-answer matching system of claim 10, wherein the language model-based question-answer matching system further comprises a pushing module, and the pushing module is configured to push a number of answer data ranked in the top in the ranking result to a user after the decision tree model predicts the ranking result.
16. The language model-based question-answer matching system of claim 15, wherein the language model-based question-answer matching system further comprises an ABtest module, the ABtest module is configured to perform an ABtest on the sorting results and then call the pushing module.
17. The language model-based question-answer matching system of claim 15, wherein the language model-based question-answer matching system further comprises a tagging module, and after the pushing module is invoked, the tagging module is configured to receive a user selection instruction, select a best answer from the sorting results according to the selection instruction, and tag the best answer.
18. The language model-based question-answer matching system of claim 10, wherein the obtaining module specifically comprises: the device comprises a word vector generating unit, a similarity calculating unit and a data returning unit;
the word vector generating unit is used for processing the question questions by adopting a language model after receiving the question questions to generate question word vectors of the question questions;
the similarity calculation unit is used for calculating the similarity between the question word vector and the word vector corresponding to each question in the question database;
the data returning unit is used for taking the question corresponding to the word vector with the maximum similarity with the question word vector in the question database as a target question and returning each answer data corresponding to the target question in the answer database.
19. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the language model-based question-answer matching method according to any one of claims 1 to 9 when executing the computer program.
20. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the language model-based question-answer matching method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711482842.5A CN110019736B (en) | 2017-12-29 | 2017-12-29 | Question-answer matching method, system, equipment and storage medium based on language model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711482842.5A CN110019736B (en) | 2017-12-29 | 2017-12-29 | Question-answer matching method, system, equipment and storage medium based on language model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110019736A CN110019736A (en) | 2019-07-16 |
| CN110019736B true CN110019736B (en) | 2021-10-01 |
Family
ID=67187166
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711482842.5A Active CN110019736B (en) | 2017-12-29 | 2017-12-29 | Question-answer matching method, system, equipment and storage medium based on language model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110019736B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110570838B (en) * | 2019-08-02 | 2022-06-07 | 北京葡萄智学科技有限公司 | Voice stream processing method and device |
| CN110569350B (en) * | 2019-08-08 | 2022-08-09 | 河北省讯飞人工智能研究院 | Legal recommendation method, equipment and storage medium |
| WO2021051404A1 (en) * | 2019-09-20 | 2021-03-25 | Beijing Didi Infinity Technology And Development Co., Ltd. | Systems and methods for auxiliary reply |
| CN111078972B (en) * | 2019-11-29 | 2023-06-16 | 支付宝(杭州)信息技术有限公司 | Questioning behavior data acquisition method, questioning behavior data acquisition device and server |
| CN111026854A (en) * | 2019-12-05 | 2020-04-17 | 电子科技大学广东电子信息工程研究院 | Answer quality assessment method |
| CN111368064B (en) * | 2020-03-26 | 2023-04-07 | 深圳平安医疗健康科技服务有限公司 | Survey information processing method, device, equipment and storage medium |
| CN113204623A (en) * | 2021-05-28 | 2021-08-03 | 中国工商银行股份有限公司 | Question answering method and device |
| CN117290694B (en) * | 2023-11-24 | 2024-03-15 | 北京并行科技股份有限公司 | Question-answering system evaluation method, device, computing equipment and storage medium |
| CN117609470B (en) * | 2023-12-08 | 2024-08-09 | 中科南京信息高铁研究院 | Question-answering system based on large language model and knowledge graph, construction method thereof and intelligent data management platform |
| CN117473071B (en) * | 2023-12-27 | 2024-04-05 | 珠海格力电器股份有限公司 | Data retrieval method, device, equipment and computer readable medium |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101118554A (en) * | 2007-09-14 | 2008-02-06 | 中兴通讯股份有限公司 | Intelligent interactive request-answering system and processing method thereof |
| CN101257512A (en) * | 2008-02-02 | 2008-09-03 | 黄伟才 | Inquiry answer matching method used for inquiry answer system as well as inquiry answer method and system |
| CN101377777A (en) * | 2007-09-03 | 2009-03-04 | 北京百问百答网络技术有限公司 | Automatic inquiring and answering method and system |
| CN101566998A (en) * | 2009-05-26 | 2009-10-28 | 华中师范大学 | Chinese question-answering system based on neural network |
| CN101656799A (en) * | 2008-08-20 | 2010-02-24 | 阿鲁策株式会社 | Automatic conversation system and conversation scenario editing device |
| CN101740024A (en) * | 2008-11-19 | 2010-06-16 | 中国科学院自动化研究所 | Method for automatic evaluation based on generalized fluent spoken language fluency |
| CN102034475A (en) * | 2010-12-08 | 2011-04-27 | 中国科学院自动化研究所 | Method for interactively scoring open short conversation by using computer |
| CN102255788A (en) * | 2010-05-19 | 2011-11-23 | 北京启明星辰信息技术股份有限公司 | Message classification decision establishing system and method and message classification system and method |
| CN102456073A (en) * | 2011-11-03 | 2012-05-16 | 中国人民解放军国防科学技术大学 | Partial extremum inquiry method |
| CN102760128A (en) * | 2011-04-26 | 2012-10-31 | 华东师范大学 | Telecommunication field package recommending method based on intelligent customer service robot interaction |
| CN104572868A (en) * | 2014-12-18 | 2015-04-29 | 清华大学 | Method and device for information matching based on questioning and answering system |
| CN104636456A (en) * | 2015-02-03 | 2015-05-20 | 大连理工大学 | Question routing method based on word vectors |
| CN107463699A (en) * | 2017-08-15 | 2017-12-12 | 济南浪潮高新科技投资发展有限公司 | A kind of method for realizing question and answer robot based on seq2seq models |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050060662A1 (en) * | 2003-08-22 | 2005-03-17 | Thomas Soares | Process for creating service action data structures |
| US9703536B2 (en) * | 2014-08-11 | 2017-07-11 | International Business Machines Corporation | Debugging code using a question and answer system based on documentation and code change records |
-
2017
- 2017-12-29 CN CN201711482842.5A patent/CN110019736B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101377777A (en) * | 2007-09-03 | 2009-03-04 | 北京百问百答网络技术有限公司 | Automatic inquiring and answering method and system |
| CN101118554A (en) * | 2007-09-14 | 2008-02-06 | 中兴通讯股份有限公司 | Intelligent interactive request-answering system and processing method thereof |
| CN101257512A (en) * | 2008-02-02 | 2008-09-03 | 黄伟才 | Inquiry answer matching method used for inquiry answer system as well as inquiry answer method and system |
| CN101656799A (en) * | 2008-08-20 | 2010-02-24 | 阿鲁策株式会社 | Automatic conversation system and conversation scenario editing device |
| CN101740024A (en) * | 2008-11-19 | 2010-06-16 | 中国科学院自动化研究所 | Method for automatic evaluation based on generalized fluent spoken language fluency |
| CN101566998A (en) * | 2009-05-26 | 2009-10-28 | 华中师范大学 | Chinese question-answering system based on neural network |
| CN102255788A (en) * | 2010-05-19 | 2011-11-23 | 北京启明星辰信息技术股份有限公司 | Message classification decision establishing system and method and message classification system and method |
| CN102034475A (en) * | 2010-12-08 | 2011-04-27 | 中国科学院自动化研究所 | Method for interactively scoring open short conversation by using computer |
| CN102760128A (en) * | 2011-04-26 | 2012-10-31 | 华东师范大学 | Telecommunication field package recommending method based on intelligent customer service robot interaction |
| CN102456073A (en) * | 2011-11-03 | 2012-05-16 | 中国人民解放军国防科学技术大学 | Partial extremum inquiry method |
| CN104572868A (en) * | 2014-12-18 | 2015-04-29 | 清华大学 | Method and device for information matching based on questioning and answering system |
| CN104636456A (en) * | 2015-02-03 | 2015-05-20 | 大连理工大学 | Question routing method based on word vectors |
| CN107463699A (en) * | 2017-08-15 | 2017-12-12 | 济南浪潮高新科技投资发展有限公司 | A kind of method for realizing question and answer robot based on seq2seq models |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110019736A (en) | 2019-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110019736B (en) | Question-answer matching method, system, equipment and storage medium based on language model | |
| US11120801B2 (en) | Generating dialogue responses utilizing an independent context-dependent additive recurrent neural network | |
| US11113479B2 (en) | Utilizing a gated self-attention memory network model for predicting a candidate answer match to a query | |
| CN110147551B (en) | Multi-category entity recognition model training, entity recognition method, server and terminal | |
| US20200050942A1 (en) | Deep learning model for cloud based technical support automation | |
| CN111444320B (en) | Text retrieval method and device, computer equipment and storage medium | |
| US20190164084A1 (en) | Method of and system for generating prediction quality parameter for a prediction model executed in a machine learning algorithm | |
| CN109376222B (en) | Question-answer matching degree calculation method, question-answer automatic matching method and device | |
| US20170351962A1 (en) | Predicting user question in question and answer system | |
| CN109086265B (en) | Semantic training method and multi-semantic word disambiguation method in short text | |
| CN111783993A (en) | Intelligent labeling method and device, intelligent platform and storage medium | |
| US11874798B2 (en) | Smart dataset collection system | |
| CN111339292A (en) | Training method, system, equipment and storage medium of text classification network | |
| CN111078881B (en) | Fine-grained sentiment analysis method and system, electronic equipment and storage medium | |
| US11797281B2 (en) | Multi-language source code search engine | |
| CN111563158A (en) | Text sorting method, sorting device, server and computer-readable storage medium | |
| US20190251473A1 (en) | Cognitive System Virtual Corpus Training and Utilization | |
| US20240160642A1 (en) | Systems and methods for categorization of ingested database entries to determine topic frequency | |
| CN109299459B (en) | Word vector training method and device for single semantic supervision | |
| CN109271633B (en) | Word vector training method and device for single semantic supervision | |
| CN112330387B (en) | Virtual broker applied to house watching software | |
| CN118245745A (en) | Task processing, object recommendation, model training method and computing device | |
| CN111445271A (en) | Model generation method, and prediction method, system, device and medium for cheating hotel | |
| CN116910357A (en) | Data processing method and related device | |
| CN110851600A (en) | Text data processing method and device based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |