CN111666393A - Verification method and device of intelligent question-answering system, computer equipment and storage medium - Google Patents
Verification method and device of intelligent question-answering system, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111666393A CN111666393A CN202010356459.0A CN202010356459A CN111666393A CN 111666393 A CN111666393 A CN 111666393A CN 202010356459 A CN202010356459 A CN 202010356459A CN 111666393 A CN111666393 A CN 111666393A
- Authority
- CN
- China
- Prior art keywords
- question
- entity
- replied
- preset
- reply information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000012795 verification Methods 0.000 title claims abstract description 28
- 238000013135 deep learning Methods 0.000 claims abstract description 22
- 230000006870 function Effects 0.000 claims abstract description 17
- 239000013598 vector Substances 0.000 claims description 62
- 238000005070 sampling Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 13
- 238000005295 random walk Methods 0.000 claims description 9
- 238000004088 simulation Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 7
- 238000012217 deletion Methods 0.000 description 4
- 230000037430 deletion Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000002787 reinforcement Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 235000013305 food Nutrition 0.000 description 1
- 235000015220 hamburgers Nutrition 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/355—Class or cluster creation or modification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Animal Behavior & Ethology (AREA)
- Evolutionary Biology (AREA)
- Human Computer Interaction (AREA)
- Probability & Statistics with Applications (AREA)
- Medical Informatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computing Systems (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
The invention discloses a verification method, a verification device, computer equipment and a storage medium of an intelligent question-answering system.A triple is generated according to a preset intention set and a knowledge graph, and comprises an initial entity and an entity relation; generating a question to be replied through a preset question template according to the entity relationship and the initial entity; sending the question to be replied to a preset question-answer simulator, and receiving reply information of the question-answer simulator to the question to be replied; and verifying the triples, the questions to be replied and the reply information by adopting a deep learning method to obtain a classification result. The problem to be replied is generated based on knowledge graph simulation, diversity and comprehensiveness of the problem to be replied are improved, the knowledge graph is combined with a deep learning method, reply of the preset question-answer simulator is verified, the problem that a return function cannot be effectively measured and quantified in deep learning is solved, and reply accuracy of the preset question-answer simulator is improved.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a verification method and device of an intelligent question-answering system, computer equipment and a storage medium
Background
With the development of computer technology, the man-machine intelligent question-answering system has attracted wide attention in recent years, and is applied to intelligent customer service, chat robots, intelligent sound boxes and the like.
At present, most task type question-answering systems realize functions by using a reinforcement learning mode, but the question-answering systems in the reinforcement learning mode need to interact with a large amount of information with users, and cannot directly use the existing linguistic data in training, so that the time cost is greatly increased. There are also ways to implement a question-answering system using a user simulator, but there is a lack of a suitable reward function in such a question-answering system, and the reward function is a direct factor affecting the performance of the question-answering system.
Disclosure of Invention
The embodiment of the invention provides a verification method and device of an intelligent question-answering system, computer equipment and a storage medium, and aims to solve the problem that the question-answering system is lack of a proper return function.
A verification method of an intelligent question-answering system comprises the following steps:
generating a triple according to a preset intention set and a knowledge graph, wherein the triple comprises a starting entity, an entity relation and an end entity;
generating a question to be replied through a preset question template according to the entity relationship and the initial entity;
sending the question to be replied to a preset question-answer simulator, and receiving reply information of the preset question-answer simulator to the question to be replied;
and verifying the triples, the questions to be replied and the reply information by adopting a deep learning method to obtain a classification result.
An authentication device of an intelligent question-answering system, comprising:
the system comprises a triple generating module, a target generating module and a target processing module, wherein the triple generating module is used for generating a triple according to a preset intention set and a knowledge graph, and the triple comprises a starting entity, an entity relation and an end entity;
the problem generation module is used for generating a problem to be replied through a preset problem template according to the entity relationship and the initial entity;
the question sending module is used for sending the question to be replied to a preset question-answer simulator and receiving reply information of the preset question-answer simulator to the question to be replied;
and the classification result generation module is used for verifying the triples, the questions to be replied and the reply information by adopting a deep learning method to obtain a classification result.
A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the authentication method of the intelligent question-answering system when executing the computer program.
A computer-readable storage medium, which stores a computer program that, when executed by a processor, implements the steps of the authentication method of the above-described intelligent question-and-answer system.
According to the verification method, the verification device, the computer equipment and the storage medium of the intelligent question-answering system, the triples are generated according to the preset intention set and the knowledge graph, and the triples comprise the initial entities and the entity relations; generating a question to be replied through a preset question template according to the entity relationship and the initial entity; sending the question to be replied to a preset question-answer simulator, and receiving reply information of the question-answer simulator to the question to be replied; and verifying the triples, the questions to be replied and the reply information by adopting a deep learning method to obtain a classification result. The problem to be replied is generated based on knowledge graph simulation, diversity and comprehensiveness of the problem to be replied are improved, the knowledge graph is combined with a deep learning method, reply of a preset question-answer simulator is verified, the problem that a return function cannot be effectively measured and quantified in deep learning is solved, and the accuracy of reply of the preset question-answer simulator is improved while the preset question-answer simulator is trained.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
FIG. 1 is a schematic diagram of an application environment of a verification method of an intelligent question answering system according to an embodiment of the present invention;
FIG. 2 is a flow chart of a verification method of the intelligent question answering system in one embodiment of the present invention;
FIG. 3 is another flow chart of a method for validating an intelligent question answering system in accordance with an embodiment of the present invention;
FIG. 4 is another flow chart of a method for validating an intelligent question answering system in accordance with an embodiment of the present invention;
FIG. 5 is another flow chart of a method for validating an intelligent question answering system in accordance with an embodiment of the present invention;
FIG. 6 is a schematic block diagram of an authentication device of the intelligent question answering system in an embodiment of the present invention;
FIG. 7 is another schematic block diagram of an authentication device of the intelligent question answering system in an embodiment of the present invention;
FIG. 8 is another functional block diagram of an authentication device of the intelligent question answering system in an embodiment of the present invention;
FIG. 9 is another functional block diagram of an authentication device of the intelligent question answering system in an embodiment of the present invention;
FIG. 10 is a schematic diagram of a computer device according to an embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The verification method of the intelligent question-answering system provided by the embodiment of the invention can be applied to the application environment shown in figure 1. Specifically, the verification method of the intelligent question-answering system is applied to a verification system of the intelligent question-answering system, the verification system of the intelligent question-answering system comprises a client and a server shown in fig. 1, and the client and the server are communicated through a network and used for solving the problem that the question-answering system lacks a proper return function. The client is also called a user side, and refers to a program corresponding to the server and providing local services for the client. The client may be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server may be implemented as a stand-alone server or as a server cluster consisting of a plurality of servers. The preset question-answer simulator can be installed in the client, and the preset question-answer simulator can be an intelligent customer service simulator, a chat robot simulator, an intelligent assistant simulator or an intelligent sound box simulator and the like.
In an embodiment, as shown in fig. 2, a verification method of an intelligent question answering system is provided, which is described by taking the application of the method to the server side in fig. 1 as an example, and includes the following steps:
s11: and generating a triple according to a preset intention set and a knowledge graph, wherein the triple comprises a starting entity, an entity relation and an end entity.
The preset intention set refers to potential intention summary in the problems proposed by the user, and different intention sets can be established according to different application scenes. A knowledge-graph is essentially a database that includes triples of entities and relationships between the entities. The starting entity can be essentially considered as an element, the starting entity being the entity in the triplet that is the starting point. The entity relationship is a relationship between a starting entity and an end entity corresponding to the starting entity. The essence of the end entity is a conclusion entity to the starting entity and entity relationships. The triple is an organization mode of entity data, the essence of the triple is a set, and the triple comprises a starting entity, an entity relation and an end entity.
Specifically, since the preset intention set contains potential intentions in the questions posed by the user, determining an intention according to the preset intention set can replace a mode of analyzing the intention by acquiring the questions posed by the user. Determining an intention in a preset intention set, taking the intention as a starting entity, inputting the intention into a knowledge graph, and determining an entity relation and an end entity corresponding to the intention in the knowledge graph.
S12: and generating the question to be replied through a preset question template according to the entity relationship and the initial entity.
The preset question template is used for generating a question to be replied, and can be set according to question sentences of users in different scenes or can be set according to question sentences of the intelligent equipment. The question to be replied is an question sentence set for the initial entity and the entity relation of the triple, and the question to be replied is a question for waiting for other intelligent equipment to reply.
Specifically, after a triple is generated according to a preset intention set and a knowledge graph, a problem to be replied is generated through a preset problem template according to an entity relation and an initial entity in the triple.
The triple comprises a starting entity, an entity relation and an end entity, and the end entity is a conclusion of the starting entity and the entity relation, so that when the problem to be replied is generated, the problem to be replied is generated through a preset problem template only according to the starting entity and the entity relation.
S13: sending the questions to be replied to a preset question-answer simulator, and receiving reply information of the preset question-answer simulator on the questions to be replied.
The preset question-answer simulator can receive and reply to the question to be replied, and can be an intelligent customer service system, a chat robot, a personal intelligent assistant or an intelligent sound box and the like. The reply information is generated by a preset question-answering simulator for the question to be replied.
Specifically, after a question to be replied is generated through a preset question template according to an entity relationship and an initial entity, the question to be replied is sent to a preset question-answer simulator, after the preset question-answer simulator receives the question to be replied, the question to be replied is replied to generate reply information, the reply information is sent to a server, and the server receives the reply information of the question to be replied by the question-answer simulator.
S14: and verifying the triples, the questions to be replied and the reply information by adopting a deep learning method to obtain a classification result.
Wherein the classification result indicates whether the reply message is a result related to and correct for the question to be replied. Optionally, the classification result includes: the first type of result is that the reply message is not related to the question to be replied, the second type of result is that the reply message is a correct reply message to the question to be replied, and the third type of result is that the reply message is not a correct reply message to the question to be replied.
Specifically, after receiving the reply information of the question to be replied by the preset question-answering simulator, the reply information is judged. Further, the step of judging the reply message comprises two steps: the first step is to judge the relevance of the reply message, namely to judge whether the reply message is relevant to the question to be replied; the second step is to judge the accuracy of the reply message, i.e. to judge whether the reply message is the correct reply to the question to be replied. Because the triple includes the starting entity, the entity relationship and the end point entity, the relevance of the reply information can be judged through the starting entity and the entity relationship, and if the reply information does not include the starting entity and the entity relationship, the reply information is judged to be irrelevant to the problem to be replied, namely, a first-class result. The end-point entity can be used to determine whether the reply message is a correct reply to the question to be replied, and if the reply message includes the relationship between the start entity and the entity but does not include the end-point entity, the reply message is not a correct reply to the question to be replied, i.e. the second type of result. If the reply message contains the relationship between the starting entity and the entity and includes the destination entity, the reply message is the correct reply message of the question to be replied, i.e. the third type result.
Further, the judgment of the reply information is completed through a deep learning method. After receiving the reply information of the question to be replied by the preset question-answering simulator, verifying the triples, the question to be replied and the reply information by adopting a deep learning method to obtain a classification result.
In this embodiment, a triplet is generated by a preset intent set and a knowledge graph, the triplet including a starting entity and an entity relationship; generating a question to be replied through a preset question template according to the entity relationship and the initial entity; sending the question to be replied to a preset question-answer simulator, and receiving reply information of the question-answer simulator to the question to be replied; and verifying the triples, the questions to be replied and the reply information by adopting a deep learning method to obtain a classification result. The problem to be replied is generated based on knowledge graph simulation, diversity and comprehensiveness of the problem to be replied are improved, the knowledge graph is combined with a deep learning method, reply of a preset question-answer simulator is verified, the problem that a return function cannot be effectively measured and quantified in deep learning is solved, and the accuracy of reply of the preset question-answer simulator is improved while the preset question-answer simulator is trained.
In an embodiment, as shown in fig. 3, the step S11 of generating the triple according to the preset intention set and the knowledge graph specifically includes the following steps:
s111: and acquiring a starting entity from a preset intention set in a preset sampling mode.
The preset sampling mode is a mode of collecting the initial entity from the intention set, and the preset sampling mode may be a random sampling mode, a sequential sampling mode, or an array sampling mode.
In a specific embodiment, a random sampling mode is adopted as the preset sampling mode. Specifically, in a preset intention set, randomly sampling an entity set, collecting an entity from the entity set in a random sampling mode after the entity set is collected, and taking the entity as a starting entity.
The entity set is used for storing a set of entities of different categories, for example, the entity set may include entity categories such as an electronic product entity category, a furniture entity category, a food entity category, or a professional entity category, and the corresponding starting entity may be an entity such as an iPhone mobile phone, a refrigerator, a hamburger, or a truck driver.
S112: and taking the initial entity as a starting point, and obtaining an entity relation and an end point entity corresponding to the initial entity from the knowledge graph by adopting a random walk method.
S113: a triplet is generated based on the starting entity, the ending entity, and the entity relationship.
The format of the triplet may be (starting entity, entity relationship, end entity).
Specifically, after a starting entity is collected from an intention set in a preset sampling mode, the starting entity is used as a starting point, a random walk method is adopted, an entity relation and an end point entity corresponding to the starting entity are determined from a knowledge graph, and a triplet is generated according to a triplet format based on the starting entity, the entity relation and the end point entity, wherein the triplet format is (starting entity, entity relation and end point entity).
Exemplarily, assuming that an initial entity collected from the intention set by a preset sampling mode is "iPhone 11", taking "iPhone 11" as a starting point, determining an entity relationship corresponding to "iPhone 11" as an official website price and an end entity as "5499" from the knowledge graph by adopting a random walk method, and generating a triplet (iPhone11, official website price, 5499) according to a triplet format based on "iPhone 11", "official website price" and "5499").
The knowledge graph stores a plurality of entities and the relationship between each entity, so that a plurality of entity relationships and end point entities corresponding to a starting entity are provided, and one of the entity relationships and the end point entities related to the starting entity is selected to generate the triples by a random walk method.
In this embodiment, a starting entity is collected from an intention set by a preset sampling manner; taking a starting entity as a starting point, and obtaining an entity relation and an end entity corresponding to the starting entity from a knowledge graph by adopting a random walk method; a triplet is generated based on the starting entity, the ending entity, and the entity relationship. The entity relationship and the end point entity corresponding to the initial entity are determined from the knowledge graph by a random walk method, the possibility of generating each triple can be covered more comprehensively, the diversity of the triples is improved, the triples are generated by the knowledge graph, the application range of the triples is wider, the knowledge in each field can be covered, the triples are stored in a triple form, and the storage space is saved.
In an embodiment, as shown in fig. 4, the step S12 of generating the question to be replied through the preset question template according to the entity relationship and the starting entity specifically includes the following steps:
s121: and taking the entity relation as an intention, and identifying a preset problem template from the template database.
Wherein, the intention refers to the potential intention in the question presented by the user. The template database is a database for storing problem templates. The problem template is used for generating a problem to be replied, and the problem template is provided with a reserved slot position for filling an initial entity corresponding to the entity relationship.
Specifically, after the triple is generated, the obtained entity relationship is used as an intention for generating the question to be replied, and according to the intention, a question template corresponding to the intention is identified from the template database.
Exemplarily, assuming that the generated triple is (iPhone11, official price 5499), the entity relationship in the triple is taken as an intention, i.e., "official price" is taken as an intention, the corresponding question template is identified from the template database as "i want to buy () with" official price ", ask what is the official price? ", where () is specifically the starting entity in the triplet corresponding to" official website price ".
S122: and filling the initial entity into a preset problem template by adopting a slot filling mode to generate a problem to be replied.
And filling the slot position by filling the starting entity into the slot position reserved by the problem template. The question to be replied is generated through a preset question template according to the entity relationship and the initial entity.
Specifically, after the entity relationship is used as an intention, a problem template corresponding to the entity relationship is identified from a template database, and then a slot filling mode is adopted to fill the initial entity corresponding to the entity relationship in the triple into a slot reserved in the problem template, so as to generate a problem to be replied.
Exemplarily, assuming a triple of (iPhone11, official price 5499), with "official price" as an intention, the question template identified from the template database is "i want to buy (), ask what is the official price? Filling the initial entity in the triple into a slot reserved in the problem template to generate a problem to be replied. Specifically, "iPhone 11" is filled in (), the question to be replied is generated as "how much do i want to buy iPhone11 asking for the official website price? ".
In this embodiment, a problem template is identified from a template database by having an entity relationship as an intent; and filling the initial entity into the problem template by adopting a slot filling mode to generate the problem to be replied. The corresponding problem template is identified in the template database through the entity relationship, and then the initial entity is filled in the problem template in a slot filling mode, so that the efficiency of generating the problem to be replied is improved, the template corresponds to the entity relationship, and the accuracy of generating the problem to be replied is improved.
In an embodiment, as shown in fig. 5, in step S14, verifying the triples, the questions to be replied, and the reply information by using a deep learning method, and obtaining the classification result specifically includes the following steps:
s141: and respectively carrying out vector coding on the triple, the question to be replied and the reply information to obtain a triple vector, a question vector and a reply information vector.
The vector coding refers to a coding mode of representing words as vectors, and the vector coding can adopt a word2vec model, a glove model, a fasttext model or a bert model. The triple vector is a vector obtained after vector encoding of the triple. The problem vector is obtained by vector coding the problem to be recovered. The reply information vector is a vector obtained by vector coding the reply information.
Preferably, the bert model is adopted to carry out vector coding on the triples, the questions to be replied and the reply information because the bert model is strong in terms of word representation capability. Specifically, the byte number of the question to be encoded and the reply information to be encoded is truncated or filled to the same byte. And splicing the triples to obtain the triples to be coded. Inputting the problem to be coded, the reply information to be coded and the triplet to be coded into a bert vector model for coding, converting the problem to be coded and the reply information to be coded into a vector format, and taking the vector subjected to the bert vector coding as an embedding vector of the triplet, so as to obtain the triplet vector, the problem vector and the reply information vector.
Wherein, the same byte can be 35 bytes, 40 bytes or 45 bytes, etc. Preferably, the byte number of the question to be replied and the reply information is truncated or filled to 40 bytes, because the truncation or the filling to 40 bytes can meet the requirement of retaining the effective byte information of the question to be replied and the reply information, the invalid byte information can be truncated, the redundancy is reduced, and the calculation complexity is reduced. The method for splicing the triples comprises the following steps: the < SEP > is used for dividing between the initial entity and the entity relation of the triple, the < SEP > is also used for dividing between the entity relation of the triple and the end point entity, the < CLS > is spliced at the front end of the initial entity, the < SEP > is spliced at the rear end of the end point entity, and the obtained triple to be coded is (the < CLS > initial entity < SEP > entity relation < SEP > end point entity < SEP >).
S142: and respectively inputting the triple vector, the problem vector and the reply information vector to corresponding GRU modules to obtain a triple result, a problem result and a reply information result.
And the triple result is a result output by the GRU module after the triple vector is input to the corresponding GRU module. The problem result is the result output by the GRU module after the problem vector is input to the corresponding GRU module. The reply information result is the result output by the GRU module after the reply information vector is input to the corresponding GRU module.
Specifically, after the triple vector, the problem vector and the reply information vector are obtained, the triple vector, the problem vector and the reply information vector are respectively input to the corresponding GRU modules, that is, the triple vector is input to the triple GRU module, the problem vector is input to the problem GRU module, and the reply information vector is input to the reply information GRU module, a result output by the triple GRU module is a triple result, a result output by the problem GRU module is a problem result, and a result output by the reply information GRU module is a reply information result.
Parameters are not shared among the three GRU modules, the problem GRU module and the reply information GRU module, and dimensions of results output by the three GRU modules are the same and are all N.
S143: and carrying out dimension splicing on the triple result, the problem result and the reply information result to obtain a splicing result.
The dimension splicing refers to merging data of the same dimension, and the dimension splicing condition is that the dimensions of the spliced data are the same. The splicing result refers to a result obtained by performing dimension splicing on the triple result, the problem result and the reply information result.
Because the dimensions of the results output by the triple GRU module, the problem GRU module and the reply information GRU module are the same and are both N, the condition of dimension splicing is met. Specifically, after a triple result, a problem result and a reply information result are obtained, dimension splicing is performed on the triple result, the problem result and the reply information result to obtain a splicing result. And the dimension of the splicing result is 3N.
S144: and calculating the splicing result through a softmax function to obtain a classification result.
The classification result is a result obtained by verifying the triple, the question to be replied and the reply information, and the classification result includes: the first type of result is that the reply message is not related to the question to be replied, the second type of result is that the reply message is a correct reply message to the question to be replied, and the third type of result is that the reply message is not a correct reply message to the question to be replied.
Specifically, after the splicing result is obtained, the splicing result is input into the full connection layer, a softmax function is accessed to the tail of the full connection layer, the splicing result is calculated through softmax, the probability value of each classification is output, and the classification with the highest probability value is extracted as the classification result according to the probability value of each classification. Wherein the probability value of each classification is integrated to 1.
Illustratively, assuming the triplet is (iPhone11, official price 5499), the question to be replied is "how much do i want to buy iPhone11 asking for official price? "the reply messages of the preset question-answering simulator to the question to be answered may be classified into the following three categories: the first type of reply message is "hello, Hua is P30 with an official price of 5499". The second type of reply message is "hello, the official price of iPhone11 is 5499". The third type of reply message is "hello, the official website price of iPhone11 is 4499". Whether the reply information and the question to be replied are related can be judged through the relationship between the starting entity and the entity; if the reply message is related to the question to be replied, the end entity determines whether the reply message is a correct reply message for the question to be replied.
Assuming that the reply information is the first type reply information, calculating the splicing result through softmax, and outputting the probability value of each classification as follows: and if the probability value of the first class result is 0.9, the probability value of the second class result is 0.07 and the probability value of the third class result is 0.03, the first class result is extracted as a classification result, namely the classification result is that the reply information is irrelevant to the problem to be replied.
Assuming that the reply information is the second type reply information, calculating the splicing result through softmax, and outputting the probability value of each classification as follows: and if the probability value of the first class result is 0.06, the probability value of the second class result is 0.92 and the probability value of the third class result is 0.02, extracting the second class result as a classification result, namely the classification result at this moment is correct reply information of the question to be replied.
Assuming that the reply information is the third type reply information, calculating the splicing result through softmax, and outputting the probability value of each classification as follows: and if the probability value of the first class result is 0.01, the probability value of the second class result is 0.1 and the probability value of the third class result is 0.89, extracting the third class result as a classification result, namely the classification result at this moment is that the reply information is not correct reply information of the problem to be replied.
In this embodiment, vector encoding is performed on the triplets, the questions to be replied and the reply information respectively to obtain a triplet vector, a question vector and a reply information vector; respectively inputting the triple vectors, the problem vectors and the reply information vectors into corresponding GRU modules to obtain triple results, problem results and reply information results; performing dimension splicing on the triple result, the problem result and the reply information result to obtain a splicing result; and calculating the splicing result through a softmax function to obtain a classification result. Vector coding is carried out on the triples, the problems to be replied and the reply information, the representation capability can be improved, a softmax function is adopted for classification, the classification result can be accurately obtained, the classification accuracy is improved, and the calculation complexity is reduced.
In an embodiment, after step S14, that is, after the triple, the question to be replied, and the reply message are verified by using the deep learning method to obtain the classification result, the verification method of the intelligent question-answering system further includes the following steps:
and sending the classification result to a preset question-answer simulator to indicate the preset question-answer simulator to adjust the weight of reply information corresponding to the classification result according to the classification result.
Specifically, the triple, the question to be replied and the reply information are verified by a deep learning method, and after a classification result is obtained, the classification result is sent to a preset question-answer simulator. And after the preset question-answering simulator receives the classification result, generating a corresponding return value according to the classification result, and adjusting the weight of the return information corresponding to the classification result by adopting the return value.
The return value is used for training a preset question-answer simulator, and the return value can be set by a user in a self-defined manner or can be a default value of the system.
Illustratively, if the classification result is a first type result indicating that the reply message is not related to the question to be replied, the return value is set to-10. If the classification result is the second type result indicating that the reply message is a correct reply to the question to be replied, the return value is set to 30. If the classification result is a third type result indicating that the reply message is an incorrect reply to the question to be replied, the return value is set to-30.
Further, the adjusting the weight of the reply information corresponding to the classification result by using the reply value specifically includes:
and if the weight of the reply information is lower than the deletion threshold, deleting the reply information, and generating new reply information according to the triple corresponding to the reply information and the problem to be replied.
And if the weight of the reply information is higher than the storage threshold value, storing the reply information into the corpus database. When a new question-answer simulator exists, the reply information can be sent to the new question-answer simulator, so that the new question-answer simulator has correct reply information corpora.
The deletion threshold is a threshold for determining whether to delete the reply message. The storage threshold is a threshold for determining whether to perform a storage operation on the reply information. The deletion threshold and the storage threshold can be set by a user in a self-defined way or can be default values of the system. Illustratively, the deletion threshold may be set to-30, -60, or-90, etc., and the storage threshold may be set to 30, 60, or 90, etc.
In this embodiment, the classification result is sent to the preset question-answer simulator to instruct the preset question-answer simulator to adjust the weight of the reply message corresponding to the classification result according to the classification result. The classification result is fed back to the preset question-answer simulator, so that the preset question-answer simulator can judge the accuracy of the generated reply information, the reply information which does not meet the requirement is deleted according to the classification result, and the accuracy of the reply information is improved. And storing the reply information meeting the requirements according to the classification result, and sending the reply information to other new question-answering simulators to help the new question-answering simulators to generate correct reply information corpora.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
In an embodiment, a verification device of an intelligent question-answering system is provided, and the verification device of the intelligent question-answering system corresponds to the verification method of the intelligent question-answering system in the embodiment one to one. As shown in fig. 6, the verification apparatus of the intelligent question-answering system includes a first triple generation module 11, a question generation module 12, a question transmission module 13, and a classification result generation module 14. The functional modules are explained in detail as follows:
the first triple generating module 11 is configured to generate a triple according to a preset intention set and a knowledge graph, where the triple includes a starting entity, an entity relationship, and an end entity.
The first question generating module 12 is configured to generate a question to be replied through a preset question template according to the entity relationship and the starting entity.
The question sending module 13 is configured to send a question to be replied to a preset question-answer simulator, and receive reply information of the preset question-answer simulator about the question to be replied.
And the classification result generating module 14 is configured to verify the triples, the questions to be replied, and the reply information by using a deep learning method, so as to obtain a classification result.
Optionally, as shown in fig. 7, the first triple generating module 11 further includes:
the first entity collecting module 111 is configured to collect a starting entity from a preset intention set through a preset sampling manner.
And a second entity collecting module 112, configured to use the starting entity as a starting point, and obtain an entity relationship and an end entity corresponding to the starting entity from the knowledge graph by using a random walk method.
A second triple generating module 113, configured to generate a triple based on the starting entity, the ending entity, and the entity relationship.
Optionally, as shown in fig. 8, the first question generating module 12 further includes:
and the template identification module 121 is configured to identify a preset problem template from the template database with the entity relationship as an intention.
The second problem generating module 122 is configured to fill the starting entity into a preset problem template in a slot filling manner, so as to generate a problem to be replied.
Optionally, as shown in fig. 9, the classification result generating module 14 further includes:
the vector encoding module 141 is configured to perform vector encoding on the triplet, the question to be replied, and the reply information, respectively, to obtain a triplet vector, a question vector, and a reply information vector.
And the neural network module 142 is configured to input the triplet vectors, the problem vectors, and the reply information vectors to the corresponding GRU modules, respectively, so as to obtain triplet results, problem results, and reply information results.
And the dimension splicing module 143 is configured to perform dimension splicing on the triple result, the problem result, and the reply information result to obtain a splicing result.
And the calculation and classification module 144 is configured to calculate the splicing result through a softmax function to obtain a classification result.
Optionally, the verification apparatus of the intelligent question-answering system further includes:
and the result sending module is used for sending the classification result to a preset question-answer simulator.
For the specific limitations of the verification device of the intelligent question-answering system, the above limitations on the verification method of the intelligent question-answering system can be referred to, and are not described herein again. The modules in the verification device of the intelligent question-answering system can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 10. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for the data used in the verification method of the intelligent question answering system in the above embodiment. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of authentication for an intelligent question-answering system.
In one embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the processor implements the authentication method of the intelligent question-answering system in the above embodiments.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the authentication method of the intelligent question answering system in the above-described embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. A verification method of an intelligent question-answering system is characterized by comprising the following steps:
generating a triple according to a preset intention set and a knowledge graph, wherein the triple comprises a starting entity, an entity relation and an end entity;
generating a question to be replied through a preset question template according to the entity relationship and the initial entity;
sending the question to be replied to a preset question-answer simulator, and receiving reply information of the preset question-answer simulator to the question to be replied;
and verifying the triples, the questions to be replied and the reply information by adopting a deep learning method to obtain a classification result.
2. The method for validating an intelligent question-answering system according to claim 1, wherein the generating of the triples according to the preset intention sets and the knowledge graph comprises:
collecting an initial entity from the preset intention set in a preset sampling mode;
taking the initial entity as a starting point, and obtaining an entity relation and an end point entity corresponding to the initial entity from the knowledge graph by adopting a random walk method;
generating a triple based on the starting entity, the ending entity, and the entity relationship.
3. The method for validating an intelligent question-answering system according to claim 1, wherein the step of generating the question to be replied through a preset question template according to the entity relationship and the starting entity comprises:
identifying the preset problem template from a template database by taking the entity relationship as an intention;
and filling the initial entity into the preset problem template by adopting a slot filling mode to generate a problem to be replied.
4. The method for verifying the intelligent question answering system according to claim 1, wherein the step of verifying the triples, the questions to be replied and the reply information by using a deep learning method to obtain a classification result comprises the steps of:
vector coding is respectively carried out on the triples, the questions to be replied and the reply information to obtain a triplet vector, a question vector and a reply information vector;
inputting the triple vectors, the problem vectors and the reply information vectors into corresponding GRU modules respectively to obtain triple results, problem results and reply information results;
performing dimension splicing on the triple result, the problem result and the reply information result to obtain a splicing result;
and calculating the splicing result through a softmax function to obtain a classification result.
5. The method for validating an intelligent question-answering system as claimed in claim 1, wherein after the triple, the question to be replied and the reply information are validated by the deep learning method to obtain the classification result, the method for validating the intelligent question-answering system further comprises:
and sending the classification result to the preset question-answer simulator to indicate the preset question-answer simulator to adjust the weight of reply information corresponding to the classification result according to the classification result.
6. An authentication device of an intelligent question answering system, comprising:
the first triple generating module is used for generating a triple according to a preset intention set and a knowledge graph, wherein the triple comprises a starting entity, an entity relation and an end entity;
the first question generation module is used for generating a question to be replied through a preset question template according to the entity relationship and the starting entity;
the question sending module is used for sending the question to be replied to a preset question-answer simulator and receiving reply information of the preset question-answer simulator to the question to be replied;
and the classification result generation module is used for verifying the triples, the questions to be replied and the reply information by adopting a deep learning method to obtain a classification result.
7. The apparatus for validating an intelligent question-answering system as claimed in claim 6, wherein the first triplet generating module further comprises:
and the first entity acquisition module is used for acquiring a starting entity from the preset intention set in a preset sampling mode.
And the second entity acquisition module is used for acquiring an entity relation and an end entity corresponding to the starting entity from the knowledge graph by taking the starting entity as a starting point and adopting a random walk method.
And the second triple generating module is used for generating a triple based on the starting entity, the end entity and the entity relation.
8. The verification apparatus of the intelligent question answering system according to claim 6, wherein the first question generating module further comprises:
and the template identification module is used for identifying the preset problem template from a template database by taking the entity relationship as an intention.
And the second problem generation module is used for filling the initial entity into the preset problem template in a slot filling mode to generate a problem to be replied.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the authentication method of the intelligent question answering system according to any one of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of the authentication method of the intelligent question-answering system according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010356459.0A CN111666393A (en) | 2020-04-29 | 2020-04-29 | Verification method and device of intelligent question-answering system, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010356459.0A CN111666393A (en) | 2020-04-29 | 2020-04-29 | Verification method and device of intelligent question-answering system, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111666393A true CN111666393A (en) | 2020-09-15 |
Family
ID=72383060
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010356459.0A Pending CN111666393A (en) | 2020-04-29 | 2020-04-29 | Verification method and device of intelligent question-answering system, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111666393A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112328871A (en) * | 2020-10-27 | 2021-02-05 | 深圳集智数字科技有限公司 | Reply generation method, device, equipment and storage medium based on RPA module |
| CN113535449A (en) * | 2021-07-08 | 2021-10-22 | 平安科技（深圳）有限公司 | Abnormal event repairing processing method and device, computer equipment and storage medium |
| WO2022121152A1 (en) * | 2020-12-11 | 2022-06-16 | 平安科技（深圳）有限公司 | Smart dialog method, apparatus, electronic device, and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107315906A (en) * | 2017-06-01 | 2017-11-03 | 北京瑞启医药信息科技有限公司 | The method and system of the automatic follow-up of chronic are realized based on chat robots |
| CN107633060A (en) * | 2017-09-20 | 2018-01-26 | 联想(北京)有限公司 | A kind of information processing method and electronic equipment |
| CN109508420A (en) * | 2018-11-26 | 2019-03-22 | 北京羽扇智信息科技有限公司 | A kind of cleaning method and device of knowledge mapping attribute |
| CN110597969A (en) * | 2019-08-12 | 2019-12-20 | 中国农业大学 | Agricultural knowledge intelligent question and answer method and system and electronic equipment |
| CN111078836A (en) * | 2019-12-10 | 2020-04-28 | 中国科学院自动化研究所 | Machine reading understanding method, system and device based on external knowledge enhancement |
-
2020
- 2020-04-29 CN CN202010356459.0A patent/CN111666393A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107315906A (en) * | 2017-06-01 | 2017-11-03 | 北京瑞启医药信息科技有限公司 | The method and system of the automatic follow-up of chronic are realized based on chat robots |
| CN107633060A (en) * | 2017-09-20 | 2018-01-26 | 联想(北京)有限公司 | A kind of information processing method and electronic equipment |
| CN109508420A (en) * | 2018-11-26 | 2019-03-22 | 北京羽扇智信息科技有限公司 | A kind of cleaning method and device of knowledge mapping attribute |
| CN110597969A (en) * | 2019-08-12 | 2019-12-20 | 中国农业大学 | Agricultural knowledge intelligent question and answer method and system and electronic equipment |
| CN111078836A (en) * | 2019-12-10 | 2020-04-28 | 中国科学院自动化研究所 | Machine reading understanding method, system and device based on external knowledge enhancement |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112328871A (en) * | 2020-10-27 | 2021-02-05 | 深圳集智数字科技有限公司 | Reply generation method, device, equipment and storage medium based on RPA module |
| CN112328871B (en) * | 2020-10-27 | 2024-04-26 | 深圳集智数字科技有限公司 | Reply generation method, device, equipment and storage medium based on RPA module |
| WO2022121152A1 (en) * | 2020-12-11 | 2022-06-16 | 平安科技（深圳）有限公司 | Smart dialog method, apparatus, electronic device, and storage medium |
| CN113535449A (en) * | 2021-07-08 | 2021-10-22 | 平安科技（深圳）有限公司 | Abnormal event repairing processing method and device, computer equipment and storage medium |
| CN113535449B (en) * | 2021-07-08 | 2024-05-03 | 平安科技（深圳）有限公司 | Abnormal event restoration processing method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111859960B (en) | Semantic matching method, device, computer equipment and medium based on knowledge distillation | |
| CN111431742B (en) | Network information detection method, device, storage medium and computer equipment | |
| CN111666393A (en) | Verification method and device of intelligent question-answering system, computer equipment and storage medium | |
| CN109446302A (en) | Question and answer data processing method, device and computer equipment based on machine learning | |
| CN111859986A (en) | Semantic matching method, device, equipment and medium based on multitask twin network | |
| CN110888911A (en) | Sample data processing method and device, computer equipment and storage medium | |
| CN111177307A (en) | Test scheme and system based on semantic understanding similarity threshold configuration | |
| CN113239176A (en) | Semantic matching model training method, device, equipment and storage medium | |
| CN111124898A (en) | Question-answering system testing method and device, computer equipment and storage medium | |
| CN113343711A (en) | Work order generation method, device, equipment and storage medium | |
| CN112765481B (en) | Data processing method, device, computer and readable storage medium | |
| CN116048944A (en) | Load trend prediction method and system for big data platform | |
| CN113360300B (en) | Interface call link generation method, device, equipment and readable storage medium | |
| CN113239272B (en) | Intention prediction method and intention prediction device of network management and control system | |
| CN113868139A (en) | Method and device for analyzing number making accuracy, electronic equipment and storage medium | |
| CN113590786A (en) | Data prediction method, device, equipment and storage medium | |
| CN114548353A (en) | Model training method, electronic device and storage medium | |
| CN112016297A (en) | Intention recognition model testing method and device, computer equipment and storage medium | |
| CN117493530B (en) | Resource demand analysis method, device, electronic equipment and storage medium | |
| CN114490996B (en) | Intention recognition method and device, computer equipment and storage medium | |
| CN116521761B (en) | Sensor operation behavior mining method and system based on artificial intelligence | |
| CN117235236B (en) | Dialogue method, dialogue device, computer equipment and storage medium | |
| CN113298636B (en) | Risk control method, device and system based on simulation resource application | |
| CN116737800A (en) | Big data mining method and system applied to supply chain platform service | |
| CN117668627A (en) | Information processing method, apparatus, device, storage medium, and product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |