CN113434652B - Intelligent question-answering method, intelligent question-answering device, equipment and storage medium - Google Patents
Intelligent question-answering method, intelligent question-answering device, equipment and storage medium Download PDFInfo
- Publication number
- CN113434652B CN113434652B CN202110740056.0A CN202110740056A CN113434652B CN 113434652 B CN113434652 B CN 113434652B CN 202110740056 A CN202110740056 A CN 202110740056A CN 113434652 B CN113434652 B CN 113434652B
- Authority
- CN
- China
- Prior art keywords
- text
- module
- trained
- bert
- encoder
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 239000013598 vector Substances 0.000 claims abstract description 110
- 238000004364 calculation method Methods 0.000 claims abstract description 69
- 238000012549 training Methods 0.000 claims description 65
- 230000006870 function Effects 0.000 claims description 15
- 238000002372 labelling Methods 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 3
- 238000004590 computer program Methods 0.000 claims 3
- 238000007781 pre-processing Methods 0.000 claims 2
- 230000008569 process Effects 0.000 abstract description 7
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 7
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Evolutionary Biology (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Machine Translation (AREA)
Abstract
The application relates to the technical field of artificial intelligence, and provides an intelligent question-answering method, an intelligent question-answering device, equipment and a storage medium, wherein the intelligent question-answering method comprises the following steps: the method comprises the steps of performing feature coding on a target text and a reference text respectively by using a first BERT module and a second BERT module of a similarity calculation model trained in advance to obtain a first sentence vector and a second sentence vector; calculating a similarity value between the target text and the reference text according to the first sentence vector and the second sentence vector by a similarity calculation module of the pre-trained similarity calculation model; and if the similarity value is equal to or greater than a preset threshold value, outputting feedback information aiming at the target text from a preset database according to the reference text. The method and the device realize that when the text similarity calculation is carried out in the intelligent question-answering process, data do not need to be marked, the application range of a text similarity calculation scheme is enlarged, and the efficiency of intelligent question-answering is improved.
Description
Technical Field
The application relates to the technical field of artificial intelligence, in particular to an intelligent question-answering method, an intelligent question-answering device, computer equipment and a storage medium.
Background
In the field of intelligent question and answer, whether the questions of the user are consistent with the entries in the document library or not needs to be judged, and the answer which is most matched with the questions of the user is found to be used as a reply.
In the existing intelligent question-answering field, a common text similarity calculation algorithm generally directly considers the similarity between texts and mostly depends on word and word frequency information. Although text similarity calculation can be performed based on a deep learning model, deeper text information is mined, and text similarity accuracy is improved to a new level, current model training based on deep learning needs to rely on a large amount of annotation data as training samples and follow a complex training process, so that similarity values between a problem text input by a user and a reference text in a preset database can be calculated in an intelligent question-answering process, and the problem of small application range exists.
Disclosure of Invention
Based on the above, it is necessary to provide an intelligent question-answering method, an intelligent question-answering device, equipment and a storage medium for solving the problem that the text similarity calculation scheme in the current intelligent question-answering field needs to rely on a large amount of labeling data, and has a small application range.
A first aspect of an embodiment of the present application provides an intelligent question-answering method, including:
The method comprises the steps of performing feature coding on a target text and a reference text respectively by using a first BERT module and a second BERT module of a similarity calculation model trained in advance to obtain a first sentence vector and a second sentence vector; the target text is a question text input by a user, and the reference text is an entry text in a preset database; calculating a similarity value between the target text and the reference text according to the first sentence vector and the second sentence vector by a similarity calculation module of the pre-trained similarity calculation model; and if the similarity value is equal to or greater than a preset threshold value, outputting feedback information aiming at the target text from a preset database according to the reference text, wherein the feedback information refers to answer text corresponding to the reference text in the preset database, and a mapping relation exists between the reference text and the feedback information.
A second aspect of an embodiment of the present application provides an intelligent question-answering apparatus, including:
Sentence vector acquisition module: the first BERT module and the second BERT module are used for respectively carrying out feature coding on the target text and the reference text by utilizing the similarity calculation model after the pre-training to obtain a first sentence vector and a second sentence vector; the target text is a question text input by a user, and the reference text is an entry text in a preset database; similarity calculation module: the similarity calculation module is used for calculating a similarity value between the target text and the reference text according to the first sentence vector and the second sentence vector through the similarity calculation model trained in advance; the feedback information acquisition module: if the similarity value is equal to or greater than a preset threshold value, outputting feedback information aiming at the target text from the preset database according to the reference text; the feedback information refers to answer text corresponding to a reference text in the preset database, and a mapping relation exists between the reference text and the feedback information.
A third aspect of the embodiments of the present application provides a computer device, including a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, the processor implementing the steps of the intelligent question-answering method described above when executing the computer readable instructions.
A fourth aspect of embodiments of the present application provides one or more readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the intelligent question-answering method as described above.
According to the intelligent question-answering method provided by the embodiment of the application, the first BERT module and the second BERT module of the pre-trained similarity calculation model are utilized to respectively perform feature coding on the target text and the reference text to obtain the first sentence vector and the second sentence vector, wherein the first BERT module and the second BERT module are obtained through the trained variation self-encoder frame, the variation self-encoder frame is trained based on non-labeling data, then the similarity value between the target text and the reference text is calculated according to the first sentence vector and the second sentence vector through the similarity calculation module of the pre-trained similarity calculation model, if the similarity value is equal to or greater than a preset threshold value, feedback information aiming at the target text is output from a preset database according to the reference text, so that a large amount of annotation work is not required to be performed on samples used for model training in the process of text similarity calculation, the application range of a text similarity calculation scheme is enlarged, and the intelligent question-answering efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments of the present application will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic view of an application environment of an intelligent question-answering method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a flow chart of an implementation of the intelligent question-answering method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a similarity calculation model of the intelligent question-answering method in the embodiment of the application;
FIG. 4 is a schematic flow chart of an implementation of the intelligent question-answering method according to another embodiment of the present application;
FIG. 5 is a schematic diagram of a variant self-encoder framework of the intelligent question-answering method in an embodiment of the present application;
FIG. 6 is a schematic diagram of a structure of the intelligent question answering device according to the embodiment of the present application;
Fig. 7 is a block diagram of a computer device in an embodiment of the application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Fig. 1 shows a schematic view of an application environment of the intelligent question-answering method in the embodiment of the application. As shown in fig. 1, the intelligent question-answering method provided by the embodiment of the application can be applied to an application environment as shown in fig. 1 and executed by a server in a server side. Wherein, the client communicates with the server. Clients include, but are not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server may be implemented by a stand-alone server or a server cluster formed by a plurality of servers.
Referring to fig. 2, fig. 2 is a flowchart showing an implementation of the intelligent question-answering method according to an embodiment of the present application, and the method is applied to the server in fig. 1, and includes the following steps:
S11: and respectively carrying out feature coding on the target text and the reference text by using a first BERT module and a second BERT module of the similarity calculation model after pre-training to obtain a first sentence vector and a second sentence vector.
In step S11, the target text refers to the question text input by the user, and the reference text refers to the preset entry text in the database of the intelligent question-answering system. Here, the intelligent question-answering system is a question-answering platform configured at the server side and is used for adapting corresponding reference text and answer content thereof according to the question text input by the user, so as to feed back to the user.
In this embodiment, the first BERT module and the second BERT module are obtained from the encoder framework based on pre-trained variations. The similarity calculation model comprises an input layer, a first BERT module, a second BERT module and a similarity calculation module, wherein a target text is input to the first BERT module through the input layer, and the first BERT module performs feature coding on the target text through an encoder to obtain a vector representation of the target text and records the vector representation as a first sentence vector. The reference text is input to a second BERT module through an input layer, and the second BERT module performs feature coding on the reference text through an encoder to obtain a vector representation of the reference text, and the vector representation is recorded as a second sentence vector.
Fig. 3 is a schematic diagram of a similarity calculation model of the intelligent question-answering method in the embodiment of the application. As shown in fig. 3, as an example, the similarity calculation model 300 includes an input layer 301, a first BERT module 302, a second BERT module 303, and a similarity calculation module 304, where the target text is input to the first BERT module through the input layer 301, the first BERT module performs feature encoding on the target text to obtain a first sentence vector, and the reference text is input to the second BERT module through the input layer 302, and the second BERT module performs feature encoding on the reference text to obtain a second sentence vector.
As another embodiment of the present application, the similarity calculation model is not limited to the first BERT module and the second BERT module, and may include a plurality of BERT modules, for example, three BERT modules, including the first BERT module, the second BERT module, and the third BERT module.
As other embodiments of the present application, the order of the input positions of the target text and the reference text is not limited, for example, the target text may be input to the second BERT module through the input layer, the reference text may be input to the first BERT module through the input layer, where the first BERT module and the second BERT module are obtained from the encoder frame through the trained variation, and the parameter structures are the same.
S12: and calculating the similarity value between the target text and the reference text according to the first sentence vector and the second sentence vector by a similarity calculation module of the similarity calculation model trained in advance.
In step S12, the pre-trained similarity calculation model refers to a similarity calculation model built by the BERT module in the encoder framework based on the trained variation, and the similarity calculation module is configured to calculate the similarity of the vector input to the module, that is, the similarity value between the first sentence vector and the second sentence vector in this embodiment.
In this embodiment, the first sentence vector and the second sentence vector are representations of the target text and the reference text in the vector space, respectively, and the reference value can be calculated as the similarity by calculating the cosine value between the vectors.
As an embodiment of the present application, step S12 includes: calculating cosine similarity of the first sentence vector and the second sentence vector; and taking the cosine similarity as a similarity value between the target text and the reference text.
In this embodiment, the first sentence vector and the second sentence vector are vector representations of the target text and the reference text, respectively. As other embodiments of the present application, the similarity value of the first sentence vector and the second sentence vector may also be calculated by euclidean distance (Euclidean Distance), manhattan distance (MANHATTAN DISTANCE), pearson correlation coefficient (Pearson), or the like. Similarly, the similarity calculation model is not limited to the first BERT module and the second BERT module, and may be multiple BERT modules, and the corresponding sentence vectors encoded by the BERT modules are not limited to the first sentence vector and the second sentence vector, and may be multiple sentence vectors.
S13: and if the similarity value is equal to or greater than a preset threshold value, outputting feedback information aiming at the target text from the preset database according to the reference text.
In this embodiment, if the similarity value between the first sentence vector and the second sentence vector is calculated to be greater than or equal to a preset threshold, that is, the similarity between the target text input by the corresponding user and the reference text in the system database is very high, and the expressions of the two are basically consistent, feedback information (i.e., an answer) corresponding to the reference text is obtained from the database, where the feedback information refers to an answer text corresponding to the reference text in the preset database, and a mapping relationship exists between the reference text and the feedback information, that is, the feedback information (i.e., the answer) corresponding to the target text, so as to complete the intelligent question-answering. The preset threshold is a preset threshold for judging the similarity, and can be set to 90% -95%.
According to the scheme, the first BERT module and the second BERT module of the similarity calculation model after training are utilized to respectively perform feature coding on the target text and the reference text to obtain the first sentence vector and the second sentence vector, wherein the first BERT module and the second BERT module are obtained from the encoder frame through the trained variation, the variation self-encoder frame is trained based on the non-labeling data, then the similarity value between the target text and the reference text is calculated according to the first sentence vector and the second sentence vector through the similarity calculation module of the similarity calculation model after training, and if the similarity value is equal to or greater than a preset threshold value, feedback information aiming at the target text is output from a preset database according to the reference text, so that when text similarity calculation is performed in an intelligent question-answering process, data do not need to be marked, the application range of the text similarity calculation scheme is enlarged, and the intelligent question-answering efficiency is improved.
Referring to fig. 4, fig. 4 is a flowchart illustrating an implementation of an intelligent question-answering method according to another embodiment of the present application. Compared with the corresponding embodiment of fig. 4, the intelligent question-answering method provided in this embodiment further includes S21 to S23 before step S11. The details are as follows:
s21: constructing a variational self-encoder framework based on a pre-trained BERT module;
s22: training the variation self-encoder frame based on sentence training samples to obtain a trained variation self-encoder frame;
s23: and constructing a similarity calculation model based on the trained variation self-encoder framework BERT module.
In step S21, a variational self-encoder framework is built based on the pre-trained BERT module.
In this embodiment, the pre-trained BERT module refers to a BERT module that retains original parameter values and is not trained yet, that is, a BERT model is a model constructed based on Tansformer Encoder, which is a language characterization model, and by means of which a downstream task can be trained, for example: text classification, similarity determination, reading understanding, etc.
In this embodiment, the pre-trained BERT model is preprocessed first, the last layer is removed, and the BERT model is used as a sentence vector encoder based on a transducer mechanism in the BERT model. And respectively taking the preprocessed BERT model as a first pre-trained BERT module and a second pre-trained BERT module, and superposing the first pre-trained BERT module and the second pre-trained BERT module to obtain the variation self-encoder frame.
As an embodiment of the present application, fig. 5 is a schematic diagram of a variant self-encoder framework in an embodiment of the present application, where the variant self-encoder framework 500 includes a first pre-trained BERT module 501, a second pre-trained BERT module 502, and a Softmax function module 503. The first pre-trained BERT module and the second pre-trained BERT module respectively serve as an encoder and a decoder in a variable self-encoder framework, namely, an input text is encoded and converted into a vector through the first BERT module, and a hidden state representation of the input text is obtained through decoding of the second BERT module.
In S22, training the variation self-encoder framework based on the sentence training sample, to obtain a trained variation self-encoder framework.
In this embodiment, the sentence training samples are composed of text without annotation data, and the variation self-encoder frame is trained by continuously inputting text without annotation data until the training purpose is achieved, so as to obtain the trained variation self-encoder frame.
As an embodiment of the present application, step S22 includes:
Randomly deleting part of characters in the text without the marked data to obtain a first text; training the variational self-encoder framework with the first text; the first pre-training BERT module is configured to perform vector conversion on the first text to obtain sentence vectors of the first text; the second pre-trained BERT module is configured to perform text reduction on the sentence vector of the first text to obtain a hidden state representation of the first text; and when the Softmax function module generates a new text according to the hidden state representation of the first text and the text content of the non-labeling data is consistent, training the variation self-encoder frame is completed, and the trained variation self-encoder frame is obtained.
As a specific implementation manner of this embodiment, a group of text without marked data is arranged, and the symbol { X i}＝1,2......N,Xi represents a specific text without marked text, which can be understood as a sentence. For each X i＝[Xi1,Xi2......XiM, wherein X i1 represents a specific word in the text, M is a sentence length constraint, deleting part of the words in X i to obtain a first text, denoted as X ' i, inputting X ' i to a first pre-trained BERT module, obtaining a sentence vector of X ' i by an encoder, then inputting the sentence vector of X ' i to a second pre-trained BERT module, obtaining a final hidden state representation of X ' i by a decoder, finally generating a second text by Softmax function reduction, and is designated X "i. The final goal of training the variance self-encoder framework is to make X i as close as possible, i.e., consistent in content, with X "i, i.e., the unlabeled text, with the resulting second text. When the two contents are consistent, training the variation self-encoder frame is completed, parameters of the trained BERT module are adjusted, sentence vectors which are closest to the original sentence semantics can be obtained, and finally the trained variation self-encoder frame is obtained. The deletion of the unlabeled text, that is, the X i in the embodiment, is to improve the robustness of the trained BERT model, and even if the original text has a part of certainty or is wrong, the correct sentence vector can be generated, and the distance between the original text and the first text is lost by the model.
As an embodiment of the present application, when the Softmax function module generates a new text according to the hidden state representation of the first text and the text content of the non-labeling data is consistent, training of the variation self-encoder framework is completed, and before obtaining the trained variation self-encoder framework, the method further includes: and circularly importing the processed unmarked text to the variable self-encoder framework, and training the variable self-encoder framework. The training of the variation self-encoder framework requires a plurality of non-labeling data texts, and continuous training is carried out until the original non-labeling text of the training target is consistent with the finally obtained second text as much as possible.
In step S23, a similarity calculation model is constructed based on the similarity calculation module and the BERT module in the trained variational self-encoder framework.
In this embodiment, the BERT module in the encoder framework is used as the first BERT module and the second BERT module of the similarity calculation model by using the variation after training. Through training the variation self-encoder framework in the step S22, parameters of the trained BERT module are adjusted, and sentence vectors which are closest to the original sentence semantics can be obtained. And taking the trained BERT module as a module for representing text vectors in a similarity calculation model, obtaining the representation of the target text input by a user and the reference text vectors in a database through encoding, and finally calculating that the similarity accuracy between the vectors is higher. It should be noted that the first BERT module and the second BERT module in the similarity calculation model are also preprocessed because they are derived from the BERT modules in the encoder framework based on the trained variations.
The above scheme, the variation self-encoder framework comprises a first pre-trained BERT module, a second pre-trained BERT module and a Softmax function module; training the variation self-encoder frame based on sentence training samples to obtain a trained variation self-encoder frame; comprising the following steps: randomly deleting part of characters in the text without the marked data to obtain a first text; training the variational self-encoder framework with the first text; the first pre-training BERT module is configured to perform vector conversion on the first text to obtain sentence vectors of the first text; the second pre-trained BERT module is configured to perform text reduction on the sentence vector of the first text to obtain a hidden state representation of the first text; and when the Softmax function module generates a new text according to the hidden state representation of the first text and the text content of the non-labeling data is consistent, training the variation self-encoder frame is completed, and the trained variation self-encoder frame is obtained. The method has the advantages that the deleted unmarked data are utilized to train the variable self-encoder framework, so that even if the original text has partial certainty or the BERT module in the error framework can generate correct and proper sentence vectors, the trained BERT module is applied to the similarity calculation model, and the model is stable and high in accuracy.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present application.
In one embodiment, an intelligent question-answering apparatus 600 is provided, which corresponds to the intelligent question-answering method in the above embodiment one by one. As shown in fig. 6, the intelligent question-answering apparatus includes a sentence vector acquisition module 601, a similarity calculation module 602, and a feedback information acquisition module 603. The functional modules are described in detail as follows:
Sentence vector obtaining module 601: the first BERT module and the second BERT module are used for respectively carrying out feature coding on the target text and the reference text by utilizing the similarity calculation model after the pre-training to obtain a first sentence vector and a second sentence vector;
Similarity calculation module 602: the similarity calculation module is used for calculating a similarity value between the target text and the reference text according to the first sentence vector and the second sentence vector through the similarity calculation model trained in advance;
feedback information acquisition module 603: and if the similarity value is equal to or greater than a preset threshold value, outputting feedback information aiming at the target text from a preset database according to the reference text.
Before the sentence vector obtaining module, the sentence vector obtaining module further comprises a frame construction sub-module, a frame training sub-module and a model construction sub-module, wherein each functional sub-module is described in detail as follows:
the frame construction submodule: for building a variational self-encoder framework based on the pre-trained BERT module;
Frame training submodule: the automatic variable component encoder framework is used for training the automatic variable component encoder framework based on sentence training samples to obtain a trained automatic variable component encoder framework;
model construction submodule: and the similarity calculation model is constructed based on the BERT module in the trained variation self-encoder framework.
Wherein, the frame training submodule includes: the training device comprises a data processing sub-module, a training sub-module and a training result acquisition sub-module, wherein each functional sub-module is described in detail as follows:
And a data processing sub-module: the method comprises the steps of randomly deleting part of characters in the text without the marked data to obtain a first text;
Training submodule: training the variational self-encoder framework with the first text; the first pre-training BERT module is configured to perform vector conversion on the first text to obtain sentence vectors of the first text; the second pre-trained BERT module is configured to perform text reduction on the sentence vector of the first text to obtain a hidden state representation of the first text;
training result acquisition submodule: and when the Softmax function module generates a new text according to the hidden state representation of the first text and the text content of the unmarked data is consistent, training the variation self-encoder framework is completed, and the trained variation self-encoder framework is obtained.
The variational self-encoder framework comprises a first pre-trained BERT module, a second pre-trained BERT module and a Softmax function module, and sentence training samples consist of text without marked data.
For specific limitations of the intelligent question answering device, reference may be made to the above limitation of the intelligent question answering method, and no further description is given here. The modules in the intelligent question answering device can be realized in whole or in part by software, hardware and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a readable storage medium, an internal memory. The readable storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the execution of an operating system and computer-readable instructions in a readable storage medium. The database of the computer device is used for storing data related to only the question-answering method. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer readable instructions when executed by the processor implement a method of intelligent question-answering. The readable storage medium provided by the present embodiment includes a nonvolatile readable storage medium and a volatile readable storage medium.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a readable storage medium, an internal memory. The non-volatile storage medium stores an operating system and computer readable instructions. The internal memory provides an environment for the execution of an operating system and computer-readable instructions in a readable storage medium. The network interface of the computer device is for communicating with an external server via a network connection. The computer readable instructions when executed by the processor implement a method of intelligent question-answering. The readable storage medium provided by the present embodiment includes a nonvolatile readable storage medium and a volatile readable storage medium.
In one embodiment, a computer device is provided that includes a memory, a processor, and computer readable instructions stored on the memory and executable on the processor, when executing the computer readable instructions, performing the steps of:
The method comprises the steps of performing feature coding on a target text and a reference text respectively by using a first BERT module and a second BERT module of a similarity calculation model trained in advance to obtain a first sentence vector and a second sentence vector;
calculating a similarity value between the target text and the reference text according to the first sentence vector and the second sentence vector by a similarity calculation module of the pre-trained similarity calculation model;
And if the similarity value is equal to or greater than a preset threshold value, outputting feedback information aiming at the target text from a preset database according to the reference text.
In one embodiment, one or more computer-readable storage media are provided having computer-readable instructions stored thereon, the readable storage media provided by the present embodiment including non-volatile readable storage media and volatile readable storage media. The readable storage medium has stored thereon computer readable instructions which when executed by one or more processors perform the steps of:
The method comprises the steps of performing feature coding on a target text and a reference text respectively by using a first BERT module and a second BERT module of a similarity calculation model trained in advance to obtain a first sentence vector and a second sentence vector;
calculating a similarity value between the target text and the reference text according to the first sentence vector and the second sentence vector by a similarity calculation module of the pre-trained similarity calculation model;
And if the similarity value is equal to or greater than a preset threshold value, outputting feedback information aiming at the target text from a preset database according to the reference text.
Those skilled in the art will appreciate that implementing all or part of the above described embodiment methods may be accomplished by instructing the associated hardware by computer readable instructions stored on a non-volatile readable storage medium or a volatile readable storage medium, which when executed may comprise the above described embodiment methods. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.
The above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.
Claims (6)
1. An intelligent question-answering method is characterized by comprising the following steps:
Preprocessing a pre-trained BERT model, respectively taking the preprocessed BERT model as a first pre-trained BERT module and a second pre-trained BERT module, superposing the first pre-trained BERT module and the second pre-trained BERT module to obtain a variation self-encoder frame, respectively making an encoder and a decoder in the variation self-encoder frame by the first pre-trained BERT module and the second pre-trained BERT module, and encoding and converting an input text into a vector by the first pre-trained BERT module and decoding by the second pre-trained BERT module to obtain a hidden state representation of the input text;
Training the variation self-encoder frame based on sentence training samples to obtain a trained variation self-encoder frame; the sentence training sample consists of a text without marking data;
The variation self-encoder framework comprises a first pretrained BERT module, a second pretrained BERT module and a Softmax function module;
Training the variation self-encoder frame based on the sentence training sample to obtain a trained variation self-encoder frame; comprising the following steps:
Randomly deleting part of characters in the text without the marked data to obtain a first text;
Training the variational self-encoder framework with the first text; the first pre-training BERT module is configured to perform vector conversion on the first text to obtain sentence vectors of the first text; the second pre-trained BERT module is configured to perform text reduction on the sentence vector of the first text to obtain a hidden state representation of the first text;
when the Softmax function module generates a new text according to the hidden state representation of the first text and the text content of the non-labeling data is consistent, training the variation self-encoder frame is completed, and the trained variation self-encoder frame is obtained;
based on a similarity calculation module and a BERT module in the trained variation self-encoder framework, constructing a similarity calculation model;
Respectively carrying out feature coding on the target text and the reference text by using a first pre-trained BERT module and a second pre-trained BERT module of the pre-trained similarity calculation model to obtain a first sentence vector and a second sentence vector; the target text is a question text input by a user, and the reference text is an entry text in a preset database;
calculating a similarity value between the target text and the reference text according to the first sentence vector and the second sentence vector by a similarity calculation module of the pre-trained similarity calculation model;
If the similarity value is equal to or greater than a preset threshold value, outputting feedback information aiming at the target text from the preset database according to the reference text; the feedback information refers to answer text corresponding to a reference text in the preset database, and a mapping relation exists between the reference text and the feedback information.
2. The intelligent question-answering method according to claim 1, wherein when the Softmax function module generates new text according to the hidden state representation of the first text and the text content of the unlabeled data is consistent, the training of the variation self-encoder framework is completed, and before the trained variation self-encoder framework is obtained, the method further comprises:
and circularly importing the processed unmarked text to the variable self-encoder framework, and training the variable self-encoder framework.
3. The intelligent question-answering method according to claim 1, wherein the similarity calculation module of the similarity calculation model trained in advance calculates a similarity value between the target text and the reference text according to the first sentence vector and the second sentence vector; comprising the following steps:
calculating cosine similarity of the first sentence vector and the second sentence vector;
And taking the cosine similarity as a similarity value between the target text and the reference text.
4. An intelligent question-answering device, comprising:
The frame construction submodule: the method comprises the steps of preprocessing a pre-trained BERT model, respectively taking the preprocessed BERT model as a first pre-trained BERT module and a second pre-trained BERT module, superposing the first pre-trained BERT module and the second pre-trained BERT module to obtain a variation self-encoder frame, respectively making an encoder and a decoder in the variation self-encoder frame by the first pre-trained BERT module and the second pre-trained BERT module, and encoding and converting an input text into a vector by the first pre-trained BERT module and decoding by the second pre-trained BERT module to obtain a hidden state representation of the input text;
Frame training submodule: the automatic variable component encoder framework is used for training the automatic variable component encoder framework based on sentence training samples to obtain a trained automatic variable component encoder framework;
The variation self-encoder framework comprises a first pretrained BERT module, a second pretrained BERT module and a Softmax function module;
The frame training sub-module includes:
and a data processing sub-module: the method comprises the steps of randomly deleting part of characters in a text without marked data to obtain a first text;
Training submodule: training the variational self-encoder framework with the first text; the first pre-training BERT module is configured to perform vector conversion on the first text to obtain sentence vectors of the first text; the second pre-trained BERT module is configured to perform text reduction on the sentence vector of the first text to obtain a hidden state representation of the first text;
training result acquisition submodule: when the Softmax function module generates a new text according to the hidden state representation of the first text and the content of the text without the marked data is consistent, training the variation self-encoder frame is completed, and the trained variation self-encoder frame is obtained;
Model construction submodule: the BERT module is used for constructing a similarity calculation model based on the trained variation self-encoder framework;
Sentence vector acquisition module: the first pre-trained BERT module and the second pre-trained BERT module are used for utilizing the pre-trained similarity calculation model to respectively perform feature coding on the target text and the reference text to obtain a first sentence vector and a second sentence vector; the target text is a question text input by a user, and the reference text is an entry text in a preset database;
Similarity calculation module: the similarity calculation module is used for calculating a similarity value between the target text and the reference text according to the first sentence vector and the second sentence vector through the similarity calculation model trained in advance;
The feedback information acquisition module: and if the similarity value is equal to or greater than a preset threshold value, outputting feedback information aiming at the target text from the preset database according to the reference text.
5. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program when executed by the processor implements the intelligent question-answering method according to any one of claims 1-3.
6. A computer readable storage medium storing a computer program which when executed by a processor implements the intelligent question-answering method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110740056.0A CN113434652B (en) | 2021-06-30 | 2021-06-30 | Intelligent question-answering method, intelligent question-answering device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110740056.0A CN113434652B (en) | 2021-06-30 | 2021-06-30 | Intelligent question-answering method, intelligent question-answering device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113434652A CN113434652A (en) | 2021-09-24 |
| CN113434652B true CN113434652B (en) | 2024-05-28 |
Family
ID=77758244
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110740056.0A Active CN113434652B (en) | 2021-06-30 | 2021-06-30 | Intelligent question-answering method, intelligent question-answering device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113434652B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113850336B (en) * | 2021-09-29 | 2024-09-27 | 平安科技（深圳）有限公司 | Evaluation method, device, equipment and storage medium of semantic similarity model |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110516055A (en) * | 2019-08-16 | 2019-11-29 | 西北工业大学 | A kind of cross-platform intelligent answer implementation method for teaching task of combination BERT |
| CN111626041A (en) * | 2020-05-07 | 2020-09-04 | 杭州东信北邮信息技术有限公司 | Music comment generation method based on deep learning |
| CN113032558A (en) * | 2021-03-11 | 2021-06-25 | 昆明理工大学 | Variational semi-supervised hundred-degree encyclopedia classification method fusing wiki knowledge |
| CN113053357A (en) * | 2021-01-29 | 2021-06-29 | 网易（杭州）网络有限公司 | Speech synthesis method, apparatus, device and computer readable storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109800294B (en) * | 2019-01-08 | 2020-10-13 | 中国科学院自动化研究所 | Autonomous evolution intelligent dialogue method, system and device based on physical environment game |
| US11281863B2 (en) * | 2019-04-18 | 2022-03-22 | Salesforce.Com, Inc. | Systems and methods for unifying question answering and text classification via span extraction |
-
2021
- 2021-06-30 CN CN202110740056.0A patent/CN113434652B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110516055A (en) * | 2019-08-16 | 2019-11-29 | 西北工业大学 | A kind of cross-platform intelligent answer implementation method for teaching task of combination BERT |
| CN111626041A (en) * | 2020-05-07 | 2020-09-04 | 杭州东信北邮信息技术有限公司 | Music comment generation method based on deep learning |
| CN113053357A (en) * | 2021-01-29 | 2021-06-29 | 网易（杭州）网络有限公司 | Speech synthesis method, apparatus, device and computer readable storage medium |
| CN113032558A (en) * | 2021-03-11 | 2021-06-25 | 昆明理工大学 | Variational semi-supervised hundred-degree encyclopedia classification method fusing wiki knowledge |
Non-Patent Citations (1)
| Title |
|---|
| 深度了解自监督学习，就看这篇解读，微软首创：运用在image领域的bert;科技猛兽;《极市平台》;第1-16页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113434652A (en) | 2021-09-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110765763B (en) | Error correction method and device for voice recognition text, computer equipment and storage medium | |
| US11816442B2 (en) | Multi-turn dialogue response generation with autoregressive transformer models | |
| CN111814466B (en) | Information extraction method based on machine reading understanding and related equipment thereof | |
| CN110765265B (en) | Information classification extraction method and device, computer equipment and storage medium | |
| CN111460807B (en) | Sequence labeling method, device, computer equipment and storage medium | |
| CN112633003B (en) | Address recognition method and device, computer equipment and storage medium | |
| CN112307168B (en) | Artificial intelligence-based inquiry session processing method and device and computer equipment | |
| CN112016274B (en) | Medical text structuring method, device, computer equipment and storage medium | |
| CN112766319B (en) | Dialogue intention recognition model training method, device, computer equipment and medium | |
| CN111666775B (en) | Text processing method, device, equipment and storage medium | |
| CN109977394B (en) | Text model training method, text analysis method, device, equipment and medium | |
| CN113987169A (en) | Text abstract generation method, device and equipment based on semantic block and storage medium | |
| CN112528637A (en) | Text processing model training method and device, computer equipment and storage medium | |
| CN111709229B (en) | Text generation method, device, computer equipment and medium based on artificial intelligence | |
| CN111400340B (en) | Natural language processing method, device, computer equipment and storage medium | |
| CN110188926A (en) | A kind of order information forecasting system and method | |
| CN111797611B (en) | Antithetical couplet generation model, antithetical couplet generation method, antithetical couplet generation device, computer equipment and medium | |
| CN112732884A (en) | Target answer sentence generation method and device, computer equipment and storage medium | |
| CN117197271A (en) | Image generation method, device, electronic equipment and storage medium | |
| CN113343711B (en) | Work order generation method, device, equipment and storage medium | |
| CN116469359A (en) | Music style migration method, device, computer equipment and storage medium | |
| CN115906815A (en) | Error correction method and device for modifying one or more types of wrong sentences | |
| CN113434652B (en) | Intelligent question-answering method, intelligent question-answering device, equipment and storage medium | |
| CN114492661A (en) | Text data classification method and device, computer equipment and storage medium | |
| CN112668343B (en) | Text rewriting method, electronic device and storage device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |