WO2021012519A1

WO2021012519A1 - Artificial intelligence-based question and answer method and apparatus, computer device, and storage medium

Info

Publication number: WO2021012519A1
Application number: PCT/CN2019/117954
Authority: WO
Inventors: 朱威; 李恬静
Original assignee: 平安科技（深圳）有限公司
Priority date: 2019-07-19
Filing date: 2019-11-13
Publication date: 2021-01-28
Also published as: CN110532397A; CN110532397B

Abstract

An artificial intelligence-based question and answer method and apparatus, a computer device, and a storage medium: on the basis of first training data, performing language model training and, on the basis of second training data and the trained language model, performing NER model training and relationship matching model training; on the basis of the trained NER model, identifying an entity in a sentence to be processed and, on the basis of the trained relationship matching model, acquiring the relationship corresponding to the sentence to be processed; on the basis of the relationship corresponding to the sentence to be processed and the entity in the sentence to be processed, determining an answer corresponding to the sentence to be processed, and outputting same. Common language model training methods are thus improved on the basis of language model transfer learning and graph transfer learning technology, achieving a higher level of accuracy by means of a smaller amount of manually labelled data, and being more suitable for constructing a knowledge graph question and answer system.

Description

Question answering method, device, computer equipment and storage medium based on artificial intelligence

This application affirms the priority of the Chinese patent application filed on July 19, 2019 with the application number CN201910655550X and titled "Artificial intelligence-based question and answer methods, devices, computer equipment and storage media", the overall content of the Chinese patent application Incorporated in this application by reference.

Technical field

This application relates to the technical field of knowledge graphs, and in particular to a question and answer method, device, computer equipment and storage medium based on artificial intelligence.

Background technique

The knowledge map is also called the scientific knowledge map. It is called the knowledge domain visualization or the knowledge domain mapping map in the library and information industry. It is a series of different graphs showing the relationship between the development process of knowledge and the structure. Because it can provide high-quality structured data, more and more fields will use knowledge graphs and question answering systems based on knowledge graphs, such as automatic question answering, search engines, and information extraction. A typical knowledge graph is usually expressed in the form of head entity, relationship, and tail entity (for example, Yao Ming, nationality, China) of the triad. The expression of this instance reflects the fact that Yao Ming’s nationality is Chinese.

However, the current knowledge graph question and answer technology is still in the stage of exploration and research and development. Most of the results and progress are mainly based on academic papers. The specific plan is: according to the question raised by the user, the corresponding paper or website can be obtained through keyword search in the database In the literature, the user clicks on the specific content of the paper to find the content he needs, which will result in poor processing efficiency for the user to raise the problem, and it cannot meet the user's requirements.

In the knowledge graph question and answer system, whether in an open domain or a vertical domain, accuracy is the main factor that limits its wide application, and the main factor of insufficient accuracy is that the amount of labeling data is too small. Since the annotation of the knowledge graph question answering system includes entity recognition annotation and relationship annotation, the cost of annotating data is huge. In order to quickly build the knowledge graph question and answer system, it is important to reduce the amount of annotation data required.

Summary of the invention

The purpose of this application is to provide a question and answer method, device, computer equipment and storage medium based on artificial intelligence to solve the problems existing in the prior art.

In order to achieve the above objectives, this application provides a question and answer method based on artificial intelligence, including:

Performing language model training based on the first training data, where the first training data is a large number of automatically labeled question corpus in a specified field;

Perform NER model training based on the second training data and the trained language model. The second training data is manually labeled NER data. Each of the NER data includes a question and the manually labeled NER corresponding to the question. mark;

Performing relationship matching model training based on the second training data and the trained language model;

Identify entities in the sentence to be processed based on the trained NER model, and obtain the relationship corresponding to the sentence to be processed based on the trained relationship matching model;

According to the relationship corresponding to the sentence to be processed and the entity in the sentence to be processed, the answer corresponding to the sentence to be processed is determined and output.

In order to achieve the above objective, this application also provides an artificial intelligence-based question and answer device, including:

The language model training module is configured to perform language model training based on first training data, where the first training data is a large number of automatically labeled question corpus in a specified field;

The NER model training module is used to perform NER model training based on the second training data and the trained language model. The second training data is manually labeled NER data. Each of the NER data includes a question and the Manually labeled NER mark corresponding to the question;

The relation matching model training module is configured to perform relation matching model training based on the second training data and the trained language model;

The entity recognition module is used to recognize the entities in the sentence to be processed based on the trained NER model;

A relationship obtaining module, configured to obtain the relationship corresponding to the sentence to be processed based on the trained relationship matching model;

And an answer output module, configured to determine and output the answer corresponding to the sentence to be processed according to the relationship corresponding to the sentence to be processed and the entity in the sentence to be processed.

In order to achieve the above objective, this application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, an artificial intelligence-based The following steps of the question and answer method:

In order to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following steps of the artificial intelligence-based question and answer method are realized:

The artificial intelligence-based question answering method, device, computer equipment and storage medium provided in this application are based on language model transfer learning and graph transfer learning techniques, which improve the commonly used training methods of language models and make them more suitable for the knowledge graph question answering system. It can achieve higher accuracy with a smaller amount of manual labeling data. Specifically, by pre-training the language model, and training the NER model and the relationship matching model based on the trained language model and a small amount of manually labeled data, respectively, to recognize entities and corresponding relationships in the sentence to be processed, where, During the training of the relationship matching model, the map structure is transferred to the relationship matching model through alternate training of the relationship matching model and the knowledge representation model, which can effectively improve the accuracy of the relationship matching model, that is, the accuracy of the corresponding relationship recognition, and achieve a stronger relationship The extraction ability reduces the cost of manual participation and improves the efficiency of constructing knowledge graphs.

Description of the drawings

FIG. 1 is a flowchart of an embodiment of a question answering method based on artificial intelligence in this application;

2 is a schematic diagram of program modules of an embodiment of an artificial intelligence-based question and answer device according to this application;

FIG. 3 is a schematic diagram of the hardware structure of an embodiment of an artificial intelligence-based question and answer device of this application.

Detailed ways

In order to make the purpose, technical solutions, and advantages of this application clearer, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and not used to limit the application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

Example one

Please refer to Figure 1. This application discloses a question and answer method based on artificial intelligence and a question answer method based on artificial intelligence, including the following steps:

S1 performs language model training based on the first training data, in which a large number of unlabeled question corpus in a designated field is collected, and the location of the entity in each question corpus is automatically marked to obtain the first training data.

In step S1, the crawler can actively collect a large number of unlabeled question corpora in designated vertical fields (such as the medical field, marine field and other industries that emphasize the depth of knowledge and require high professionalism). The question corpus is mainly question and answer interactive data , And the larger the corpus, the better (generally no less than 500,000 data); the task of the above language model training is to predict a word in a sentence when the word is occluded. As a preferred solution, in step S1, dictionary matching can be used to automatically mark which part of the sentence is an entity (although it will be somewhat inaccurate). If there is an entity in the sentence, focus the training of the language model on this part, that is, predict a certain word of the entity through the language model, so that the trained language model can well capture the entity information. If the entity is not matched through the dictionary, it is randomly selected.

The language model is to predict what the next word is based on the context, that is, to realize the function of predicting the word when a word in the sentence is occluded. The selection of the language model is not restricted, such as the commonly used google transformer language model or LSMT ( Long Sort Term Memory) language models can be used, and the language model training program can also adopt the conventional training program in this field.

The following uses the language model as the google transformer language model to further explain the training steps in this step. If the language model selects google transformer, the language model training may include the following steps: input the first training data into the google transformer language The embedding layer of the model is vectorized, and then input to the coding layer to obtain the self-attention calculation matrix. The calculation matrix is input to the loss function and based on the Adam optimization algorithm to minimize the loss function of the google transformer language model, and finally save the google transformer language model parameter settings.

S2 performs NER model training based on the second training data and the language model after training. In the training process of the NER model, the parameters of the language model are also regarded as the parameters of the upper-layer NER model and are trained together. In step S2, the network of the NER model is added to the trained language model, and the NER model is trained based on the artificially labeled NER data. During the training process, the parameters of the language model are also regarded as the parameters of the upper NER model and proceed together training. Through gradient descent, update parameters, and repeat the training process, the loss function can be continuously reduced, and better and better predictions can be obtained. Wherein, the second training data is manually labeled NER data, and each of the NER data includes a question sentence and the manually labeled NER mark corresponding to the question sentence; wherein the requirement for the quantity of the second training data is a thousand level OK.

In this embodiment, in step S2, NER model training includes:

S21 input the question sentence in the second training data to the NER model through the language model after training to represent the vector sequence or matrix, so as to output the predicted NER mark;

S22 compares the artificially annotated NER mark with the corresponding predicted NER mark to calculate a loss function; in this embodiment, the loss function is selected as Categorical cross entropy.

S23 updates and optimizes the language model and NER model parameters based on the gradient optimization algorithm.

The above training steps are common to NER models, such as LSTM+CRF models or other entity recognition models.

S3 performs relationship matching model training based on the second training data and the trained language model. During the training process, the question vector and the relationship vector are trained on the relationship matching model for each epoch, and the relationship vector continues to perform a separate epoch knowledge Represents model training, the above-mentioned relation matching model training and knowledge representation model training alternate until all epoch training is completed. Wherein, the question sentence in the second training data is outputted by the language model after the training of the question sentence vector, the relationship in the question sentence in the second training data is randomly initialized by the embedding layer and expressed as a relationship vector, and the above one epoch refers to all The data is sent to the network to complete a forward calculation and back propagation process.

Entities have multiple relationships and attributes in the knowledge graph, but which one matches the question best needs to be confirmed based on the relationship matching model. Generally, it is necessary to select the only correct relationship among 100-200 relationships. Exemplarily, how should atorvastatin be taken to prevent coronary heart disease? The relationship is: how to take <drugs> to prevent and treat <disease>? Both <drug> and <disease> represent medical entities. In this embodiment, through the pre-training of the language model and the pre-training of the knowledge representation, the semantics and the knowledge graph structure are combined, and the relationship matching model is trained for one epoch. During the training process, the language model is also included in the relationship matching model, namely All parameters of the language model are regarded as the parameters of the relational matching model and updated during training; the relational vector is also updated in the relational matching model, and at the end of an epoch, the relational vector is also put into the knowledge representation model such as HolE model (holographic Embeddings of knowledge graphs, knowledge graph holographic embedding representation model) is trained for an epoch. The above relationship matching model training and knowledge representation model training are performed alternately, which can better perform relationship matching.

Relational matching model training includes the following steps:

S31 encode the question in the second training data through the trained language model, and output the question vector; in this step, first copy a trained language model, and then express the question vector Output from the language model, where the requirement on the quantity of the second training data is also a thousand level;

In S32, the relationship in the question sentence in the second training data is randomly initialized as a relationship vector through the embedding layer; in step S32, the relationship vector is randomly initialized in the second training data;

S33 relation matching model training, during the training process, the question vector and relation vector participate in the training of an epoch relation matching model, and based on the gradient optimization algorithm, update and optimize the language model and the parameters of the relation matching model; At the end of each epoch, the relationship vector is put into the knowledge representation model to perform one epoch training, and the above training process is alternated until all epochs are processed.

In step S33, through the pre-training of the language model and the pre-training of the knowledge representation, the semantics and the knowledge graph structure are combined, and the relationship matching model is trained for an epoch. During the training process, the language model is also included in the relationship matching model, that is, language All parameters of the model are regarded as the parameters of the relational matching model, which are updated during training, that is, the language model is fine-tuned; the relational vector is also updated in the relational matching model, and at the end of an epoch, the relational vector is also put into the knowledge representation Models such as Holographic embeddings of knowledge graphs (holographic embeddings of knowledge graphs, knowledge graph holographic embedding representation model) are trained for an epoch. The above-mentioned relationship matching model training and knowledge representation model training are performed alternately, which can better perform relationship matching.

S34 saves the google transformer language model and the parameters of the relationship matching model.

The above-mentioned relationship matching model is not limited, and general semantic matching models are available, for example, based on ESIM (enhanced semantic inference model), or other semantic matching models.

S4 identifies entities in the sentence to be processed based on the trained NER model, and obtains the relationship corresponding to the sentence to be processed based on the trained relationship matching model.

S5 determines and outputs the answer corresponding to the sentence to be processed according to the relationship corresponding to the sentence to be processed and the entity in the sentence to be processed.

It is understandable that each entity will have a corresponding node content on the knowledge graph. In a specific implementation, it is specifically used to search for the entity in the sentence to be processed in the corresponding node on the knowledge graph. The content corresponding to the relationship of the sentence to be processed; the found content is determined as the answer corresponding to the sentence to be processed, and the answer is output.

Therefore, in summary, the artificial intelligence-based question answering method shown in this application, based on language model transfer learning and graph transfer learning technology, improves the commonly used training methods of language models, making it more suitable for graph question answering The system can achieve higher accuracy with less manual labeling data. Specifically, by pre-training the language model, and training the NER model and the relationship matching model based on the trained language model and a small amount of manually labeled data, respectively, to recognize entities and corresponding relationships in the sentence to be processed, where, During the training of the relationship matching model, the map structure is transferred to the relationship matching model through alternate training of the relationship matching model and the knowledge representation model, which can effectively improve the accuracy of the relationship matching model, that is, the accuracy of the corresponding relationship recognition, and achieve a stronger relationship The extraction ability reduces the cost of manual participation and improves the efficiency of constructing knowledge graphs.

Example two

Please continue to refer to FIG. 2. This application shows a question and answer device 10 based on artificial intelligence. In this embodiment, the question and answer device 10 based on artificial intelligence may include or be divided into one or more program modules, one or more Each program module is stored in a storage medium and executed by one or more processors to complete the application and realize the above-mentioned question and answer method based on artificial intelligence. The program module referred to in this application refers to a series of computer program instruction segments capable of completing specific functions, and is more suitable than the program itself to describe the execution process of the artificial intelligence-based question and answer device 10 in the storage medium.

The following description will specifically introduce the functions of each program module in this embodiment:

The artificial intelligence-based question answering device 10 shown in this application includes:

The language model training module 11 is configured to perform language model training based on first training data, where the first training data is a large number of automatically labeled question corpus in a specified field;

The NER model training module 12 is configured to perform NER model training based on the second training data and the trained language model. The second training data is manually labeled NER data, and each of the NER data includes a question and Manually mark the NER mark corresponding to the question;

The relation matching model training module 13 is configured to perform relation matching model training based on the second training data and the trained language model;

The entity recognition module 14 is used to recognize entities in the sentence to be processed based on the trained NER model;

The relationship obtaining module 15 is configured to obtain the relationship corresponding to the sentence to be processed based on the trained relationship matching model;

And the answer output module 16 is used to determine and output the answer corresponding to the sentence to be processed according to the relationship corresponding to the sentence to be processed and the entity in the sentence to be processed.

As a preferred solution, the language model training module 11 includes a first training data acquisition sub-module, which is used to collect a large number of unlabeled question corpus in a specified field, and automatically mark the location of the entity in each question corpus , To obtain the first training data.

Further, in the first training data acquisition submodule, the location of the entity in each question corpus is automatically marked through dictionary matching, and if the question corpus does not match the entity, it is randomly selected.

As a preferred solution, the language model is a google transformer language model, and the language model training module further includes:

The vectorization sub-module is used to input the first training data into the embedding layer of the google transformer language model for vectorization;

The matrix acquisition sub-module is used to input the vector to the coding layer to obtain the self-attention calculation matrix;

The first optimization sub-module is configured to input the calculation matrix into the loss function and update and optimize the parameters of the google transformer language model based on a gradient optimization algorithm;

The first saving sub-module is used to save the google transformer language model parameter settings.

As a preferred solution, the NER model training module 12 includes:

A predictive NER tag acquisition sub-module, configured to express a vector sequence or matrix of the question sentence in the second training data through the language model after training, and input it into the NER model to output a predictive NER tag;

The comparison sub-module is used to compare the artificially annotated NER mark with the corresponding predicted NER mark, and calculate a loss function;

The second optimization sub-module is used to update and optimize the language model and the NER model parameters based on the gradient optimization algorithm.

As a preferred solution, the relationship matching model training module 13 includes:

The question vector obtaining submodule is used to obtain the question vector of the second training data;

The relation vector obtaining sub-module is used to obtain the relation vector of the second training data;

The training sub-module is used to perform the interaction and training of the attention mechanism with the output of the question vector and the relation vector. During the training process, the question vector and the relation vector participate in the training of an epoch relation matching model, and Based on the gradient optimization algorithm, the parameters of the language model and the relationship matching model are updated and optimized; at the same time, at the end of each epoch, the relationship vector is put into the knowledge representation model for one epoch training, alternating the above training Process until all epochs are processed;

The third saving submodule is used to save the language model and the parameters of the relationship matching model.

Further, in the question vector obtaining sub-module, the question sentence in the second training data outputs the question sentence vector through the trained language model.

Further, in the relationship vector obtaining submodule, the relationship in the question sentence in the second training data is randomly initialized by an embedding layer and expressed as a relationship vector.

In summary, the artificial intelligence-based question answering device 10 shown in this application improves the commonly used training methods of language models based on language model transfer learning and graph transfer learning techniques, making it more suitable for performing graph question answering systems. Achieve higher accuracy with a smaller amount of manual labeling data. Specifically, by pre-training the language model, and training the NER model and the relationship matching model based on the trained language model and a small amount of manually labeled data, respectively, to recognize entities and corresponding relationships in the sentence to be processed, where, During the training of the relationship matching model, the map structure is transferred to the relationship matching model through alternate training of the relationship matching model and the knowledge representation model, which can effectively improve the accuracy of the relationship matching model, that is, the accuracy of the corresponding relationship recognition, and achieve a stronger relationship The extraction capability reduces the cost of manual participation and improves the efficiency of constructing knowledge graphs.

Example three

This application also provides a computer device, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a cabinet server (including independent servers, or more A server cluster composed of two servers), etc. The computer device 20 in this embodiment at least includes but is not limited to: a memory 21 and a processor 22 that can be communicatively connected to each other through a system bus, as shown in FIG. 3. It should be pointed out that FIG. 3 only shows the computer device 20 with components 21-22, but it should be understood that it is not required to implement all the illustrated components, and more or fewer components may be implemented instead.

In this embodiment, the memory 21 (ie, readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), Read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 21 may be an internal storage unit of the computer device 20, such as a hard disk or memory of the computer device 20. In other embodiments, the memory 21 may also be an external storage device of the computer device 20, such as a plug-in hard disk, a smart media card (SMC), and a secure digital (Secure Digital, SD card, Flash Card, etc. Of course, the memory 21 may also include both the internal storage unit of the computer device 20 and its external storage device. In this embodiment, the memory 21 is generally used to store an operating system and various application software installed in the computer device 20, such as the program code of the artificial intelligence-based question answering device 10 in the first embodiment. In addition, the memory 21 can also be used to temporarily store various types of data that have been output or will be output.

The processor 22 may be a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 22 is generally used to control the overall operation of the computer device 20. In this embodiment, the processor 22 is used to run the program code or process data stored in the memory 21, for example, to run the question and answer device 10 based on artificial intelligence, so as to implement the question and answer method based on artificial intelligence in the first embodiment.

Example four

This application also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM), Magnetic Memory, Disk, CD, Server, App Store, etc., on which computer programs are stored The corresponding function is realized when executed by the processor. The computer-readable storage medium of this embodiment is used to store the question and answer device 10 based on artificial intelligence, and when executed by a processor, it implements the question and answer method based on artificial intelligence in the first embodiment.

The serial numbers of the foregoing embodiments of the present application are for description only, and do not represent the superiority of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。

The above are only preferred embodiments of this application, and do not limit the scope of this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly used in other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

A question and answer method based on artificial intelligence, characterized in that it comprises the following steps:

Performing language model training based on the first training data, where the first training data is a large number of automatically labeled question corpus in a specified field;

Perform NER model training based on the second training data and the trained language model. The second training data is manually labeled NER data. Each of the NER data includes a question and the manually labeled NER corresponding to the question. mark;

Performing relationship matching model training based on the second training data and the trained language model;

Identify entities in the sentence to be processed based on the trained NER model, and obtain the relationship corresponding to the sentence to be processed based on the trained relationship matching model;

According to the relationship corresponding to the sentence to be processed and the entity in the sentence to be processed, the answer corresponding to the sentence to be processed is determined and output.
The artificial intelligence-based question answering method according to claim 1, wherein a large number of unlabeled question corpus in a designated vertical field is collected through a crawler tool.
The artificial intelligence-based question answering method according to claim 1 or 2, characterized in that the location of the entity in each question corpus is automatically marked through dictionary matching, and if the question corpus does not match the entity, then Randomly selected.
The artificial intelligence-based question answering method according to claim 1, wherein the language model is a google transformer language model, and the google transformer language model training includes:

Inputting the first training data to the middle embedding layer of the google transformer language model for vectorization;

Input the vectorized first training data to the coding layer to obtain the self-attention calculation matrix;

Input the self-attention calculation matrix into the loss function of the google transformer language model, and update and optimize the parameters of the google transformer language model based on a gradient optimization algorithm;

Save the parameter settings of the google transformer language model.
The question answering method based on artificial intelligence according to claim 1, wherein the NER model training comprises:

The question sentence in the second training data is processed by the language model after training to obtain a vector sequence or matrix, and input to the NER model to output a predicted NER label;

Comparing the artificially annotated NER mark with the corresponding predicted NER mark, and calculating the loss function of the NER model;

Based on the gradient optimization algorithm, update and optimize the language model and the NER model parameters.
The question answering method based on artificial intelligence according to claim 1, wherein the relationship matching model training comprises:

Acquiring the question vector and the relation vector of the second training data;

The question vector and the relation vector are output for interaction and training of the attention mechanism. The question vector and the relation vector participate in the training of an epoch relation matching model, and the language is updated and optimized based on the gradient optimization algorithm Model and the parameters of the relationship matching model; at the same time, at the end of each epoch, the relationship vector is put into the knowledge representation model for one epoch training, and the above training process is alternated until all epochs are processed;

Save the language model and the parameters of the relationship matching model.
The question answering method based on artificial intelligence according to claim 6, characterized in that: the question sentence in the second training data outputs the question sentence vector through the trained language model;

The relationship in the question sentence in the second training data is randomly initialized by the embedding layer and expressed as a relationship vector.
A question and answer device based on artificial intelligence, characterized in that it comprises:

The language model training module is configured to perform language model training based on first training data, where the first training data is a large number of automatically labeled question corpus in a specified field;

The NER model training module is used to perform NER model training based on the second training data and the trained language model. The second training data is manually labeled NER data. Each of the NER data includes a question and the Manually labeled NER mark corresponding to the question;

The relation matching model training module is configured to perform relation matching model training based on the second training data and the trained language model;

The entity recognition module is used to recognize the entities in the sentence to be processed based on the trained NER model;

A relationship obtaining module, configured to obtain the relationship corresponding to the sentence to be processed based on the trained relationship matching model;

And an answer output module, configured to determine and output the answer corresponding to the sentence to be processed according to the relationship corresponding to the sentence to be processed and the entity in the sentence to be processed.
The artificial intelligence-based question answering device according to claim 8, wherein the language model training module comprises a first training data acquisition sub-module for collecting a large amount of unlabeled question corpus in a specified field through a crawler tool, And automatically mark the location of the entity in each corpus to obtain the first training data.
The artificial intelligence-based question answering device according to claim 8, wherein in the first training data acquisition sub-module, the location of the entity in each question corpus is automatically marked through dictionary matching, if the If the question corpus does not match the entity, it is randomly selected.
The artificial intelligence-based question answering device according to claim 8, wherein the language model is a google transformer language model, and the language model training module comprises:

The vectorization sub-module is used to input the first training data into the embedding layer (embedding layer) of the google transformer language model for vectorization;

The matrix acquisition sub-module is used to input the vectorized first training data to the coding layer to obtain the self-attention calculation matrix;

The first optimization sub-module is configured to input the calculation matrix into the loss function and update and optimize the parameters of the google transformer language model based on a gradient optimization algorithm;

The first saving sub-module is used to save the google transformer language model parameter settings.
The artificial intelligence-based question answering device according to claim 8, wherein the NER model training module comprises:

A predictive NER tag acquisition sub-module, configured to express a vector sequence or matrix of the question sentence in the second training data through the language model after training, and input it into the NER model to output a predictive NER tag;

The comparison sub-module is used to compare the artificially annotated NER mark with the corresponding predicted NER mark, and calculate a loss function;

The second optimization sub-module is used to update and optimize the language model and the NER model parameters based on the gradient optimization algorithm.
The artificial intelligence-based question answering device according to claim 8, wherein the relationship matching model training module comprises:

The question vector obtaining submodule is used to obtain the question vector of the second training data;

The relation vector obtaining sub-module is used to obtain the relation vector of the second training data;

The training sub-module is used to perform the interaction and training of the attention mechanism with the output of the question vector and the relation vector. During the training process, the question vector and the relation vector participate in the training of an epoch relation matching model, and Based on the gradient optimization algorithm, the parameters of the language model and the relationship matching model are updated and optimized; at the same time, at the end of each epoch, the relationship vector is put into the knowledge representation model for one epoch training, alternating the above training Process until all epochs are processed;

The third saving submodule is used to save the language model and the parameters of the relationship matching model.
The artificial intelligence-based question answering device according to claim 13, wherein in the question vector obtaining submodule, the question sentence in the second training data outputs the question sentence vector through the trained language model .
The artificial intelligence-based question answering device according to claim 13, wherein in the relation vector obtaining sub-module, the relation in the question sentence in the second training data is randomly initialized and expressed as a relation vector by an embedding layer.
A computer device, which is characterized by: comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, the following steps of an artificial intelligence-based question and answer method are implemented :

Performing language model training based on the first training data, where the first training data is a large number of automatically labeled question corpus in a specified field;

Perform NER model training based on the second training data and the trained language model. The second training data is manually labeled NER data. Each of the NER data includes a question and the manually labeled NER corresponding to the question. mark;

Performing relationship matching model training based on the second training data and the trained language model;

Identify entities in the sentence to be processed based on the trained NER model, and obtain the relationship corresponding to the sentence to be processed based on the trained relationship matching model;

According to the relationship corresponding to the sentence to be processed and the entity in the sentence to be processed, the answer corresponding to the sentence to be processed is determined and output.
The question and answer method based on artificial intelligence according to claim 16, wherein the language model is a google transformer language model, and the training of the google transformer language model includes:

Inputting the first training data to the middle embedding layer of the google transformer language model for vectorization;

Input the vectorized first training data to the coding layer to obtain the self-attention calculation matrix;

Input the self-attention calculation matrix into the loss function of the google transformer language model, and update and optimize the parameters of the google transformer language model based on a gradient optimization algorithm;

Save the parameter settings of the google transformer language model.
The question answering method based on artificial intelligence according to claim 16, wherein the NER model training comprises:

The question sentence in the second training data is processed by the language model after training to obtain a vector sequence or matrix, and input to the NER model to output a predicted NER label;

Comparing the artificially annotated NER mark with the corresponding predicted NER mark, and calculating the loss function of the NER model;

Based on the gradient optimization algorithm, update and optimize the language model and the NER model parameters.
A computer-readable storage medium with a computer program stored thereon, characterized in that: when the computer program is executed by a processor, the following steps of an artificial intelligence-based question and answer method are implemented:

Performing language model training based on the first training data, where the first training data is a large number of automatically labeled question corpus in a specified field;

Perform NER model training based on the second training data and the trained language model. The second training data is manually labeled NER data. Each of the NER data includes a question and the manually labeled NER corresponding to the question. mark;

Performing relationship matching model training based on the second training data and the trained language model;

Identify entities in the sentence to be processed based on the trained NER model, and obtain the relationship corresponding to the sentence to be processed based on the trained relationship matching model;

According to the relationship corresponding to the sentence to be processed and the entity in the sentence to be processed, the answer corresponding to the sentence to be processed is determined and output.
The question and answer method based on artificial intelligence according to claim 19, wherein the language model is a google transformer language model, and the training of the google transformer language model includes:

Inputting the first training data to the middle embedding layer of the google transformer language model for vectorization;

Input the vectorized first training data to the coding layer to obtain the self-attention calculation matrix;

Input the self-attention calculation matrix into the loss function of the google transformer language model, and update and optimize the parameters of the google transformer language model based on a gradient optimization algorithm;

Save the parameter settings of the google transformer language model.