CN114003693A

CN114003693A - Question answering method, model training method, equipment and program product thereof

Info

Publication number: CN114003693A
Application number: CN202111277605.1A
Authority: CN
Inventors: 王春宇; 夏源; 施振辉; 代小亚; 黄海峰; 王磊; 陆超
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-10-29
Filing date: 2021-10-29
Publication date: 2022-02-01

Abstract

The scheme provided by the disclosure can inquire the content of an examination point for solving the question in a preset knowledge point base, so that the correct answer of the question can be inferred based on the content of the examination point, and the answer analysis content of the question can be determined according to the content of the examination point. In the embodiment, answers of the questions can be output, answer analysis contents of the questions can also be output, the answer analysis contents are obtained based on the questions and the preset examination point library, and manual pre-labeling is not needed, so that the method is high in generalization capability and can be applied to scenes of answering various questions, the scheme provided by the method can automatically provide answers of the questions and the question analysis contents for users, and user experience is improved.

Description

Question answering method, model training method, equipment and program product thereof

Technical Field

The present disclosure relates to big data and deep learning techniques in artificial intelligence technology, and more particularly, to a question answering method, a model training method, and devices and program products thereof.

Background

There are some application programs with answering functions that users can import into, and which can output answers to, topics. There are various ways in the prior art to generate this type of application.

In one implementation, the model may be trained using deep learning techniques so that the model can output answers to questions, thereby obtaining an application with question answering functionality.

However, in the implementation manner of the prior art, the problem solving application can only output answers to the questions, but cannot output the problem solving process and basis of the questions, which has little reference meaning to the user, resulting in poor user experience.

Disclosure of Invention

The present disclosure provides a question answering method, a model training method, and an apparatus and a program product thereof, thereby outputting answers and analysis contents of questions in an automatic question answering.

According to a first aspect of the present disclosure, there is provided a title solving method, including:

obtaining a question to be processed, and performing word segmentation processing on a question stem of the question and each option to obtain a keyword;

determining examination point contents having an association relation with the question in a preset knowledge point library according to the key words;

and determining answers of the questions according to the question stems, the options and the test point content, determining answer analysis content according to the test point content, and outputting the answers of the questions and the answer analysis content.

According to a second aspect of the present disclosure, there is provided a model training method, comprising:

obtaining questions for training a model, wherein the questions comprise question stems and options;

acquiring answer analysis information of the question, and training an evidence selection model by using the question stem, all options in the question and the answer analysis information of the question; the evidence selection model is used for determining the test point content associated with the to-be-processed question, and the test point content associated with the to-be-processed question is the answer analysis content of the to-be-processed question;

obtaining answers of the questions, determining the content of the test points related to the questions by utilizing the evidence selection model, and training an answer reasoning model according to the question stems, all options in the questions, the answers of the questions and the content of the test points related to the questions; the answer reasoning model is used for determining the answer of the question to be processed.

According to a third aspect of the present disclosure, there is provided a title solving device, including:

the acquisition unit is used for acquiring the question to be processed and performing word segmentation processing on the question stem and each option of the question to obtain a keyword;

the examination point determining unit is used for determining examination point contents which have an association relation with the question in a preset knowledge point base according to the key words;

and the question solving unit is used for determining answers of the questions according to the question stem, the options and the test point content, determining answer analysis content according to the test point content, and outputting the answers of the questions and the answer analysis content.

According to a fourth aspect of the present disclosure, there is provided a title solving device, including:

According to a fifth aspect of the present disclosure, there is provided a model training apparatus comprising:

the question acquisition unit is used for acquiring questions used for training the model, and the questions comprise question stems and options;

the first training unit is used for acquiring answer analysis information of the question and training an evidence selection model by using the question stem, all options in the question and the answer analysis information of the question; the evidence selection model is used for determining the test point content associated with the to-be-processed question, and the test point content associated with the to-be-processed question is the answer analysis content of the to-be-processed question;

the second training unit is used for obtaining answers of the questions, determining the test point content associated with the questions by utilizing the evidence selection model, and training an answer reasoning model according to the question stem, all options in the questions, the answers of the questions and the test point content associated with the questions; the answer reasoning model is used for determining the answer of the question to be processed.

According to a sixth aspect of the present disclosure, there is provided an electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first or second aspect.

According to a sixth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of the first or second aspect.

According to a seventh aspect of the present disclosure, there is provided a computer program product comprising: a computer program, stored in a readable storage medium, from which at least one processor of an electronic device can read the computer program, execution of the computer program by the at least one processor causing the electronic device to perform the method of the first or second aspect.

The question answering method, the model training method, the equipment and the program product thereof can inquire the examination point content for answering the question in the preset knowledge point base, so that the correct answer of the question can be deduced based on the examination point content, and the answer analysis content of the question can be determined according to the examination point content. In the embodiment, answers of the questions can be output, answer analysis contents of the questions can also be output, the answer analysis contents are obtained based on the questions and the preset examination point library, and manual pre-labeling is not needed, so that the scheme provided by the disclosure can automatically provide answers of the questions and the answer analysis contents of the questions for the user, and the user experience is improved.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a schematic illustration of an interactive interface shown in an exemplary embodiment of the present disclosure;

FIG. 2 is a schematic illustration of an interactive interface shown in another exemplary embodiment of the present disclosure;

FIG. 3 is a schematic flow chart diagram illustrating a topic solving method according to an exemplary embodiment of the present disclosure;

FIG. 4 is an interactive interface diagram of a topic input process shown in an exemplary embodiment of the present disclosure;

FIG. 5 is an interactive interface diagram of a topic input process shown in another exemplary embodiment of the present disclosure;

FIG. 6 is a schematic flow chart of a topic solving method according to another embodiment of the present disclosure;

FIG. 7 is a diagram illustrating a topic solving process according to an exemplary embodiment of the present disclosure;

FIG. 8 is a flow chart diagram illustrating a model training method according to an exemplary embodiment of the present disclosure;

FIG. 9 is a schematic flow chart diagram of a model training method according to another exemplary embodiment of the present disclosure;

FIG. 10 is a schematic view showing a structure of a title solving apparatus according to an exemplary embodiment of the present disclosure;

FIG. 11 is a schematic view showing a structure of a title solving device according to another exemplary embodiment of the present disclosure;

FIG. 12 is a schematic diagram illustrating a model training apparatus according to an exemplary embodiment of the present disclosure;

FIG. 13 is a schematic diagram illustrating a model training apparatus according to another exemplary embodiment of the present disclosure;

FIG. 14 is a block diagram of an electronic device used to implement methods of embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1 is a schematic view of an interactive interface shown in an exemplary embodiment of the present disclosure.

As shown in fig. 1, an application having a problem solving function may be provided in the electronic device, and the user may operate the electronic device to run the application.

The user can enter the topic to be processed in the application program. For example, the title can be input by taking a picture, and can also be input by text. The application may output the answer to the topic, such as the answer being either option a, option B, or the like.

Such problem solving programs can only provide answers to the questions, which does not contribute much to the user's ability to solve the problems.

Fig. 2 is a schematic view of an interactive interface shown in another exemplary embodiment of the present disclosure.

As shown in fig. 2, there is another solving application which can output a topic answer after a user inputs a topic, and the answer parsing contents. However, the answer analysis content is artificially labeled in advance, only the questions can be solved, and the generalization does not exist.

For example, a large number of questions are stored in the background service of the question solving application, and the expert staff marks answers to the questions. When the questions input by the user belong to preset questions, the background service can acquire the pre-labeled answers and feed the answers back to the question solving application.

However, the method for manually labeling the answer to analyze the content is inefficient and has insufficient generalization. And if the question input by the user does not belong to the preset question, the question solving application cannot output the answer analysis content of the question.

In order to solve the technical problem, in the scheme provided by the disclosure, the examination point content related to the question can be queried according to the question to be processed, the queried examination point content can be used for determining the answer of the question, the queried examination point content can be used as the answer analysis content, and then the information capable of improving the user question solving capability is output, so that the user experience is improved.

FIG. 3 is a flowchart illustrating a topic solving method according to an exemplary embodiment of the present disclosure.

As shown in fig. 3, the title answering method provided by the present disclosure includes:

step 301, obtaining a topic to be processed, and performing word segmentation processing on the topic stem of the topic and each option to obtain a keyword.

The method provided by the present disclosure may be executed by an electronic device with computing capability, and the electronic device may be a user terminal, such as a mobile phone, a computer, a tablet computer, and the like. And the system can also be a server, such as a cloud server, a distributed server and the like.

Specifically, the electronic device may obtain a topic to be processed, and the electronic device may determine an answer and an answer analysis content of the topic.

Further, if the electronic device is a user terminal, the user can input the title, so that the electronic device obtains the title to be processed.

FIG. 4 is an interactive interface diagram of a topic input process shown in an exemplary embodiment of the present disclosure.

As shown in FIG. 4, the user can click on location 41 and perform text editing to enter a title.

FIG. 5 is an interactive interface diagram of a topic input process shown in another exemplary embodiment of the present disclosure.

As shown in fig. 5, the user can click on the position 51 to trigger the electronic device to display a picture input title or take a picture to obtain a title. If the user selects to input a title through a picture, the electronic device may display a folder for storing the picture, in which the user may select the picture. If the user selects the mode of obtaining the questions through photographing, the electronic equipment can start the camera, and therefore the questions are input through the photographing mode based on user operation.

In practical application, if the electronic device is a background server for problem solving application, the user terminal may send the problem to be processed to the server. For example, after a user inputs a title in a user terminal, the user terminal may send the title to a server, so that the server obtains the title to be processed.

After the electronic equipment acquires the to-be-processed title, word segmentation processing can be performed on the to-be-processed title, and then keywords of the title are obtained.

Specifically, the subjects to be processed may include a stem and a plurality of options, for example, one type of qualifying test subject includes 1 stem and 5 options, and another type of qualifying test subject includes 1 stem and 4 options. The electronic equipment can perform word segmentation processing on the question stem and each option of the question to obtain a plurality of keywords.

Furthermore, a word segmentation algorithm can be preset, and word segmentation processing is performed on the to-be-processed question through the preset word segmentation algorithm.

Step 302, according to the keywords, determining the content of the test points having the association relation with the keywords in a preset knowledge point library.

In practical application, a knowledge point base can be preset, and the knowledge point base stores examination point contents related to the to-be-processed question. For example, a knowledge point library can be generated according to the test outline, so that the content of test points for solving the problems can be inquired in the knowledge point library.

Wherein, the knowledge point base can be set according to the question range for the solution provided by the method of the present disclosure. For example, if the method provided by the present disclosure is used for solving the problem of a doctor qualification test, a knowledge point library may be generated according to the outline of the doctor qualification test, and if the method provided by the present disclosure is used for solving the problem of a teacher qualification test, a knowledge point library may be generated according to the outline of the teacher qualification test.

Specifically, the content of the test outline can be organized to form a plurality of test points, and each test point can be mapped into a quintuple of < test point, title, source, test point content, unique identifier >, thereby forming a knowledge point library.

Further, a knowledge point base can be set based on the ElasticSearch (ES, Apache Lucene (TM) based open source search engine), and an inverted index structure and a word segmentation device can also be set in the knowledge point base. In one embodiment, when a topic to be processed is obtained, the topic may be segmented based on a segmenter in the ES, and the examination point content related to the topic is screened based on an inverted index structure in the ES.

In practical application, if multiple examination point contents are obtained from the preset knowledge point library, the examination point content with the strongest relevance with the question can be screened from the multiple examination point contents by combining the questions to be processed, and step 303 is executed based on the examination point content with the strongest relevance.

And determining the content of the test point with the strongest association with the question from the screened content of the test points according to all the keywords of the question to be processed. An evidence selection model can be preset, the questions to be processed and the screened test point contents can be input into the evidence selection model, and then the test point contents with strong relevance with the questions can be obtained.

The number of the determined test point contents can be one or more, and the test point contents are used for reasoning answers of the questions.

Step 303, determining answers of the questions according to the question stems, the options and the test point contents, determining answer analysis contents according to the test point contents, and outputting answers of the questions and the answer analysis contents.

Specifically, after the examination point content having an association relationship with the question is determined, the answer to the question can be determined according to the question and the examination point content.

Furthermore, as the question comprises a plurality of options, whether each option is correct can be determined according to the question stem and the content of the examination point, so that the correct answer of the question can be determined.

In practical application, an answer reasoning model for reasoning correct answers can be preset. The question stem, the options and the examination point content can be input into an answer reasoning model, the reasoning result of each option is output, and then the correct answer of the question can be determined based on the reasoning result. For example, the stem, a choice in the question and the content of the test point can be input into the answer reasoning model as input data, and the answer reasoning model can output the reasoning result of the input data, such as correct or wrong.

Wherein the electronic device can determine a correct answer to the topic based on the inference result for each option.

If a plurality of test point contents for reasoning the answers of the questions are determined, the plurality of test point contents can be combined to determine whether each option is correct.

Specifically, the electronic device may further determine answer analysis content according to the test point content, for example, the test point content may be directly used as the answer analysis content of the question, and if the answer to the question is inferred based on the multiple test point contents, the answer analysis content may be generated according to the multiple test point contents, for example, the multiple test point contents may be spliced to obtain the answer analysis content.

Further, the electronic device may further output the correct answer and the answer analysis content of the question, for example, when the electronic device is a user terminal, the user terminal may display the correct answer and the answer analysis content of the question, and when the electronic device is a server, the server may feed back the correct answer and the answer analysis content of the question to the user terminal. In the embodiment, the answers of the questions can be inferred, the analysis content of the answers of the questions can be output, and further richer question solving information is provided for the user.

The title answering method provided by the disclosure comprises the following steps: obtaining a question to be processed, and performing word segmentation processing on a question stem and each option of the question to obtain a keyword; determining examination point contents having an association relation with the keywords in a preset knowledge point library according to the keywords; and determining answers of the questions according to the question stems, the options and the test point contents, determining answer analysis contents according to the test point contents, and outputting the answers of the questions and the answer analysis contents. The question answering method provided by the disclosure can inquire the examination point content for answering the question in the preset knowledge point base, so that the correct answer of the question can be inferred based on the examination point content, and the answer analysis content of the question can be determined according to the examination point content. In the embodiment, answers of the questions can be output, and answer analysis contents of the questions can also be output, and the answer analysis contents are obtained based on the questions and the preset examination point library and do not need manual pre-labeling, so that the method provided by the disclosure can automatically provide answers of the questions and the answer analysis contents of the questions for the user, and the user experience is improved.

FIG. 6 is a schematic flow chart of a topic solving method according to another embodiment of the present disclosure.

As shown in fig. 6, the title answering method provided by the present disclosure includes:

step 601, obtaining a topic to be processed, and performing word segmentation processing on the topic stem and each option of the topic to obtain a keyword.

Step 601 is similar to the implementation of step 301, and is not described again.

Step 602, screening out first examination point contents associated with the keywords in a knowledge point base.

The first examination point content can be screened out from a preset knowledge point base according to the keywords. The knowledge point base may be preset.

Specifically, for example, M first test point contents may be screened from the knowledge point library in advance, where the first test point contents are test point contents associated with keywords of the title.

Furthermore, similarity between the keywords and the content of the test points in the knowledge point base can be determined, and the first content of the test points related to the keywords is screened out from the knowledge point base according to the similarity. The similarity algorithm may be preset, and may be, for example, the BM25 algorithm. The similarity between the keywords and the content of the examination points in the knowledge point base can be determined through a preset similarity algorithm.

In practical application, the first content of the test points may be screened according to the determined similarity result, for example, M content of test points with the highest similarity may be screened as the first content of the test points.

In this embodiment, a certain number of test point contents which may have an association relationship with a topic can be screened from the knowledge point library, and when the test point contents having an association with a topic are determined, the range of the test point contents can be narrowed, so that the calculation amount for determining the test point contents having an association with a topic is reduced.

In an alternative embodiment, the index structure of the knowledge point base may also be preset, for example, the index structure may be an inverted index structure. A certain number of test point contents can be retrieved from the knowledge point library by utilizing the inverted index structure and the keywords of the titles, and then M test point contents with higher correlation with the keywords are screened from the test point contents.

Step 603, combining the question stem, all the options and each first examination point content of the question to obtain a question evidence candidate set corresponding to each first examination point content.

The first examination point content can be continuously screened, and second examination point content which is more strongly related to the question is screened out. Specifically, an evidence selection model may be preset to screen out second content of the first content of the test points.

Specifically, the question stem, all the options and the content of each first examination point of the question can be combined to obtain a question evidence candidate set. For example, the topic evidence candidate sets for < (topic stem, all options), first point of interest content > can be combined. A set of topic evidence candidates can be generated for each first point of reference content. For example, 50 first test point contents of the topics are screened from the knowledge point library, and then 50 topic evidence candidate sets can be obtained by combination.

Step 604, inputting each question evidence candidate set into a preset evidence selection model to obtain second test point content having an association relation with the question; the evidence selection model is used for determining second examination point content of the topic; and the second test point content is the test point content which has an association relation with the title.

Further, the evidence selection model can output an evaluation result of each topic evidence candidate set, and further can determine second examination point content having an association relation with the topic according to the evaluation result of each topic evidence candidate set.

The electronic device can input each topic evidence candidate set into the evidence selection model, and then obtain the evaluation result of each topic evidence candidate set output by the evidence selection model. The evaluation result may be, for example, a score or the like, and is used to characterize the association between the topics in the topic evidence candidate set and the first test point content.

Specifically, the electronic device may screen out the second content of the test point from the first content of the test point according to the evaluation result, for example, the electronic device may screen out the second content of the test point with stronger association with the question from the first content of the test point according to the evaluation result. For example, if the number of the first examination point contents is M, M question evidence candidate sets may be obtained by combination, and then M evaluation results may be obtained. The electronic device may determine N high-confidence evaluation results according to the M evaluation results, and determine the first point of interest content corresponding to the N high-confidence evaluation results as the second point of interest content.

For example, 100 first test point contents can be screened from a preset knowledge point library, and then 5 second test point contents are determined in the first test point contents by using an evidence selection model, wherein the 5 second test point contents are used for reasoning answers of questions and generating answer parsing contents of the questions.

In this embodiment, first test point content which may have an association relationship with a question may be screened from a knowledge point library, so as to narrow the range of the test point content, then a second test point content for reasoning the question answer and determining the answer analysis content is screened from the first test point content by using an evidence selection model, and a question evidence candidate set input into the evidence selection model includes a question stem of the question and all options, so that the second test point content which has a strong association with the complete question can be determined from the first test point content by using the evidence selection model.

Step 605, inputting the question stem, the options and the test point content associated with the question into a preset answer reasoning model to obtain an answer to the question, wherein the answer reasoning model is used for determining the answer to the question.

Specifically, an answer reasoning model can be preset, and the answer reasoning model is used for reasoning answers of the questions. The answer inference model may be derived through training.

Further, the electronic device can input the stem, the options and the content of the test point into a preset answer reasoning model, and the answer reasoning model can output an evaluation result of each option, wherein the evaluation result is used for representing the probability that the option is a correct answer. The content of the test point here may be, for example, the content of the second test point determined by step 604.

In practical application, the electronic equipment can determine the correct answer of the question according to the evaluation result output by the answer reasoning model. For example, the answer reasoning model can output scores of the options, and the electronic device can use the option with the highest score as the correct answer to the topic.

In the embodiment, the electronic equipment can reason the answer to the question according to the determined examination point content with strong relevance to the question, so that an answer result can be obtained in an automatic mode, and the accurate answer to the question can be obtained by reasoning based on the examination point content.

The electronic equipment can combine the question stem, each option and the examination point content to obtain problem solving information corresponding to each option. For example, the problem solving information of < (one option in question), the content of the examination point > can be obtained by combination.

Specifically, if the electronic device determines a plurality of second examination point contents, the examination point contents may be spliced to obtain examination point contents for solving the problems. Each solution information obtained by the electronic device combination may include the content of the test point, that is, include a plurality of second test point contents.

In another embodiment, if the electronic device determines a plurality of second examination point contents, the electronic device may further combine the question stem, each option, and each examination point content to obtain answer information corresponding to each option and each examination point content, for example, if it is determined that two second examination point contents are respectively an examination point content a and an examination point content B, the questions have 5 options, and the option a, the option B, the option C, the option D, and the option E are provided, the electronic device may combine to obtain 10 answer information. Further, the problem solving information generated by the electronic device corresponds to options of the topic, for example, if one topic includes 5 options, 5 pieces of problem solving information of the topic can be generated.

In practical application, the electronic equipment can input each solution question information into an answer reasoning model to obtain an answer label corresponding to each solution question information; the electronic device can determine an answer to the question based on the answer label.

After the answer information is input into the answer reasoning model, the answer reasoning model can output an answer label of each answer information. The answer labels may be, for example, probability values that characterize the probability that the choices in the solution information are correct answers. The option in the problem solving information with the highest probability value can be used as the correct answer, and the examination point content in the problem solving information with the highest probability value can be used as the basis for solving the problems.

Specifically, the electronic device may determine the answer to the question according to the answer label of each solution information. For example, if a question includes 5 options, 5 pieces of solution information can be obtained by combining the 5 options, the 5 pieces of solution information can be input into the answer reasoning model, and the answer reasoning model outputs answer labels of the 5 pieces of solution information. The problem solving information including the correct answers can be determined according to the answer labels, and options in the problem solving information are determined as the correct answers of the problems.

In this embodiment, the question stem, each option and the content of the test point for solving the question can be used as input data to input into the answer reasoning model, so that the correct answer of the question can be deduced based on the content of the test point with strong relevance to the question to obtain the correct answer of the question.

Step 606, determining answer analysis content according to the examination point content.

Step 607, outputting the topic answer and the answer analysis content.

The implementation manner of step 606 and step 607 is similar to that of step 303, and is not described again.

FIG. 7 is a diagram illustrating a topic solving process according to an exemplary embodiment of the present disclosure.

As shown in fig. 7, after obtaining the topic to be processed, the electronic device may first perform word segmentation on the topic to obtain a keyword, and then obtain first examination point content associated with the keyword from a preset knowledge point database 71.

The first point of interest content, the subject stem of the subject, and the options can be input into the evidence selection model 72 to obtain a second point of interest content with a strong association with the subject.

Then the question stem, the options and the second examination point content of the question are input into the answer reasoning model 73, and then the question answer can be obtained.

The electronic device can output the topic answers and the answer parsing contents.

Fig. 8 is a flowchart illustrating a model training method according to an exemplary embodiment of the present disclosure.

As shown in fig. 8, the model training method provided by the present disclosure is used for training an evidence selection model and an answer reasoning model, which can be used for solving questions, wherein the evidence selection model can be used for determining the content of a test point for solving the questions, and the answer reasoning model is used for reasoning answers to the questions according to the content of the test point.

The method specifically comprises the following steps:

step 801, obtaining topics for training a model, wherein the topics comprise topic stems and options.

The method provided by the present disclosure is executed by an electronic device with computing capability, such as a server.

Specifically, training data for training the model may be set, and the data may be specifically a topic. The questions include a question stem and options.

For example, if a topic includes A, B, C, D, E five options, then the topic includes a stem and 5 options.

Further, the questions used for training the model may be real examination questions or simulation questions, for example, new simulation questions may be generated according to the existing examination questions.

In practice, the topic may also have answers and/or answer parsing information. For example, a preset topic can have an answer, e.g., the correct answer for the topic is option B. For another example, the questions may also have answer parsing information, and the answer parsing information may be pre-labeled, for example, a part of the test questions is collected, and the answer parsing information of the real questions may be pre-labeled.

In an alternative embodiment, if the topic is a generated simulation topic, the topic may include only the answer without answer parsing information.

If the obtained question has an answer, an answer reasoning model can be trained based on the question, and if the obtained question has answer analysis content, a model can be selected based on the question training evidence. If the acquired question has both an answer and answer parsing content, an answer reasoning model and an evidence selection model may be trained based on the question.

Step 802, obtaining answer analysis information of the question, and training an evidence selection model by using the question stem, all options in the question and the answer analysis information of the question; the evidence selection model is used for determining the test point content associated with the to-be-processed question, and the test point content associated with the to-be-processed question is the answer analysis content of the to-be-processed question.

In practical application, the question may further have answer parsing information, and the answer parsing information may be information for solving the question or may be examination point content unrelated to the question.

The model can be set up in advance, and the model is trained by taking a sample comprising question and answer analysis information as training data to obtain an evidence selection model. The sample may be a positive sample or a negative sample, and if the answer parsing information in the sample can be used for solving the question, the sample may be determined to be the positive sample, and if the answer parsing information in the sample is not related to the question, the sample may be determined to be the negative sample.

Specifically, the sample used for training the evidence selection model may further have a label for characterizing positive and negative attributes of the sample, for example, when the sample is positive, the label may be 1, and when the sample is negative, the label may be 0.

Further, the model may be trained using samples including topic and answer parsing information as training data. The model can determine the relevance between the answer analysis information and the question according to the question stem of the question and all the options, and compares the determined result with the sample label, thereby adjusting the parameters in the model based on the comparison result. Through the mode of gradual adjustment, can train and obtain comparatively accurate evidence selection model.

In practical application, the electronic equipment can construct a loss function according to the relevance result determined by the model and the sample label to determine the difference between the model output result and the real result, and when the difference meets the preset condition, the model can be considered to be trained completely to obtain an evidence selection model.

Step 803, obtaining answers of the questions, determining examination point contents associated with the questions by using an evidence selection model, and training an answer reasoning model according to the question stem, all options in the questions, the answers of the questions and the examination point contents associated with the questions; the answer reasoning model is used for determining the answer of the question to be processed.

In practice, the topic can also have an answer, for example, the B-option in the topic is a correct answer to the topic.

Wherein, the evidence selection model can be used to determine the content of the examination point associated with the title. The method specifically comprises the steps of performing word segmentation processing on a question stem and each option of a question to obtain a keyword; then screening first examination point contents associated with the keywords from a preset knowledge point library, and combining the question stem, all options and each first examination point content of the question to obtain a question evidence candidate set corresponding to each first examination point content; each question evidence candidate set can be input into a preset evidence selection model, and the examination point content with a strong association relation with the question is obtained.

Specifically, after the examination point content of the question is obtained, an answer reasoning model can be trained according to the question and the examination point content.

Further, the question includes a plurality of options, wherein a part of the options are answers to the question, and another part of the options are wrong answers, so that whether the option is a correct answer or not can be used as a label for each option, for example, if one option is a correct answer, the label is 1, and if one option is not a correct answer, the label is 0.

In practical application, the sample for training the answer reasoning model can include a stem of a question, one option in the question and the content of a test point, and the sample also has a label corresponding to the option.

A model can be set up in advance, and the model is trained by taking a sample comprising a stem, an option and the content of a test point as training data to obtain an answer reasoning model. The sample can be a positive sample or a negative sample, if the option in the sample is a correct answer, the sample is the positive sample, the corresponding label is the label corresponding to the positive sample, if the option in the sample is not the answer to the question, the sample can be determined to be the negative sample, and the corresponding label is the label corresponding to the negative sample.

Furthermore, the model can reason whether the options are correct or not according to the content of the examination points and the question stem, and compares the determined result with the sample label, so as to adjust the parameters in the model based on the comparison result. Through the mode of gradual adjustment, a more accurate answer reasoning model can be obtained through training.

In practical application, the electronic equipment can establish a loss function according to the result that whether the options are correct determined by the model and the sample label so as to determine the difference between the output result of the model and the real result, and when the difference meets the preset condition, the model can be considered to be trained completely, so that the answer inference model is obtained.

The evidence selection model and the answer reasoning model trained based on the model training method provided by the disclosure can be applied to the embodiments shown in fig. 3 or fig. 6, the test point content with strong relevance with the question can be determined based on the two models, the question answer can also be deduced, the determined test point content can be used as the answer analysis content of the question, and then the question answer and the answer analysis content of the question can be output during the question solving.

The method provided by the disclosure is a model training method, comprising the following steps: obtaining questions for training a model, wherein the questions comprise question stems and options; acquiring answer analysis information of the questions, and training an evidence selection model by using the question stem, all options in the questions and the answer analysis information of the questions; the evidence selection model is used for determining the examination point content associated with the to-be-processed question, and the examination point content associated with the to-be-processed question is the answer analysis content of the to-be-processed question; obtaining answers of the questions, determining examination point contents associated with the questions by utilizing an evidence selection model, and training an answer reasoning model according to the question stem, all options in the questions, the answers of the questions and the examination point contents associated with the questions; the answer reasoning model is used for determining the answer of the question to be processed. The method provided by the disclosure can train to obtain an evidence selection model for determining the test point content associated with the to-be-processed question, the test point content determined by the model and associated with the to-be-processed question is the answer analysis content of the to-be-processed question, and then the answer analysis content of the question can be output during solving the question. And an answer reasoning model can be obtained through training, and the answer reasoning model can reason the answer of the question according to the content of the test point with strong relevance with the question, so that a more accurate answer can be obtained.

FIG. 9 is a schematic flow chart diagram illustrating a model training method according to another exemplary embodiment of the present disclosure.

As shown in fig. 9, the model training method provided by the present disclosure includes:

step 901, a first preset topic is obtained from a preset topic library, and a second preset topic related to the first preset topic test point is retrieved from the preset topic library.

The question bank can be preset, and collected questions, such as real questions of a test in a calendar year, which can be qualified by a doctor, can be stored in the question bank. In order to enrich data for training the model, other topics can be generated according to the existing topics in the preset topic library.

Specifically, a first preset topic can be obtained from a preset topic library, and the first preset topic can be any topic in the topic library.

Furthermore, retrieval recall can be carried out in a preset topic library to obtain a second preset topic related to the first preset topic. For example, a retrieval recall may be performed based on the question stem, and a second preset question with a higher relevance to the question stem of the first preset question may be screened from the question bank. And the retrieval recall can be carried out based on the options, and a second preset topic with higher relevance to the options of the first preset topic can be screened from the topics.

And 902, generating other questions according to the first preset question and the second preset question, wherein the other questions are used for training an evidence selection model and an answer reasoning model.

In practical application, the first preset theme and the second preset theme can be recombined to obtain other themes. The other topics are simulation topics, but the information in the other topics is obtained through the topics in the topic library, so the other topics also include the examination point content related to the true topic, and can also be used for training the model.

The number of questions which can be collected by the mode of collecting the questions is small, for example, only real questions of an examination in a past year are collected, the number of the questions is limited, and a large amount of sample data is needed in model training.

Specifically, a first preset topic and a second preset topic can be combined to generate another topic, and a large number of other topics can be obtained in this way.

Further, when the first preset question and the second preset question are recombined, a first question stem and a first answer of the first preset question can be obtained specifically, and a second option of the second preset question is obtained; and combining the first question stem, the first answer and the second option to generate other questions.

For example, the first preset questions include a question stem 1, an option 1A, an option 1B, an option 1C, and an option 1D, where the correct answer is the option 1C, and a plurality of second preset questions related to the first preset questions may be retrieved from the preset question library, where one of the second preset questions includes a question stem 2, an option 2A, an option 2B, an option 2C, and an option 2D. The stem 1 of the first preset topic and the option 1C of the correct answer may be retained, and the remaining options are randomly selected from the second preset topic, and combined to obtain a new other topic, for example, the options 2A, 2B, and 2C are randomly selected, and the combined other topic includes the stem 1, and the four options of the option 1C, 2A, 2B, and 2C.

In actual application, the model can be trained by using the generated other questions and the original questions in the preset question library.

Step 903, obtaining a preset question from a preset question library, translating a Chinese question stem in the preset question into question stems of other language types, and translating the question stems of other language types into Chinese question stems.

In another embodiment, other topics can be obtained by performing language conversion on the topic stem to enrich data for training the model.

The method comprises the steps of obtaining a preset question from any one of preset question libraries, carrying out multiple language type conversion on the preset question, and finally translating the preset question into a Chinese question stem. For example, the Chinese stem can be obtained by translating the stem into English, then into Korean, and then into Chinese.

Specifically, the number of times of translation of the question stem is not limited, and the Chinese question stem can be obtained through translation finally. However, since the excessive number of translations may cause the sentence with the stem to be unsmooth, a threshold value of the number of translations may be set in advance, for example, 4 translations at most.

Furthermore, the sentence sequence of the translated Chinese question stem can be adjusted, so that the readability of the Chinese question stem is better, and the question stem can be discarded if the translated Chinese question stem sentences are not smooth or unclear in meaning. For example, it can be determined whether the obtained Chinese question stem is semantically clear and is suitable as a question stem based on a semantic recognition technology.

And 904, combining the translated Chinese question stem with options of preset questions to generate other questions, wherein the other questions are used for training the model.

In practical application, if the translated Chinese question stem is suitable for being used as the question stem of the question, the Chinese question stem and the options of the preset question can be combined to obtain other questions. In actual application, the model can be trained by using the generated other questions and the original questions in the preset question library.

Step 905, generating a simulation question according to the entity relationship pair and the label thereof in the preset knowledge graph, wherein the simulation question comprises a question stem and options; the knowledge graph comprises an entity relation pair between the knowledge information and the knowledge answers, and the label is used for representing whether the corresponding relation is correct or not.

The knowledge graph can be preset, the SPO triples in the knowledge graph can be extracted, each triplet represents a pair of entity relations, and the knowledge graph comprises an entity relation pair between knowledge information and a knowledge answer. For example, there is an entity relationship that "the test for liver failure is" - "the serum prealbumin assay", wherein the knowledge information is "the test for liver failure is" and the knowledge answer is "the serum prealbumin assay".

Specifically, each entity relationship pair further has a label for indicating whether the corresponding relationship of the entity relationship pair is correct, for example, if the corresponding relationship between "liver failure test is correct" - "serum prealbumin determination" is correct, the corresponding relationship of the entity relationship pair can be indicated as correct.

Further, the electronic device may generate a simulation question according to a preset knowledge graph and a tag thereof, for example, if the tag is 1, the entity relationship pair is correct, if the tag is 0, the entity relationship pair is incorrect, and if a radio question needs to be generated, 1 entity relationship pair with a tag of 1 and 3 entity relationship pairs with a tag of 0 may be selected from the knowledge graph, so as to generate the simulation question.

In practical application, the generated simulated subjects can be used for training the model.

Step 906, acquiring the content of the positive sample examination point of the question according to the preset labeling information of the question; and/or obtaining the content of the negative sample examination point of the topic in a preset knowledge point base.

In order to train a model with an accurate recognition result, the model needs to be trained by using a positive sample and a negative sample together.

Specifically, when the evidence selection model is trained, the content of the test point included in the positive sample has strong relevance with the question, and the answer of the question can be deduced based on the content of the test point. The relevance of the test point content and the question included in the negative sample is weak, and the correct answer of the question cannot be deduced based on the test point content.

Furthermore, the questions can be labeled in advance, for example, the questions in a preset question library can be labeled by an expert to obtain answer analysis contents of each question, and the electronic device can obtain preset labeling information so as to obtain the positive sample examination point content of the question. The positive sample test point content obtained in this way is the test point content from which correct answers can be inferred. When the expert annotates the preset questions, the expert can annotate the preset questions based on the content in the knowledge point base.

In practical application, the content of the negative sample examination point of the topic can be obtained from a preset knowledge point library.

For example, a knowledge point may be randomly extracted from a knowledge point library to serve as the examination point content of a question, and since the relevance between the randomly extracted examination point content and the question is poor, the examination point content of the correct answer cannot be deduced from the negative sample examination point content obtained in this way.

In an optional implementation manner, after the content of the examination point is randomly acquired from the knowledge point library, the content of the examination point may be compared with the content of the positive sample examination point of the question, and if the content is similar to the content of the positive sample examination point of the question, the content of the examination point may not be used as the content of the negative sample examination point of the question.

For another example, the content of the test points related to the topic may be obtained from a preset knowledge point library, for example, the topic may be split to obtain a plurality of keywords, the content of the test points having an association relationship with each keyword is obtained from the preset knowledge point library through a similarity algorithm, and the content of other test points than the content of the positive sample test points in the related test points is used as the content of the negative sample test points. For example, if 100 pieces of test point content related to the topic are screened out through the keywords, wherein 3 pieces of content are used for reasoning the answer of the topic, the remaining 97 pieces of test point content can be used as negative sample test point content of the topic.

In the embodiment, the model can be trained by using the examination point content which is related to the question but can not deduce the answer of the question, so that the identification accuracy of the trained model is improved.

For another example, the content of the test point can be randomly acquired from a preset knowledge point library according to the question stem of the question and the options of other questions, and the acquired content of the relevant test point is used as the content of the negative sample test point. Reference questions can be obtained according to the obtained question stems of the questions and option combinations of other questions, examination point contents related to the reference questions are searched in a preset knowledge point library, and the examination point contents can be used as negative sample examination point contents of the questions.

Step 907, determining sample labels according to the examination point content of the questions.

If the examination point content of the question is the positive sample examination point content, the sample label of the examination point content can be determined to be positive, and if the examination point content is the negative sample examination point content, the sample label of the examination point content can be determined to be negative.

And 908, generating evidence sample data according to the question stem, all options in the question, the content of the examination point and the sample label.

Specifically, all options in the question stem and the question, the content of the examination point and the sample label can be combined to generate evidence sample data. For example, evidence sample data of < (stem, all options), test point content, sample tags > can be constructed. Each piece of evidence sample data may include one or a combination of a plurality of content of the test point.

Further, each topic can have a plurality of point of interest contents, which can include point of interest contents of positive samples and negative samples. Thus, multiple evidence sample data can be generated for one topic.

And step 909, training a pre-built first model by using the evidence sample data to obtain an evidence selection model.

The first model can be set up in advance, constructed evidence sample data is input into the first model, the first model can determine the relevance between answer analysis information and questions according to the question stem and all options in the evidence sample data, the determined result is compared with the sample label, and then parameters in the model are adjusted based on the comparison result. Through the mode of gradual adjustment, can train and obtain comparatively accurate evidence selection model.

In practical application, the electronic equipment can construct a loss function according to the relevance result determined by the model and the sample label to determine the difference between the model output result and the sample label, and when the difference meets the preset condition, the model can be considered to be trained completely to obtain an evidence selection model.

Step 910, performing word segmentation on the stem of the topic and each option to obtain a keyword.

Step 911, screening out the first examination point content associated with the keyword from the knowledge point base.

Step 912, combining the question stem, all the options and each first examination point content of the question to obtain a question evidence candidate set corresponding to each first examination point content.

Step 913, inputting each question evidence candidate set into a preset evidence selection model to obtain the examination point content having an association relationship with the question.

The specific implementation manner of step 910-913 is similar to that described in the embodiment of fig. 6, and is not repeated herein.

After the evidence selection model is obtained through training, the model can be used for determining the content of the test points associated with the various questions, and the content of the test points is used for reasoning the correct answers of the questions.

Step 914, obtaining the answer of the question, and determining a reasoning label corresponding to each option in the question according to the answer of the question.

The correct answers of the questions can be preset, if the options of the questions are correct answers, the inference label of the options can be determined to be positive, and if the options of the questions are not correct answers, the inference label of the options can be determined to be negative. For example, if a topic has A, B, C, D four options, where option B is a correct answer, it can be determined that the inference label of option B is positive and the inference label of option A, C, D is negative.

And 915, generating a reasoning training sample corresponding to each option according to the question stem of the question, each option, the examination point content associated with the question and the reasoning label corresponding to each option.

Specifically, an option in the question stem and the question, the examination point content associated with the question and the inference label of the option can be combined to generate an inference training sample. For example, inference training samples of < (stem, an option), the content of the test points associated with the topics, and inference labels > can be constructed. Each inference training sample can comprise a question option, if the number of the test point contents associated with the questions is multiple, the multiple test point contents can be combined, and the combined data is used as the test point contents in the inference training sample.

And step 916, training a pre-established second model by using the inference sample data to obtain an answer inference model.

The second model can be pre-established, constructed inference sample data is input into the second model, whether options in the inference sample data are correct answers can be inferred according to the contents of the question stem and the examination point in the inference sample data by the second model, the determined result is compared with the inference label in the inference sample data, and then parameters in the model are adjusted based on the comparison result. Through the mode of gradual adjustment, a more accurate answer reasoning model can be obtained through training.

In practical application, the electronic equipment can construct a loss function according to the result determined by the model and the inference label so as to determine the difference between the output result of the model and the inference label, and when the difference meets the preset condition, the model can be considered to be trained completely to obtain an answer inference model.

FIG. 10 is a schematic structural diagram of a topic solving device according to an exemplary embodiment of the present disclosure.

As shown in fig. 10, the title solving device 1000 provided by the present disclosure includes:

an obtaining unit 1010, configured to obtain a topic to be processed, and perform word segmentation processing on a topic stem of the topic and each option to obtain a keyword;

an examination point determining unit 1020, configured to determine, according to the keyword, examination point content having an association relationship with the question in a preset knowledge point library;

the problem solving unit 1030 is configured to determine an answer to the question according to the question stem, each option, and the test point content, determine an answer analysis content according to the test point content, and output the question answer and the answer analysis content.

The question answering device provided by the disclosure can inquire the examination point content for answering questions in the preset knowledge point base, so that correct answers of the questions can be deduced based on the examination point content, and the answer analysis content of the questions can be determined according to the examination point content. In the embodiment, answers of the questions can be output, and answer analysis contents of the questions can also be output, and the answer analysis contents are obtained based on the questions and the preset examination point library and do not need manual pre-labeling, so that the device provided by the disclosure can automatically provide answers of the questions and the answer analysis contents of the questions for the user, and the user experience is improved.

FIG. 11 is a schematic structural view of a title solving device according to another exemplary embodiment of the present disclosure.

As shown in fig. 11, in the title answering device 1100 provided by the present disclosure, the acquisition unit 1110 is similar to the acquisition unit 1010 in fig. 10, the test point determination unit 1120 is similar to the test point determination unit 1020 in fig. 10, and the solution unit 1130 is similar to the solution unit 1030 in fig. 10.

In an optional implementation, the examination point determining unit 1120 includes:

a screening module 1121, configured to screen out, from the knowledge point library, first examination point content associated with the keyword;

a first combining module 1122, configured to combine the question stem, all the options, and each of the first examination point contents of the question to obtain a question evidence candidate set corresponding to each of the first examination point contents;

the evidence determining module 1123 is configured to input each question evidence candidate set into a preset evidence selection model, so as to obtain second test point content having an association relationship with the question; wherein the evidence selection model is used for determining second examination point content of the topic; and the second test point content is the test point content which has an association relation with the question.

In an optional implementation manner, the screening module 1121 is specifically configured to:

determining similarity between the keywords and the content of the test points in the knowledge point base, and screening the first content of the test points related to the keywords in the knowledge point base according to the similarity.

In an optional implementation, the problem solving unit 1130 is specifically configured to:

and inputting the question stem, the options and the test point content into a preset answer reasoning model to obtain an answer of the question, wherein the answer reasoning model is used for determining the answer of the question.

In an alternative embodiment, the problem solving unit 1130 includes:

the second combination module 1131 is configured to combine the question stem, each option, and the examination point content to obtain problem solving information corresponding to each option;

the label identification module 1132 is configured to input each solution question information into the answer reasoning model to obtain an answer label corresponding to each solution question information;

an answer determining module 1133, configured to determine the answer to the question according to the answer tag.

FIG. 12 is a schematic diagram illustrating a model training apparatus according to an exemplary embodiment of the present disclosure.

As shown in fig. 12, the present disclosure provides a model training apparatus 1200 including:

the question acquiring unit 1210 is used for acquiring a question for training a model, wherein the question comprises a question stem and options;

a first training unit 1220, configured to acquire answer parsing information of a question, and train an evidence selection model using the question stem, all options in the question, and the answer parsing information of the question; the evidence selection model is used for determining the test point content associated with the to-be-processed question, and the test point content associated with the to-be-processed question is the answer analysis content of the to-be-processed question;

the second training unit 1230 is configured to obtain answers of the questions, determine the content of the test points associated with the questions by using the evidence selection model, and train an answer inference model according to the question stem, all options in the questions, the answers of the questions, and the content of the test points associated with the questions; the answer reasoning model is used for determining the answer of the question to be processed.

The model training device provided by the disclosure can be used for training to obtain an evidence selection model used for determining the test point content associated with the to-be-processed question, the test point content determined by the model and associated with the to-be-processed question is the answer analysis content of the to-be-processed question, and then the answer analysis content of the question can be output during question solving. And an answer reasoning model can be obtained through training, and the answer reasoning model can reason the answer of the question according to the content of the test point with strong relevance with the question, so that a more accurate answer can be obtained.

Fig. 13 is a schematic structural diagram of a model training apparatus according to another exemplary embodiment of the present disclosure.

As shown in fig. 13, the present disclosure provides a model training device 1300 in which a topic acquisition unit 1310 is similar to the topic acquisition unit 1210 in fig. 12, a first training unit 1320 is similar to the first training unit 1220 in fig. 12, and a second training unit 1330 is similar to the second training unit 1230 in fig. 12.

In an alternative embodiment, as shown in fig. 13, the title obtaining unit 1310 includes:

the question acquisition module 1311 is configured to acquire a first preset question in a preset question library, and retrieve a second preset question related to the first preset question examination point in the preset question library;

a question generating module 1312 is configured to generate other questions according to the first preset question and the second preset question, where the other questions are used for training the evidence selection model and the answer reasoning model.

In an optional implementation manner, the topic generation module 1312 is specifically configured to:

acquiring a first question stem and a first answer of the first preset question, and acquiring a second option of the second preset question;

and combining the first question stem, the first answer and the second option to generate other questions.

In an optional implementation manner, the title obtaining unit 1310 includes:

the translation module 1313 is configured to obtain a preset question from a preset question library, translate a Chinese question stem in the preset question into question stems of other language types, and translate question stems of other language types into a Chinese question stem;

and the topic combination module 1314 is configured to combine the translated Chinese topic stem with the options of the preset topic to generate other topics, where the other topics are used for training a model.

In an optional implementation manner, the title obtaining unit 1310 includes:

the simulation question generation module 1315 is configured to generate a simulation question according to an entity relationship pair and a label thereof in a preset knowledge graph, where the simulation question includes a question stem and an option; the knowledge graph comprises an entity relation pair between knowledge information and a knowledge answer, and the label is used for representing whether the entity relation pair is correct or not.

In an alternative embodiment, the first training unit 1320 includes:

a positive sample obtaining module 1321, configured to obtain, according to the preset labeling information of the question, positive sample examination point content of the question;

and/or, a negative sample obtaining module 1322, configured to obtain negative sample examination point content of the topic from a preset knowledge point library.

In an alternative embodiment, the negative sample obtaining module 1322 is specifically configured to:

randomly acquiring examination point content from a preset knowledge point library to serve as negative sample examination point content of the question;

or acquiring the test point content related to the question from a preset knowledge point library, and taking other test point contents except the positive sample test point content in the related test point content as negative sample test point contents;

or randomly acquiring the content of the test point from a preset knowledge point library according to the question stem of the question and the options of other questions, and taking the acquired related test point content as the content of the negative sample test point.

In an alternative embodiment, the first training unit 1320 includes:

a sample label determining module 1323, configured to determine a sample label according to the evidence information of the topic;

an evidence sample generating module 1324, configured to generate evidence sample data according to the question stem, all options in the question, the evidence information, and the sample label;

and the first training module 1325 is configured to train a pre-built first model by using the evidence sample data to obtain the evidence selection model.

In an alternative embodiment, the second training unit 1330 includes:

a word segmentation module 1331, configured to perform word segmentation on the stem of the question and each option to obtain a keyword;

an association module 1332, configured to screen out, from a preset knowledge point library, first examination point content associated with the keyword;

an information combination module 1333, configured to combine the question stem, all the options, and each of the first examination point contents of the question to obtain a question evidence candidate set corresponding to each of the first examination point contents;

and the test point determining module 1334 is configured to input each topic evidence candidate set into a preset evidence selection model, so as to obtain the test point content having an association relationship with the topic.

In an alternative embodiment, the second training unit 1330 includes:

a reasoning label determining module 1335, configured to determine a reasoning label corresponding to each option in the question according to the answer to the question;

a reasoning sample generating module 1336, configured to generate a reasoning training sample corresponding to each option according to the question stem of the question, each option, the examination point content associated with the question, and the reasoning label corresponding to each option;

and a reasoning model training module 1337, configured to train a pre-built second model with the reasoning sample data to obtain the answer reasoning model.

The present disclosure provides a question answering method, a model training method, a device thereof, and a program product, which are applied to a big data and deep learning technology in an artificial intelligence technology to output answers and analysis contents of questions during automatic question answering.

In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the personal information of the related user are all in accordance with the regulations of related laws and regulations and do not violate the good customs of the public order.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

According to an embodiment of the present disclosure, the present disclosure also provides a computer program product comprising: a computer program, stored in a readable storage medium, from which at least one processor of the electronic device can read the computer program, the at least one processor executing the computer program causing the electronic device to perform the solution provided by any of the embodiments described above.

FIG. 14 shows a schematic block diagram of an example electronic device 1400 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 14, the device 1400 includes a computing unit 1401 that can perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM)1402 or a computer program loaded from a storage unit 1408 into a Random Access Memory (RAM) 1403. In the RAM 1403, various programs and data required for the operation of the device 1400 can also be stored. The calculation unit 1401, the ROM 1402, and the RAM 1403 are connected to each other via a bus 1404. An input/output (I/O) interface 1405 is also connected to bus 1404.

Various components in device 1400 connect to I/O interface 1405, including: an input unit 1406 such as a keyboard, a mouse, or the like; an output unit 1407 such as various types of displays, speakers, and the like; a storage unit 1408 such as a magnetic disk, optical disk, or the like; and a communication unit 1409 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 1409 allows the device 1400 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The computing unit 1401 may be a variety of general purpose and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 1401 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The computing unit 1401 performs the respective methods and processes described above, such as the title solution method, the model training method. For example, in some embodiments, the topic solution method, the model training method, can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1408. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 1400 via ROM 1402 and/or communication unit 1409. When the computer program is loaded into the RAM 1403 and executed by the computing unit 1401, one or more steps of the topic solution method, the model training method described above can be performed. Alternatively, in other embodiments, the computing unit 1401 may be configured to perform the topic solution method, the model training method, by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server can be a cloud Server, also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A question answering method includes:

2. The method according to claim 1, wherein determining, according to the keyword, the content of the test point having an association relationship with the topic in a preset knowledge point library comprises:

screening out first examination point contents associated with the keywords from the knowledge point base;

combining the question stem, all options and each first examination point content of the question to obtain a question evidence candidate set corresponding to each first examination point content;

inputting each question evidence candidate set into a preset evidence selection model to obtain second test point content having an association relation with the question; wherein the evidence selection model is used for determining second examination point content of the topic; and the second test point content is the test point content which has an association relation with the question.

3. The method of claim 2, wherein screening the knowledge point base for first point of interest content corresponding to the keyword comprises:

4. The method of any one of claims 1-3, wherein said determining a topic answer from the stem, the options, and the test point content comprises:

and inputting the question stem, the options and the test point content associated with the question into a preset answer reasoning model to obtain the answer of the question, wherein the answer reasoning model is used for determining the answer of the question.

5. The method of claim 4, wherein the inputting the stem, the option and the test point content into a preset answer reasoning model to obtain the answer of the question comprises:

combining the question stem, each option and the examination point content to obtain problem solving information corresponding to each option;

inputting each solution question information into the answer reasoning model to obtain an answer label corresponding to each solution question information;

and determining the answer of the question according to the answer label.

6. A model training method, comprising:

7. The method of claim 6, wherein the obtaining topics for training a model comprises:

acquiring a first preset question from a preset question library, and searching a second preset question related to the first preset question examination point from the preset question library;

and generating other questions according to the first preset question and the second preset question, wherein the other questions are used for training the evidence selection model and the answer reasoning model.

8. The method of claim 7, wherein generating other topics from the first and second preset topics, the other topics being used to train a model comprises:

9. The method of claim 6, wherein the obtaining topics for training a model comprises:

acquiring a preset question from a preset question library, translating a Chinese question stem in the preset question into question stems of other language types, and translating the question stems of other language types into a Chinese question stem;

and combining the Chinese question stem obtained by translation and the options of the preset questions to generate other questions, wherein the other questions are used for training the model.

10. The method of claim 6, wherein the obtaining topics for training a model comprises:

generating a simulation question according to an entity relationship pair and a label thereof in a preset knowledge graph, wherein the simulation question comprises a question stem and options; the knowledge graph comprises an entity relation pair between knowledge information and a knowledge answer, and the label is used for representing whether the entity relation pair is correct or not.

11. The method according to any one of claims 6-10, wherein the obtaining answer resolution information for a topic comprises:

acquiring the content of a positive sample examination point of the subject according to preset labeling information of the subject;

and/or obtaining the content of the negative sample examination point of the subject from a preset knowledge point base.

12. The method of claim 11, wherein the obtaining of the negative sample examination point content of the topic in a preset knowledge point base comprises:

13. The method of any of claims 6-12, wherein the training an evidence selection model using the stem and all options in the topic and the evidence information for the topic comprises:

determining a sample label according to the evidence information of the subject;

generating evidence sample data according to the question stem, all options in the question, the evidence information and the sample label;

and training a pre-built first model by using the evidence sample data to obtain the evidence selection model.

14. The method of claim 6, wherein said determining, using the evidence selection model, point of interest content associated with the topic comprises:

performing word segmentation processing on the subject stem of the subject and each option to obtain a keyword;

screening out first examination point content associated with the keywords from a preset knowledge point base;

and inputting each question evidence candidate set into a preset evidence selection model to obtain the test point content having an association relation with the question.

15. The method of claim 6, wherein the training of an answer reasoning model from the stem, all of the options in the topic, the answer to the topic, and the test point content associated with the topic comprises:

determining a reasoning label corresponding to each option in the question according to the answer of the question;

generating a reasoning training sample corresponding to each option according to the question stem of the question, each option, the examination point content associated with the question and a reasoning label corresponding to each option;

and training a pre-built second model by using the inference sample data to obtain the answer inference model.

16. A topic solving apparatus comprising:

17. The apparatus of claim 16, wherein the examination point determining unit comprises:

the screening module is used for screening out first examination point contents related to the key words from the knowledge point base;

the first combination module is used for combining the question stem, all options and each first examination point content of the question to obtain a question evidence candidate set corresponding to each first examination point content;

the evidence determining module is used for inputting each question evidence candidate set into a preset evidence selection model to obtain second test point content which has an association relation with the question; wherein the evidence selection model is used for determining second examination point content of the topic; and the second test point content is the test point content which has an association relation with the question.

18. The apparatus of claim 17, wherein the screening module is specifically configured to:

19. The apparatus according to any one of claims 16-18, wherein the problem solving unit is specifically configured to:

20. The apparatus of claim 19, wherein the problem solving unit comprises:

the second combination module is used for combining the question stem, each option and the examination point content to obtain problem solving information corresponding to each option;

the label identification module is used for inputting each solution question information into the answer reasoning model to obtain an answer label corresponding to each solution question information;

and the answer determining module is used for determining the answer of the question according to the answer label.

21. A model training apparatus comprising:

22. The apparatus of claim 21, wherein the title acquisition unit comprises:

the question acquisition module is used for acquiring a first preset question from a preset question library and searching a second preset question related to the first preset question examination point from the preset question library;

and the question generation module is used for generating other questions according to the first preset question and the second preset question, and the other questions are used for training the evidence selection model and the answer reasoning model.

23. The apparatus of claim 22, wherein the topic generation module is specifically configured to:

24. The apparatus of claim 21, wherein the title acquisition unit comprises:

the translation module is used for acquiring preset questions from a preset question library, translating the Chinese question stem in the preset questions into question stems of other language types and then translating the question stems of other language types into Chinese question stems;

and the question combination module is used for combining the Chinese question stem obtained by translation and the options of the preset questions to generate other questions, and the other questions are used for training the model.

25. The apparatus of claim 21, wherein the title acquisition unit comprises:

the simulation question generation module is used for generating a simulation question according to an entity relationship pair and a label thereof in a preset knowledge graph, wherein the simulation question comprises a question stem and options; the knowledge graph comprises an entity relation pair between knowledge information and a knowledge answer, and the label is used for representing whether the entity relation pair is correct or not.

26. The apparatus of any one of claims 21-25, wherein the first training unit comprises:

the positive sample acquisition module is used for acquiring the content of a positive sample examination point of the question according to the preset labeling information of the question;

and/or the negative sample acquisition module is used for acquiring the negative sample examination point content of the topic from a preset knowledge point base.

27. The apparatus of claim 26, wherein the negative example acquisition module is specifically configured to:

28. The apparatus of any one of claims 21-27, wherein the first training unit comprises:

the sample label determining module is used for determining a sample label according to the evidence information of the subject;

the evidence sample generating module is used for generating evidence sample data according to the question stem, all options in the question, the evidence information and the sample label;

and the first training module is used for training a pre-built first model by using the evidence sample data to obtain the evidence selection model.

29. The apparatus of claim 21, wherein the second training unit comprises:

the word segmentation module is used for carrying out word segmentation processing on the question stem and each option of the question to obtain a keyword;

the association module is used for screening out first examination point content associated with the key words from a preset knowledge point base;

the information combination module is used for combining the question stem, all options and each first examination point content of the question to obtain a question evidence candidate set corresponding to each first examination point content;

and the test point determining module is used for inputting each question evidence candidate set into a preset evidence selection model to obtain the test point content which has an association relation with the question.

30. The apparatus of claim 21, wherein the second training unit comprises:

the reasoning label determining module is used for determining a reasoning label corresponding to each option in the question according to the answer of the question;

the reasoning sample generating module is used for generating a reasoning training sample corresponding to each option according to the question stem of the question, each option, the examination point content associated with the question and the reasoning label corresponding to each option;

and the reasoning model training module is used for training a pre-built second model by using the reasoning sample data to obtain the answer reasoning model.

31. An electronic device, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-15.

32. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-15.

33. A computer program product comprising a computer program which, when executed by a processor, carries out the steps of the method of any one of claims 1 to 15.