CN109065154A

CN109065154A - A kind of result of decision determines method, apparatus, equipment and readable storage medium storing program for executing

Info

Publication number: CN109065154A
Application number: CN201810844020.5A
Authority: CN
Inventors: 吴及; 刘喜恩; 吕萍; 贺志阳
Original assignee: Tsinghua University; iFlytek Co Ltd
Current assignee: Tsinghua University; iFlytek Co Ltd
Priority date: 2018-07-27
Filing date: 2018-07-27
Publication date: 2018-12-21
Anticipated expiration: 2038-07-27
Also published as: CN109065154B

Abstract

This application discloses a kind of result of decision to determine method, apparatus, equipment and readable storage medium storing program for executing, angle of the application from natural language understanding and reasoning, the building and maintenance of traditional complicated medical knowledge base are replaced by means of the embedded vector expression way of knowledge, avoid the building and maintenance of complicated inference rule, it saves cost and guarantees that rule conflict is not present in decision process, so that the result of decision determined is relatively reliable.

Description

Decision result determination method, device, equipment and readable storage medium

Technical Field

The present application relates to the field of natural language processing technologies, and in particular, to a method, an apparatus, a device, and a readable storage medium for determining a decision result.

Background

Although the medical industry of China is rapidly developed in recent years, the huge requirements of the public on medical treatment cannot be completely met, so that the national policy greatly supports the development of basic medical treatment at present. And for the development of primary medicine, the core lies in improving the diagnosis and treatment level of primary doctors.

The Clinical Decision Support System (CDSS) is a computer-aided application System for doctor diagnosis and treatment, which stores and uses a lot of medical knowledge and provides many-sided assistance and prompts in the diagnosis and treatment process of the doctor according to the basic information, illness state information and the like of the patient, so as to help the doctor to complete diagnosis and treatment work more reasonably and efficiently and improve the overall medical service level. Existing clinical decision support systems generally include: a knowledge base module and an inference module.

1. The knowledge base module is mainly responsible for storing and calling knowledge, and the knowledge stored in the knowledge base can be structured knowledge, such as a disease base, a check base, a medicine base and the relationship among the disease base, the check base, the medicine base and the like. The construction of the knowledge base requires manual involvement and requires continuous knowledge updating.

2. The reasoning module comprises a series of reasoning rules based on a knowledge base summarized by medical experts, and the rules are directly reflected by the experience and knowledge of the experts. The reasoning module carries out logic judgment by using a reasoning rule according to the user data to obtain a diagnosis conclusion for the reference of medical care personnel.

The inventor of the present application finds that the existing clinical decision support system has certain disadvantages, such as the construction and the updating of a knowledge base need a great deal of manual investment of medical experts, and the cost is high. Moreover, since medical knowledge is complicated and complicated, it is not easy to summarize the knowledge into reasoning rules that can be used for logical reasoning, and a large amount of expert investment is required. And are not easily represented as rules since much medical knowledge and experience is ambiguous. Also, when the rules reach a certain number, there may be a logical conflict.

Therefore, there is a need in the art for a new decision result determination scheme to avoid the drawbacks of the prior art.

Disclosure of Invention

In view of the above, the present application provides a method, an apparatus, a device, and a readable storage medium for determining a decision result, which are used to reduce the dependency on the knowledge base and achieve the decision result determination without summarizing the inference rule.

In order to achieve the above object, the following solutions are proposed:

a method of decision result determination, comprising:

acquiring at least two candidate answers corresponding to the target question in a target scene;

inquiring an embedded vector corresponding to each word in the target question and the candidate answer in a preset embedded vector library, wherein the embedded vector corresponding to the word stored in the embedded vector library represents the reasoning relation between the question and the corresponding matched answer in the target scene;

for each candidate answer, determining the matching degree of the candidate answer and the target question according to the embedded vector of the word contained in the candidate answer and the embedded vector of the word contained in the target question;

and determining a matching answer of the target question in each candidate answer according to the matching degree of each candidate answer and the target question.

Preferably, the generating process of the embedded vector library comprises:

acquiring question training data and corresponding candidate answer training data in a target scene;

training a matching degree determination model by using the participles contained in the question training data and the candidate answer training data and the labeling result of whether each candidate answer training data is the matching answer of the question training data or not, and iteratively updating the embedded vector of the participles until the matching degree of the question training data and each candidate answer training data is the highest in the matching degree of the question training data and each candidate answer training data according to the embedded vector updated by the participles;

and storing the word segmentation and the finally updated embedded vector into an embedded vector library.

Preferably, the process of determining the matching degree of the candidate answer and the target question includes:

respectively calculating the vector distance between the embedded vector of each word contained in the candidate answer and the embedded vector of each word contained in the target question;

calculating the sum of all vector distances M₁And the matching degree of the candidate answer and the target question is used as the matching degree of the candidate answer and the target question.

Preferably, the process of determining the matching degree between the candidate answer and the target question further includes:

for the M according to a first noise cancellation function₁Performing a first denoising process to obtain a first denoised sum value A₁The first noise elimination function is used for suppressing the interference of the candidate answer and the synonym or the similar word in the target question on the matching degree;

according to said M₁And said A₁And determining the matching degree of the candidate answer and the target question.

for said M according to a second noise cancellation function₁Performing a second denoising process to obtain a second denoised sum value A₂The second noise elimination function is used for suppressing the interference of the candidate answer and the word pair matching degree of the weak inference relation in the target question;

according to said M₁And said A₂And determining the matching degree of the candidate answer and the target question.

for said M according to a second noise cancellation function₁Or said A₁Performing a third denoising treatment to obtain a third denoised sum value A₃The second noise elimination function is used for suppressing the interference of the candidate answer and the word pair matching degree of the weak inference relation in the target question;

according to said M₁The above-mentioned A₁And said A₃And determining the matching degree of the candidate answer and the target question.

Preferably, the process of training the matching degree determination model includes:

determining a parameter updating module of the model by utilizing the matching degree, and iteratively updating the embedded vectors of the participles;

calculating candidates using inference module for matching degree determination modelThe updated embedded vector of each word included in the answer training data and the vector distance of the updated embedded vector of each word included in the question training data are calculated, and the sum value M of all the vector distances is calculated₁＇；

A matching degree determination module for determining model by using matching degree, and M₁"degree of matching of candidate answer training data with question training data;

and determining whether the matching degree of the question training data and the candidate answer training data serving as the matching answer is the highest in the matching degree of the question training data and the candidate answer training data by using a condition judgment module of the matching degree determination model, and if so, controlling to stop iterating and updating the embedded vector of the participle.

Preferably, the process of training the matching degree determination model further includes:

a first noise reduction module for determining a model using the degree of matching, for said M according to a first noise cancellation function₁Carrying out a first denoising process to obtain a first denoised sum A₁＇；

A second noise reduction module for determining a model using the degree of matching, for said M according to a second noise cancellation function₁' or the A₁Subjecting to a third denoising process to obtain a third denoised sum A₃＇；

A matching degree determination module for determining a model using a matching degree according to the M₁', said A₁' and said A₃Determining a degree of match of the candidate answer training data with the question training data.

A decision result determination apparatus comprising:

the data acquisition unit is used for acquiring at least two candidate answers corresponding to the target question in a target scene;

the embedded vector library query unit is used for querying an embedded vector corresponding to each word in the target question and the candidate answer in a preset embedded vector library, and the embedded vector corresponding to the word stored in the embedded vector library represents the reasoning relation between the question and the corresponding matched answer in the target scene;

a matching degree determining unit, configured to determine, for each candidate answer, a matching degree between the candidate answer and the target question according to an embedded vector of a word included in the candidate answer and an embedded vector of a word included in the target question;

and the matching answer determining unit is used for determining the matching answer of the target question in each candidate answer according to the matching degree of each candidate answer and the target question.

Optionally, the method further includes:

an embedded vector library generating unit for generating an embedded vector library, the embedded vector library generating unit comprising:

the training data acquisition unit is used for acquiring question training data and corresponding candidate answer training data in a target scene;

the model training unit is used for training a matching degree determination model by utilizing the participles contained in the question training data and the candidate answer training data and the labeling result of whether each candidate answer training data is the matching answer of the question training data or not, and iteratively updating the embedded vector of the participles until the matching degree of the question training data and each candidate answer training data is the highest in the matching degree of the question training data and each candidate answer training data determined according to the embedded vector updated by the participles;

and the data storage unit is used for storing the participles and the finally updated embedded vectors into the embedded vector library.

Optionally, the matching degree determining unit includes:

a vector distance calculation unit, configured to calculate a vector distance between an embedded vector of each word included in the candidate answer and an embedded vector of each word included in the target question;

a distance sum value calculation unit for calculating the sum value M of all vector distances₁And the matching degree of the candidate answer and the target question is used as the matching degree of the candidate answer and the target question.

Optionally, the model training unit includes:

the first model training subunit is used for determining a parameter updating module of the model by using the matching degree and iteratively updating the embedded vectors of the participles;

a second model training subunit, configured to determine a reasoning module of the model by using the matching degree, calculate an updated embedded vector of each word included in the candidate answer training data, a vector distance between the updated embedded vector of each word included in the question training data, and calculate a sum M of all vector distances₁＇；

A third model training subunit, a matching degree determining module for determining the model by using the matching degree, and a third model training subunit for training the M₁"degree of matching of candidate answer training data with question training data;

and the fourth model training subunit is used for determining a condition judgment module of the model by utilizing the matching degree, judging whether the matching degree of the question training data and the candidate answer training data serving as the matching answer is the highest in the matching degree of the question training data and the candidate answer training data, and if so, controlling to stop iterating and updating the embedded vector of the participle.

A decision result determination device comprising a memory and a processor;

the memory is used for storing programs;

the processor is configured to execute the program to implement the steps of the decision result determining method.

A readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the decision result determining method as described above.

It can be seen from the foregoing technical solutions that, in the decision result determining method provided in the embodiments of the present application, an embedded vector of a word is determined according to a reasoning relationship between a question and a corresponding matching answer in a target scene, and then an embedded vector library is formed by using the word and the corresponding embedded vector, when at least two candidate answers corresponding to the target question in the target scene are obtained, an embedded vector corresponding to each word in the target question and the candidate answers is queried from the embedded vector library, for each candidate answer, a matching degree between the candidate answer and the target question is determined according to the embedded vector of the word included in the candidate answer and the embedded vector of the word included in the target question, and a matching answer of the target question is determined in each candidate answer according to the matching degree. According to the method and the system, from the angle of natural language understanding and reasoning, the traditional construction and maintenance of a complex medical knowledge base are replaced by an embedded vector expression mode of knowledge, the construction and maintenance of a complex reasoning rule are avoided, the cost is saved, and the decision process is ensured to have no rule conflict, so that the determined decision result is more reliable.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

Fig. 1 is a flowchart of a decision result determining method disclosed in an embodiment of the present application;

FIG. 2 illustrates an inference relationship between questions and answers;

FIG. 3 illustrates a schematic diagram of a determination of an inference embedding matrix;

FIG. 4 illustrates another inference relationship between questions and answers;

FIG. 5 illustrates a graph of the suppression of synonyms or near synonyms by a first noise cancellation function;

FIG. 6 illustrates a graph of the suppression of words of weak inference relations by a second noise cancellation function;

FIG. 7 illustrates a graph of noise suppression effect of a combination of a first noise cancellation function and a second noise cancellation function;

fig. 8 is a schematic structural diagram of a decision result determining apparatus disclosed in an embodiment of the present application;

fig. 9 is a block diagram of a hardware structure of a decision result determining device disclosed in an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In many scenarios, there is a question-answer requirement, that is, after a question is given, a matching answer corresponding to the question needs to be given. Example scenarios may include: medical diagnosis scenes, case trial scenes and the like. Taking a medical diagnosis scenario as an example, the clinical decision support system solves the question-answer requirement. The input is a medical question that needs to be given a corresponding medical answer. If the medical problem is: is it likely to attack cranial nerves, especially the facial nerves? The corresponding medical answer is: tubercular meningitis. And taking a case trial scene of the law as an example, inputting the problem of the law and needing to give a corresponding result of the law. If the problem of law is: which of the following belong to the blunt object, candidates include: A. stick, B, dagger, C, iron nail. The corresponding legal results are: A. a stick.

Still taking the clinical decision support system of the medical diagnosis scenario as an example, there are many drawbacks:

the construction and the updating of the knowledge base need a great deal of manual input of medical experts, and the cost is high. Moreover, since medical knowledge is complicated and complicated, it is not easy to summarize the knowledge into reasoning rules that can be used for logical reasoning, and a large amount of expert investment is required. And are not easily represented as rules since much medical knowledge and experience is ambiguous. Also, when the rules reach a certain number, there may be a logical conflict.

The inventor of the present application provides a solution for solving the defects of the prior art creatively, and under the condition of avoiding a knowledge base and an inference module, directly from the perspective of natural language understanding and inference, the inference relation between a question and a corresponding matching answer in a target scene is implicitly expressed in the form of embedded vectors of words, namely, each word forming the question and the answer in the target scene corresponds to an embedded vector, and the embedded vector represents the inference relation between the question and the corresponding matching answer. The embedded vector expression of the words replaces the construction of a complex medical knowledge base, the construction and maintenance of complex reasoning rules are avoided, the matching degree between the candidate answers and the target question can be determined by means of the embedded vector expression of the words, and then the matching answer of the target question is determined based on the matching degree.

Next, referring to the scheme of the present application in detail, as shown in fig. 1, the method includes:

step S100, at least two candidate answers corresponding to the target question in the target scene are obtained.

Specifically, the scheme of the application can be applied to various scenes as long as a question-answer requirement exists. The target scenario may be a medical diagnostic scenario or other scenario.

In this step, at least two candidate answers corresponding to the target question in the target scene are obtained. The target question is a given question to be solved, and the user can give a plurality of candidate answers corresponding to the target question. Of course, if the user does not give a candidate answer, all possible answers to the target question may be used as candidate answers.

For medical diagnosis as an example, the target problems are: is it likely to attack cranial nerves, especially the facial nerves? Several candidate answers may be provided by the user, or all inflammations may be taken as candidate answers.

Optionally, a process of preprocessing the target question and the candidate answer may be added in this step, where the preprocessing process may include: performing word segmentation, removing special characters and punctuations, removing stop words, performing format conversion on the specified type of text, and the like. The format conversion process for the text of the specified type includes, for example: the digital information is converted into words with corresponding meanings, such as converting the age of 20 into young, converting the age of 55 into old, and the like.

Step S110, querying an embedded vector corresponding to each word in the target question and the candidate answer in a preset embedded vector library.

And the embedded vectors corresponding to the words stored in the embedded vector library represent reasoning relations between the questions and the corresponding matching answers in the target scene.

The embedded vector library is constructed in a machine learning mode, specifically, the reasoning relation between the question and the corresponding matched answer in the learning target scene is established, and the reasoning relation can be expressed in the form of the embedded vector of the word because the question and the answer are both formed by the word. Compared with the traditional expression mode of the knowledge base, the expression mode of the knowledge is more flexible.

And respectively searching an embedded vector library aiming at the target question and the candidate answer obtained in the last step to obtain an embedded vector of each word contained in the target question and an embedded vector of each word contained in each candidate answer.

Step S120, for each candidate answer, determining a matching degree between the candidate answer and the target question according to the embedded vector of the word included in the candidate answer and the embedded vector of the word included in the target question.

Specifically, by calculating a vector distance between an embedded vector of a word included in the candidate answer and an embedded vector of a word included in the target question, the vector distance can represent a matching degree between the candidate answer and the target question, and thus a matching degree between each candidate answer and the target question can be determined.

It is understood that a higher degree of matching represents a higher probability that the corresponding candidate answer is the matching answer for the target question.

Step S130, determining a matching answer to the target question in each of the candidate answers according to the matching degree between each of the candidate answers and the target question.

Specifically, the matching degree of each candidate answer with the target question is determined, so in this step, the matching answer of the target question is determined among the candidate answers according to the matching degree.

It is understood that the candidate answer as the matching answer to the target question should have a higher matching degree with the target question, for example, one or a set number (e.g., topN) of candidate answers with the highest matching degree is selected as the matching answer to the target question.

The decision result determining method provided by the embodiment of the application replaces the traditional construction and maintenance of a complex medical knowledge base with the embedded vector expression mode of knowledge from the perspective of natural language understanding and reasoning, avoids the construction and maintenance of a complex reasoning rule, saves the cost, ensures that no rule conflict exists in the decision process, and ensures that the determined decision result is more reliable.

In one embodiment of the present application, the above-described generation process of the embedded vector library is introduced.

The embodiment of the application learns knowledge and reasoning relation between question answering and matched answers in a target scene by means of the neural network model, obtains vector expressions of words forming the question and the answers through the trained neural network model, and uses the vector expressions as embedded vectors of the words.

The problems exemplified in table 1 below are exemplified:

TABLE 1

As exemplified in table 1 above, the matching answer to the question is a: hydrogen bonding. This embodiment learns such knowledge and reasoning relationships from the question to the matching answer from the vocabulary granularity, as shown in fig. 2. And dividing the questions according to the granularity of the vocabularies, and learning the knowledge and reasoning relationship between each word and the matched answers.

In the specific training process, question training data and corresponding candidate answer training data in a target scene need to be obtained first. The above table 1 is an example of a piece of question training data and corresponding candidate answer training data.

And further, performing word segmentation on the question training data and the candidate answer training data, and training a matching degree determination model by using the word segmentation included in the question training data and the candidate answer training data and the labeling result of whether each candidate answer training data is the matching answer of the question training data.

And the training process of the matching degree determination model is a process of iteratively updating the embedded vector of the participle, and the embedded vector of the participle is continuously iteratively updated until the matching degree of the question training data and each candidate answer training data is determined according to the embedded vector after the participle is updated, the matching degree of the question training data and the candidate answer serving as the matching answer is the highest, so that the knowledge and reasoning relation between the question and the matching answer is accurately embedded into the vector expression form of the word.

The present embodiment may store the word segmentation and the finally updated embedded vector thereof into the embedded vector library to obtain the generated embedded vector library.

Further, the process of training the matching degree determination model is described in an expansion mode.

The matching degree determination model can comprise a parameter updating module, an inference module, a matching degree determination module and a condition judgment module. Based on this, the training process may include:

and S1, determining a parameter updating module of the model by using the matching degree, and iteratively updating the embedding vector of the participle.

Specifically, each participle may set an initial embedding vector, which may be the same value, or may be randomly initialized or otherwise determined. When the following condition judgment module judges that the word is not embedded, the embedded vector of the word segmentation can be updated through the parameter updating module.

S2, using the inference module of the matching degree determination model to calculate the updated embedded vector of each word contained in the candidate answer training data and the vector distance of the updated embedded vector of each word contained in the question training data, and calculating the sum M of all the vector distances₁＇。

Specifically, for candidate answer training data and question training data, calculating and calculating the vector distance between the updated embedded vector of each word included in the candidate answer training data and the updated embedded vector of each word included in the question training data, wherein each vector distance forms a reasoning embedded matrix M.

Wherein M is equal to R^x′×y′And then:

wherein,an embedded vector representing the ith word w (o', i) in the candidate answer,the embedded vector represents the jth word w (q ', j) in the question training data, cosine () represents the calculated cosine distance, x ' represents the number of words contained in the question training data, and y ' represents the number of words contained in the candidate answer.

Summing the inference embedding matrix M to obtain a sum value M₁＇。

As shown in fig. 3, a schematic diagram illustrating a determination manner of the inference embedding matrix is illustrated.

Fig. 3 merely illustrates the process of computing the inference embedding matrix for a question and one candidate answer, wherein the question contains four words, w (q,1), w (q,2), w (q,3), and w (q, 4). The candidate answer includes two words, w (o,1) and w (o, 2).

And aiming at each participle contained in the question and the candidate answer, respectively searching a corresponding embedded vector in an embedded vector library by using the lolup. Further, a vector distance m between each two is calculated, which may be the cosine distance between two vectors. Taking w (o,1) and w (q,1) as examples, the calculated vector distance is denoted as m (1, 1). And finally, forming an inference embedding matrix M by all vector distances.

S3, matching degree determination module using matching degree determination model, and M₁"degree of matching of candidate answer training data with question training data.

In particular, inference embedding matrix M has given the degree of support of candidate answers to the question at word granularity, so summing the results M for inference embedding matrix M₁"can be used as a candidate answer as a matching degree of a matching answer to a question.

And S4, determining whether the matching degree of the question training data and the candidate answer training data serving as the matching answer is the highest in the matching degrees of the question training data and the candidate answer training data by using a condition judgment module of the matching degree determination model, and if so, controlling to stop iterating and updating the embedded vector of the participle.

Specifically, the condition determining module is used as a determining module for determining whether the matching degree determining model reaches a training termination condition, where the training termination condition may be understood as that, of the matching degrees of the question training data and each candidate answer, the matching degree of the question training data and the candidate answer training data labeled as the matching answer is the highest. That is, the model prediction result is consistent with the real annotation result.

And when the training termination condition is judged to be reached, controlling to stop iteratively updating the embedded vectors of the participles, wherein the embedded vectors of the participles at the moment are used as final embedded vectors.

Further, when it is determined that the training termination condition is not met, training is required, so that the parameter updating module is required to continuously update the embedded vectors of the participles in an iterative manner.

On the basis of the training process of the matching degree determination model in the above example, the inventor finds that partial noise interference exists in the matching degree measurement mode, and the noise can be mainly divided into two types, namely, noise type I and noise type II.

Noise type I:

the noise type I mainly refers to the interference of the candidate answer and the synonym or near-synonym in the target question on the matching degree. The synonyms may be homonyms or non-homonyms but have the same meaning. It can be understood that, when the candidate answer of the non-matching answer includes the synonymous word or the similar word in the target question, the cosine distance between the embedded vectors of the two words is calculated as above, and thus the matching degree between the candidate answer and the target question is increased by mistake. Since the present application is a certain logical inference relationship in a measurement target scenario, taking a medical diagnosis scenario as an example, the measurement is a certain logical inference relationship in medicine, not a synonymy or a near-synonymy relationship, it is necessary to suppress the type I noise.

TABLE 2

Table 2 above illustrates a noise type I interference example. The non-matching answers B and D contain the same word "sifting" as the question, which results in a higher matching score for the two answers, while the matching answer is E.

Noise type II:

the noise type II mainly refers to the interference of the candidate answer and the word pair matching degree of the weak inference relation in the target question. It will be appreciated that not every word in the target question contributes to reasoning about matching answers. As illustrated in fig. 4, the target question word is divided into: "dimension", "DNA double strand", "gap", "base pairing", "chemical bond" and "is". The matching answer to the target question is: "Hydrogen bonds". And if the target question is divided according to a set contribution threshold, defining that the contribution is lower than the contribution threshold as low contribution and defining that the contribution is higher than the contribution threshold as high contribution. In fig. 4, the words with low degree of contribution are illustrated by dashed lines, and the words with high degree of contribution are illustrated by solid lines. It can be seen that the terms "dimension", "between", "is" and "is" in the target question have a very low contribution to the inference of matching answers, while "DNA duplex", "base pairing" and "chemical bond" have a very high contribution to the inference of matching answers.

In this embodiment, words with low contribution to reasoning to obtain a matching answer in the candidate answers may be defined as words with weak reasoning relationship between the candidate answers and the target question.

In order to avoid the interference of the type I and type II noise described above, it is necessary to suppress and remove the two types of noise in the process of training the matching degree determination model.

First, in order to remove type I noise, the present application introduces a first noise reduction module in the matching degree determination model. The process of training the matching degree determination model may further include, based on the foregoing descriptions of S1-S4:

s5, determining a first noise reduction module of the model by using the matching degree, and carrying out noise elimination on the M according to a first noise elimination function₁Carrying out a first denoising process to obtain a first denoised sum A₁＇。

The first noise elimination function is used for suppressing the interference of synonyms or synonyms in the candidate answer training data and the question training data on the matching degree.

Optionally, this embodiment illustrates an expression of the first noise cancellation function, which is as follows:

A₁′＝sigmoid(k₁(M₁′+threshold₁))·sigmoid(-k₁(M₁′-threshold₁))

wherein k is₁Is the amplification factor, threshold₁Is a filtering threshold, and the size can be preset, such as 0.7.

Through the first noise elimination function, word pairs with the same or similar semantemes in the question and the candidate answers can be filtered out, and therefore the interference effect of pseudo inference of the suppressor is suppressed.

Referring to fig. 5, the suppression of synonyms or near synonyms by the first noise cancellation function is illustrated. In FIG. 5, the horizontal axis represents M₁' longitudinal and longitudinal indicate A₁". It can be seen that the first noise cancellation function is for M caused by synonyms or synonyms₁Excessive cases have good inhibitory effect.

S6, a matching degree determining module for determining a model by using the matching degree according to the M₁' and said A₁' doAnd determining the matching degree of the candidate answer training data and the question training data.

Specifically, the embodiment may be M₁' and A₁Respectively setting weights, and weighting and adding the weights, wherein the result is the matching degree of the candidate answer training data and the question training data. It will be appreciated that M₁' and A₁' the weights may be different, or even M may be weighted₁' set the weight of A to 0₁' is set to 1.

In addition to this, M may be₁' and A₁And multiplying the result of the multiplication to obtain the matching degree of the candidate answer training data and the question training data.

According to the scheme provided by the embodiment, the I-type noise of the matching degree determination model can be removed, so that the result of model prediction is more accurate.

It should be noted that the number of the first noise reduction modules may be multiple, that is, the I-type noise may be removed multiple times through multi-stage setting, so that the I-type noise removal effect is better.

For the questions and the candidate answers illustrated in table 2 above, the matching degree between the candidate answers and the questions is calculated by using the word-embedded vector library obtained by the model without removing the type I noise and the model without removing the type I noise, and the result is as follows:

TABLE 3

As can be seen from table 3 above:

when the matching degree of the candidate answer and the question is calculated by using the word embedding vector library obtained by the model without removing the type I noise, the matching degree of the candidate answer B is the highest, so that B is selected wrongly. And when the matching degree of the candidate answer and the question is calculated by embedding the words obtained by the model for removing the I-type noise into the vector library, the influence of screening the interfering words is filtered, so that the matching degree score of the matching answer E is the highest, and the correct matching answer is obtained.

Further, in order to remove type I noise, the present application introduces a second noise reduction module in the matching degree determination model. The process of training the matching degree determination model may further include, based on the foregoing descriptions of S1-S4:

s7, determining a second noise reduction module of the model by using the matching degree, and carrying out comparison on the M according to a second noise elimination function₁"second denoise to obtain a first denoised sum A2'.

And the second noise elimination function is used for suppressing the interference of the candidate answer and the word pair matching degree of the weak inference relation in the target question.

Optionally, this embodiment illustrates an expression manner of the second noise cancellation function, which is as follows:

A₂′＝sigmoid(k₂(M₁′-threshold₂))·sigmoid(-k₂(M₁′+threshold₂))

wherein k is₂Is the amplification factor, threshold₂Is a filtering threshold, and the size can be preset, such as 0.1. General, threshold₂Less than threshold₁。

Through the second noise elimination function, word pairs with weak inference relations in the questions and the candidate answers can be filtered, and therefore the interference effect of pseudo inference of the suppressor is suppressed.

See fig. 6, which illustrates the suppression of words of weak inference relations by the second noise cancellation function. In FIG. 6, the horizontal axis represents M₁' longitudinal and longitudinal indicate A₂". It follows that the second noise cancellation function leads to M for words of weak inference relations₁Too small a number has a very good inhibitory effect.

S8, a matching degree determining module for determining a model by using the matching degree according to the M₁' and said A₂Determining a degree of match of the candidate answer training data with the question training data.

Specifically, the embodiment may be M₁' and A₂Respectively setting weights, and weighting and adding the weights, wherein the result is the matching degree of the candidate answer training data and the question training data. It will be appreciated that M₁' and A₂' the weights may be different, or even M may be weighted₁' set the weight of A to 0₂' is set to 1.

In addition to this, M may be₁' and A₂And multiplying the result of the multiplication to obtain the matching degree of the candidate answer training data and the question training data.

According to the scheme provided by the embodiment, II-type noise can be removed by the matching degree determination model, so that the result of model prediction is more accurate.

It should be noted that the number of the second noise reduction modules may be multiple, that is, the type II noise may be removed multiple times through multi-stage setting, so that the type II noise removal effect is better.

Still further, type I and type II noise are removed simultaneously in this embodiment, and the first noise reduction module and the second noise reduction module are introduced simultaneously in the matching degree determination model according to the present application. The process of training the matching degree determination model may further include, based on the foregoing descriptions of S1-S4:

s9, determining a first noise reduction module of the model by using the matching degree, and carrying out noise elimination on the M according to a first noise elimination function₁Carrying out a first denoising process to obtain a first denoised sum A₁＇。

In this embodiment, S9 corresponds to S5 in the previous embodiments, and reference is made to the above description for details.

S10, determining a second noise reduction module of the model by using the matching degree, and carrying out comparison on the M according to a second noise elimination function₁' or the A₁Subjecting to a third denoising process to obtain a third denoiseThree denoised sum values A₃＇。

Alternatively, if the second noise cancellation function pair M₁' performing third denoising, the expression may be:

A₃′＝sigmoid(k₃(M₁′-threshold₃))·sigmoid(-k₃(M₁′+threshold₃))

if the second noise cancellation function pair A₁' performing third denoising, the expression may be:

A₃′＝sigmoid(k₃(A₁′-threshold₃))·sigmoid(-k₃(A₁′+threshold₃))

wherein k is₃Is the amplification factor, threshold₃Is a filtering threshold, and the size can be preset.

It will be appreciated that since A₁Is the first noise reduction function pair M₁After performing type I denoising, the second noise cancellation function is thus applied to A₁After the third denoising process, the resulting third denoised sum A₃' type I and type II noise have been removed synthetically, and the two types of noise suppression effects are shown in fig. 7.

In FIG. 7, the horizontal axis represents M₁' longitudinal and longitudinal indicate A₃". It follows that M caused by the first and second noise cancellation functions for synonyms or synonyms₁M caused by words of' too big case, and weak inference relation₁Too small a number has a very good inhibitory effect.

S11, a matching degree determining module for determining a model by using the matching degree according to the M₁', said A₁' and said A₃Determining a degree of match of the candidate answer training data with the question training data.

Specifically, the present embodiment mayIs M₁＇、A₁' and A₃Respectively setting weights, and weighting and adding the three, wherein the result is the matching degree of the candidate answer training data and the question training data. It will be appreciated that M₁＇、A₁' and A₃' the weights may be different, or even M may be weighted₁' and A₁'is set to 0, and A3' is set to 1.

In addition to this, M may be₁＇、A₁' and A₃The three are multiplied, and the multiplication result is used as the matching degree of the candidate answer training data and the question training data.

According to the scheme provided by the embodiment, the I-type noise and the II-type noise can be removed by the matching degree determination model at the same time, so that the prediction result of the model is more accurate.

It should be noted that the number of the first noise reduction module and the number of the second noise reduction module may be multiple, that is, the type I noise and the type II noise may be removed multiple times through multi-stage setting, so that the two types of noise removal effects are better.

An exemplary case is given next:

the problems are as follows: among the diseases described below, the cranial nerve, especially the facial nerve, is vulnerable to (). The candidate answers include: tuberculous meningitis, B-toxic encephalopathy, C-viral encephalitis, D-cryptococcal meningitis, and E-suppurative meningitis.

The correct matching answer is known to be a.

For this case, the matching degree between the candidate answer and the question is calculated by using the word embedding vector library obtained by the models without removing the type I noise, simply removing the type I noise and simultaneously removing the type I noise and the type II noise, and the result is as follows in table 4:

candidate answers	A	B	C	D	E
						Without removing noise	0.1514	0.1090	0.1814	0.2985	0.2597
Removing type I noise	0.2011	0.1604	0.2003	0.2772	0.1610
						Removing type I and II noise	0.2599	0.1430	0.1939	0.2204	0.1828

TABLE 4

As can be seen from Table 4 above:

when the matching degree of the candidate answer and the question is calculated by using the word embedding vector library obtained by the model without removing the noise and the model with removing the type I noise, the matching degree of the candidate answer D is the highest, so that D is selected wrongly. And when the matching degree of the candidate answer and the question is calculated by using the word embedding vector library obtained by the model for removing the type I and type II noises, the influence of the two types of noises is filtered, so that the matching degree score of the matching answer A is the highest, and the correct matching answer is obtained.

In summary, the embodiment of the present application provides several different training processes for a matching degree determination model, including a most basic matching degree determination model determined in S1-S4, and based on the most basic matching degree determination model, simply removing type I noise in S1-S4 in combination with S5-S6, based on the most basic matching degree determination model, simply removing type II noise in S1-S4 in combination with S7-S8, based on the most basic matching degree determination model, and simultaneously removing type I and type II noise in S1-S4 in combination with S9-S11 in combination with the most basic matching degree determination model.

The obtained word embedding vectors are not completely the same based on the training modes of the different matching degree determination models, for example, the word embedding vectors obtained based on the training processes of S1-S4 can represent the reasoning relation between the question and the matching answer, but include noise interference. Further, if type I noise is suppressed during the training process, the inference relationship between the question and the matching answer characterized by the embedded vector of the obtained word also has removed the type I noise, and similarly, if type II noise is suppressed during the training process, the inference relationship between the question and the matching answer characterized by the embedded vector of the obtained word also has removed the type II noise.

Based on the training process of the matching degree determination model described in the above embodiment, another embodiment of the present application further describes the process of determining the matching degree between the candidate answer and the target question in the foregoing step S120.

In an optional manner, the matching degree determining process may include:

1) and respectively calculating the vector distance between the embedded vector of each word contained in the candidate answer and the embedded vector of each word contained in the target question.

The vector distances form an inference embedding matrix M.

Wherein M is equal to R^x×yAnd then:

wherein,an embedded vector representing the ith word w (o, i) in the candidate answer,an embedded vector representing the jth word w (q, j) in the target question, cosine () representing the distance of cosine, x being the number of words contained in the target question, y representing the number of words contained in the candidate answer.

It should be noted that the embedded vector library used in this embodiment may be obtained by determining a model training process based on any one of the matching degrees. That is, the embedded vectors embedded into the words stored in the vector library may be capable of characterizing the inference relationship between the question and the matching answer, but contain noise interference; or, the reasoning relation between the question and the matching answer can be represented, and the type I noise interference is removed; it may be that the inference relationship between the question and the matching answer has been characterized and the type II noise interference has been removed, or it may be that the inference relationship between the question and the matching answer has been characterized and the type I and type II noise interference has been removed.

2) Calculating the sum of all vector distances M₁And the matching degree of the candidate answer and the target question is used as the matching degree of the candidate answer and the target question.

Summing the inference embedding matrix M to obtain a sum value M₁。

Further optionally, considering that the word embedding vector used in determining the inference embedding matrix M may include noise interference, or even if the word embedding vector used has removed the noise interference, the removal may not be thorough, so that the following scheme of this embodiment may further add a process of removing type I noise and type II noise on the basis of the above 1) -2), as detailed below:

firstly, adding a process for removing type I noise on the basis of the above, that is, determining the degree of matching on the basis of 1) -2), the method may further include:

3) for the M according to a first noise cancellation function₁Performing a first denoising process to obtain a first denoised sum value A₁。

Wherein the first noise elimination function is used for suppressing the interference of the candidate answer and the synonym or the synonym in the target question on the matching degree.

The first noise cancellation function is expressed in a similar manner to the first noise cancellation function introduced in the model training process described above, as follows:

A₁＝sigmoid(k₁(M₁+threshold₁))·sigmoid(-k₁(M₁-threshold₁))

4) According to said M₁And said A₁And determining the matching degree of the candidate answer and the target question.

Specifically, the embodiment may be M₁And A₁And respectively setting weights, and carrying out weighted addition on the weights and the weights to obtain a result as the matching degree of the candidate answer and the target question. It will be appreciated that M₁And A₁May be different, even M may be used₁Is set to 0, A is set₁Is set to 1.

In addition to this, M may be₁And A₁And multiplying, wherein the multiplication result is used as the matching degree of the candidate answer and the target question.

Further, adding a process for removing type II noise on the basis of the foregoing, that is, determining the degree of matching on the basis of 1) -2) above, may further include:

5) for said M according to a second noise cancellation function₁Performing a second denoising process to obtain a second denoised sum value A₂。

And the second noise elimination function is used for inhibiting the interference of the candidate answer and the word pair matching degree of the weak inference relation in the target question.

The second noise cancellation function is expressed in a similar manner to the second noise cancellation function introduced in the model training process described above, as follows:

A₂＝sigmoid(k₂(M₁-threshold₂))·sigmoid(-k₂(M₁+threshold₂))

6) According to said M₁And said A₂And determining the matching degree of the candidate answer and the target question.

Specifically, the embodiment may be M₁And A₂And respectively setting weights, and carrying out weighted addition on the weights and the weights to obtain a result as the matching degree of the candidate answer and the target question. It will be appreciated that M₁And A₂May be different, even M may be used₁Is set to 0, A is set₂Is set to 1.

In addition to this, M may be₁And A₂Multiplying, the result of the multiplication being a candidate answerDegree of match to the target problem.

Still further, adding a process for removing type I and type II noise simultaneously on the basis of the above, that is, determining the degree of matching on the basis of the above 1) -2), may further include:

7) for the M according to a first noise cancellation function₁Performing a first denoising process to obtain a first denoised sum value A₁。

Step 7) in this embodiment corresponds to step 3) in the foregoing embodiment, and the foregoing description is referred to in detail.

8) For said M according to a second noise cancellation function₁Or said A₁Performing a third denoising treatment to obtain a third denoised sum value A₃。

Alternatively, if the second noise cancellation function pair M₁If the third denoising process is performed, the expression manner may be:

A₃＝sigmoid(k₃(M₁-threshold₃))·sigmoid(-k₃(M₁+threshold₃))

if the second noise cancellation function pair A₁If the third denoising process is performed, the expression manner may be:

A₃＝sigmoid(k₃(A₁-threshold₃))·sigmoid(-k₃(A₁+threshold₃))

9) According to said M₁The above-mentioned A₁And said A₃And determining the matching degree of the candidate answer and the target question.

Specifically, the embodiment may be M₁、A₁And A₃Setting weights respectively, and adding the weights to obtain a candidate answerMatching degree of case and target problem. It will be appreciated that M₁、A₁And A₃May be different, even M may be used₁And A₁Is set to 0 and the weight of a3 is set to 1.

In addition to this, M may be₁、A₁And A₃Multiplying the three, and taking the multiplication result as the matching degree of the candidate answer and the target question.

The decision result determining device provided in the embodiment of the present application is described below, and the decision result determining device described below and the decision result determining method described above may be referred to correspondingly.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a decision result determining apparatus disclosed in the embodiment of the present application. As shown in fig. 8, the apparatus may include:

the data acquisition unit 11 is configured to acquire at least two candidate answers corresponding to the target question in a target scene;

an embedded vector library query unit 12, configured to query, in a preset embedded vector library, an embedded vector corresponding to each word in the target question and the candidate answer, where the embedded vector corresponding to the word stored in the embedded vector library represents a reasoning relationship between the question and the corresponding matching answer in the target scene;

a matching degree determining unit 13, configured to determine, for each candidate answer, a matching degree between the candidate answer and the target question according to an embedded vector of a word included in the candidate answer and an embedded vector of a word included in the target question;

a matching answer determining unit 14, configured to determine a matching answer to the target question in each of the candidate answers according to a matching degree between each of the candidate answers and the target question.

Optionally, the apparatus of the present application may further include:

Optionally, the matching degree determining unit may include:

Optionally, the embodiment of the present application further illustrates several extension structures of the matching degree determining unit, and on the basis, there are three extension structures respectively, as follows:

in the first extended structure, the matching degree determination unit may further include:

a first noise reduction unit for reducing noise of the M according to a first noise elimination function₁Performing a first denoising process to obtain a first denoised sum value A₁Said first noise cancellation function being for suppressingMaking interference of synonyms or near-synonyms in the candidate answers and the target questions on matching degrees;

a first denoising weighting unit for weighting the M₁And said A₁And determining the matching degree of the candidate answer and the target question.

In the second extension structure, the matching degree determination unit may further include:

a second noise reduction unit for removing noise from the M signal according to a second noise cancellation function₁Performing a second denoising process to obtain a second denoised sum value A₂The second noise elimination function is used for suppressing the interference of the candidate answer and the word pair matching degree of the weak inference relation in the target question;

a second denoising weighting unit for weighting the M₁And said A₂And determining the matching degree of the candidate answer and the target question.

In a third extended configuration, the matching degree determination unit may further include:

a third noise reduction unit for removing noise from the M signal according to a second noise cancellation function₁Or said A₁Performing a third denoising treatment to obtain a third denoised sum value A₃The second noise elimination function is used for suppressing the interference of the candidate answer and the word pair matching degree of the weak inference relation in the target question;

a third denoising weighting unit for calculating M₁The above-mentioned A₁And said A₃And determining the matching degree of the candidate answer and the target question.

Optionally, the model training unit may include:

a second model training subunit for utilizing the pieceThe reasoning module of the matching degree determination model calculates the updated embedded vector of each word contained in the candidate answer training data and the vector distance between the updated embedded vector of each word contained in the question training data, and calculates the sum value M of all the vector distances₁＇；

Optionally, the embodiment of the present application further illustrates extension structures of several model training units, and on the basis, there are three extension structures respectively, as follows:

in the first extension structure, the model training unit may further include:

a fifth model training subunit, configured to determine a first noise reduction module of the model by using the matching degree, and perform noise cancellation on the M according to a first noise cancellation function₁Carrying out a first denoising process to obtain a first denoised sum A₁＇；

A sixth model training subunit, configured to determine a matching degree determination module of the model according to the matching degree, according to the M₁' and said A₁Determining a degree of match of the candidate answer training data with the question training data.

In a second extension structure, the model training unit may further include:

a seventh model training subunit, configured to determine a second noise reduction module of the model by using the matching degree, and perform noise cancellation on the M according to a second noise cancellation function₁"second denoise to obtain a first denoised sum A2";

an eighth model training subunit, configured to determine a matching degree determination module of the model according to the matching degree, according to the M₁' and said A₂Determining a degree of match of the candidate answer training data with the question training data.

In a third extension, the model training unit may further include:

a ninth model training subunit, configured to determine a first noise reduction module of the model by using the matching degree, and perform a first noise cancellation on the M₁Carrying out a first denoising process to obtain a first denoised sum A₁＇；

A tenth model training subunit, configured to determine a second noise reduction module of the model by using the matching degree, and perform noise cancellation on the M according to a second noise cancellation function₁' or the A₁Subjecting to a third denoising process to obtain a third denoised sum A₃＇；

An eleventh model training subunit, configured to determine a matching degree determination module of the model using the matching degree, according to the M₁', said A₁' and said A₃Determining a degree of match of the candidate answer training data with the question training data.

The decision result determining device provided by the embodiment of the application can be applied to decision result determining equipment such as a PC terminal, a cloud platform, a server cluster and the like. Alternatively, fig. 9 shows a block diagram of a hardware structure of the decision result determining device, and referring to fig. 9, the hardware structure of the decision result determining device may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

in the embodiment of the application, the number of the processor 1, the communication interface 2, the memory 3 and the communication bus 4 is at least one, and the processor 1, the communication interface 2 and the memory 3 complete mutual communication through the communication bus 4;

the processor 1 may be a central processing unit CPU, or an application specific Integrated circuit asic, or one or more Integrated circuits configured to implement embodiments of the present invention, etc.;

the memory 3 may include a high-speed RAM memory, and may further include a non-volatile memory (non-volatile memory) or the like, such as at least one disk memory;

wherein the memory stores a program and the processor can call the program stored in the memory, the program for:

Alternatively, the detailed function and the extended function of the program may be as described above.

Embodiments of the present application further provide a readable storage medium, where a program suitable for being executed by a processor may be stored, where the program is configured to:

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining a decision result, comprising:

2. The method of claim 1, wherein the generating of the embedded vector library comprises:

3. The method of claim 1, wherein determining the degree of matching between the candidate answer and the target question comprises:

4. The method of claim 3, wherein the process of determining the degree of matching of the candidate answer to the target question further comprises:

according to the first noise eliminationFunction of said M₁Performing a first denoising process to obtain a first denoised sum value A₁The first noise elimination function is used for suppressing the interference of the candidate answer and the synonym or the similar word in the target question on the matching degree;

5. The method of claim 3, wherein the process of determining the degree of matching of the candidate answer to the target question further comprises:

6. The method of claim 4, wherein the process of determining the degree of matching of the candidate answer to the target question further comprises:

7. The method of claim 2, wherein the process of training the goodness-of-fit determination model comprises:

calculating the updated embedded vector of each word contained in the candidate answer training data and the vector distance between the updated embedded vector of each word contained in the question training data by using the reasoning module of the matching degree determination model, and calculating the sum value M of all the vector distances₁＇；

8. The method of claim 7, wherein the process of training the goodness-of-fit determination model further comprises:

9. A decision result determining apparatus, comprising:

10. The apparatus of claim 9, further comprising:

11. The apparatus according to claim 9, wherein the matching degree determination unit includes:

12. The apparatus of claim 10, wherein the model training unit comprises:

13. A decision result determination device comprising a memory and a processor;

the memory is used for storing programs;

the processor, configured to execute the program, implementing the steps of the decision result determination method according to any one of claims 1-8.

14. A readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the decision result determination method according to any one of claims 1-8.