WO2023088278A1 - 用于验证表述的真实性的方法、设备、装置和介质 - Google Patents

用于验证表述的真实性的方法、设备、装置和介质 Download PDF

Info

Publication number
WO2023088278A1
WO2023088278A1 PCT/CN2022/132139 CN2022132139W WO2023088278A1 WO 2023088278 A1 WO2023088278 A1 WO 2023088278A1 CN 2022132139 W CN2022132139 W CN 2022132139W WO 2023088278 A1 WO2023088278 A1 WO 2023088278A1
Authority
WO
WIPO (PCT)
Prior art keywords
phrase
representation
evidence
phrases
expression
Prior art date
Application number
PCT/CN2022/132139
Other languages
English (en)
French (fr)
Inventor
张欣勃
陈江捷
包乔奔
孙长志
陈家泽
周浩
肖仰华
李磊
Original Assignee
北京有竹居网络技术有限公司
复旦大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京有竹居网络技术有限公司, 复旦大学 filed Critical 北京有竹居网络技术有限公司
Publication of WO2023088278A1 publication Critical patent/WO2023088278A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/253—Grammatical analysis; Style critique
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • Exemplary implementations of the present disclosure relate generally to the field of computers, and in particular to methods, apparatus, apparatus, and computer-readable storage media for verifying the authenticity of representations.
  • a scheme for verifying the authenticity of an expression is provided.
  • a method for verifying the authenticity of a statement is provided.
  • training data including representations, evidence sets, and labels are acquired, the representations represent content to be verified, the evidence set includes at least one evidence supporting the authenticity of the verification representation, and the labels represent verification of the representation based on the evidence set Authenticity results.
  • the expression is divided into phrases.
  • a phrase verification model is trained based on the training data and the plurality of phrases such that the phrase verification model determines a plurality of phrase authenticities for the plurality of phrases respectively based on the evidence set.
  • the representation verification model is trained based on the training data and a plurality of phrases, such that the representation verification model determines representation truthfulness of the representation based on the evidence set, wherein the plurality of phrase truths provide an explanation for the representation truthfulness.
  • an electronic device comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, The instructions, when executed by at least one processing unit, cause the device to perform actions.
  • the action includes: obtaining training data including representations, evidence sets, and labels, the representations representing content to be verified, the evidence set including at least one evidence supporting the verification of the representation's authenticity, and the labels representing the verification of the representation's authenticity based on the evidence set
  • the expression is divided into multiple phrases; the phrase verification model is trained based on the training data and the multiple phrases, so that the phrase verification model determines the multiple phrases of the multiple phrases based on the evidence set authenticity; and training the representation verification model based on the training data and the plurality of phrases, such that the representation verification model determines the representation truthfulness of the representation based on the evidence set, wherein the plurality of phrase veracities provide an explanation for the representation truthfulness.
  • a method for verifying the authenticity of a statement is provided.
  • an expression and a set of evidence associated with the expression are obtained, the expression represents what is to be verified, and the set of evidence includes at least one evidence supporting the verification of the authenticity of the expression; based on the syntax analysis of the expression, the expression Divided into multiple phrases; use the phrase verification model to determine the multiple phrase authenticity of multiple phrases based on the evidence set; and use the representation verification model to determine the statement authenticity of the representation based on the evidence set and multiple phrases, where The truth of a phrase provides an explanation for the truth of an expression.
  • an electronic device comprising: at least one processing unit; and at least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, The instructions, when executed by at least one processing unit, cause the device to perform actions.
  • the actions include: obtaining an expression and a set of evidence associated with the expression, the expression representing what is to be verified, and the set of evidence including at least one evidence supporting the verification of the authenticity of the expression; based on a syntactic analysis of the expression, dividing the expression into is a plurality of phrases; using the phrase verification model, based on the evidence set to determine the plurality of phrase authenticity of the plurality of phrases; Phrase truth provides an explanation for the expression truth.
  • an apparatus for verifying the authenticity of a statement includes: an acquisition module configured to acquire training data including a representation, an evidence set, and a label, the representation representing content to be verified, the evidence set including at least one evidence supporting the authenticity of the verification representation, and the label representing evidence-based Set to verify the result of the authenticity of the expression; the partition module is configured to divide the expression into multiple phrases based on the parsing of the expression; the phrase verification module is configured to train the phrase verification model based on the training data and the plurality of phrases , so that the phrase verification model respectively determines a plurality of phrase authenticity of the plurality of phrases based on the evidence set; and a representation verification module configured to train the representation verification model based on the training data and the plurality of phrases, such that the representation verification model is based on the evidence sets to determine the representational truth of an utterance, where multiple phrase truths provide explanations for the representational truthfulness.
  • an apparatus for verifying the authenticity of a representation includes: an acquisition module configured to acquire a statement and a set of evidence associated with the statement, the statement represents content to be verified, and the set of evidence includes at least one piece of evidence supporting the authenticity of the statement; a partitioning module configured for dividing an utterance into a plurality of phrases based on a grammatical analysis of the utterance; a phrase verification module configured to utilize a phrase verification model to determine a plurality of phrase truths for the plurality of phrases, respectively, based on a set of evidence; and a representation verification module, Configured to utilize a representation verification model to determine a representational truth of a representation based on a set of evidence and a plurality of phrases, wherein the plurality of phrase truths provides an explanation for the representational truthfulness.
  • a computer readable storage medium is provided.
  • a computer program is stored on the medium, and when the program is executed by the processor, the method in the first aspect is realized.
  • a computer readable storage medium is provided.
  • a computer program is stored on the medium, and when the program is executed by the processor, the method of the third aspect is realized.
  • FIG. 1 shows a block diagram of an example environment in which implementations of the present disclosure can be implemented
  • Figure 2 shows a block diagram of a verification model for verifying the authenticity of representations, according to some implementations of the present disclosure
  • Figure 3 shows a block diagram for determining local premises according to some implementations of the present disclosure
  • FIG. 4 shows a block diagram for training a machine reading comprehension (Machine Reading Comprehension, MRC) model according to some implementations of the present disclosure
  • FIG. 5 shows a block diagram for training a phrase verification model and an utterance verification model, according to some implementations of the present disclosure
  • FIG. 6 illustrates a block diagram for performing inference based on a phrase verification model and an expression verification model, according to some implementations of the present disclosure
  • FIG. 7A shows a block diagram for determining authenticity of a statement, according to some implementations of the present disclosure
  • FIG. 7B shows a block diagram for determining authenticity of a statement, according to some implementations of the present disclosure
  • FIG. 8 shows a flowchart of a method for verifying the authenticity of an expression according to some implementations of the present disclosure
  • FIG. 9 shows a flowchart of a method for verifying the authenticity of an expression according to some implementations of the present disclosure.
  • Figure 10A shows a block diagram of an apparatus for verifying the authenticity of a representation according to some implementations of the present disclosure
  • Figure 10B shows a block diagram of an apparatus for verifying the authenticity of a representation, according to some implementations of the present disclosure.
  • Figure 11 shows a block diagram of a device capable of implementing various implementations of the present disclosure.
  • model can learn the relationship between the corresponding input and output from the training data, so that the corresponding output can be generated for the given input after the training is completed.
  • the generation of the model may be based on machine learning techniques.
  • Deep learning is a machine learning algorithm that uses multiple layers of processing units to process input and provide corresponding output.
  • a neural network model is an example of a deep learning based model.
  • a “model” may also be referred to herein as a "machine learning model,” “learning model,” “machine learning network,” or “learning network,” and these terms are used interchangeably herein.
  • a "neural network” is a machine learning network based on deep learning.
  • a neural network is capable of processing input and providing a corresponding output, and typically includes an input layer and an output layer and one or more hidden layers between the input layer and the output layer.
  • Neural networks used in deep learning applications often include many hidden layers, increasing the depth of the network.
  • the layers of the neural network are connected in sequence so that the output of the previous layer is provided as the input of the subsequent layer, where the input layer receives the input of the neural network, and the output of the output layer serves as the final output of the neural network.
  • Each layer of a neural network consists of one or more nodes (also known as processing nodes or neurons), each of which processes input from the previous layer.
  • machine learning can roughly include three phases, namely training phase, testing phase and application phase (also known as inference phase).
  • training phase a given model can be trained using a large amount of training data, and the parameter values are updated iteratively until the model can obtain consistent inferences that meet the expected goals from the training data.
  • a model can be thought of as being able to learn associations from inputs to outputs (also known as input-to-output mappings) from the training data.
  • the parameter values of the trained model are determined.
  • test input is applied to the trained model to test whether the model can provide the correct output, thereby determining the performance of the model.
  • the model can be used to process the actual input and determine the corresponding output based on the parameter values obtained by training.
  • a large amount of training data including representations, evidence sets, and labels can be used to train the verification model, so that the verification model can verify the authenticity of the representation based on the input evidence set.
  • FIG. 1 shows a block diagram of an example environment 100 in which implementations of the present disclosure can be implemented.
  • a model ie, verification model 130
  • the environment 100 includes a model training system 150 and a model application system 152 , and a verification model 130 may be implemented using machine learning techniques.
  • the upper part of Fig. 1 shows the process of the model training phase, and the lower part shows the process of the model application phase.
  • the parameter values of the verification model 130 may have initial values, or may have pre-trained parameter values obtained through a pre-training process.
  • the parameter values of the verification model 130 can be updated and adjusted.
  • a verification model 130' can be obtained after training is complete. At this point, the parameter values of the verification model 130' have been updated, and based on the updated parameter values, the verification model 130' can be used to implement the verification task in the application phase.
  • each training data 112 may refer to a triplet format, including, for example, representations 120 , evidence sets 122 (eg, including evidence from sources such as encyclopedias and news), and tags 124 .
  • representations and evidence sets may be expressed in one or more natural languages. In the following, only English will be used as an example of natural language to describe specific details about the verification process. According to an exemplary implementation of the present disclosure, the expression 120 and the evidence set 122 may also be expressed in any language, including but not limited to Chinese, Japanese, French, Russian, Spanish, and so on.
  • expression 120 may be expressed in natural language, for example expression 120 may include "Bob won the 2020 election", evidence set 122 may include one or more evidence from sources such as encyclopedias and news, and tags 124 A tag may be included to indicate whether the content of the representation 120 is authentic or not. For example, “for,” “against,” or “not sure.”
  • Validation model 130 may be trained using training data 112 including representations 120 , evidence set 122 and labels 124 . Specifically, the training process can be performed iteratively using a large amount of training data. After the training is completed, the verification model 130' can verify whether the content of the representation is true based on the representation and the evidence set in the input data.
  • the verification model 130' can be invoked by the model application system 152 (the verification model 130' has the parameter values after training at this time), and the above verification tasks can be performed.
  • input 140 including representations 142 and evidence sets 144
  • verification results 146 output may be received.
  • the model training system 150 and the model application system 152 may include any computing system with computing capabilities, such as various computing devices/systems, terminal devices, servers, and the like.
  • Terminal equipment may refer to any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, or any combination of the foregoing , including accessories and peripherals for these devices, or any combination thereof.
  • Servers include, but are not limited to, mainframes, edge computing nodes, computing devices in cloud environments, and the like.
  • model training system 150 and model application system 152 may be integrated in the same system or device. Implementations of the present disclosure are not limited in this respect. Exemplary implementations of model training and model application will be described below with continued reference to the accompanying drawings.
  • an expression expressed in natural language (eg, denoted as c) may be received, and an evidence set (eg, denoted as E) for supporting the authenticity verification process may be acquired.
  • the evidence set can be obtained based on various technologies that are currently known and/or will be developed in the future, for example, the KGAT algorithm can be used to obtain the evidence set. How to verify the authenticity of the statement based on the acquired evidence set will be described in detail below.
  • a statement may represent the content being verified
  • a set of evidence may include at least one evidence supporting the verification of the truth of the statement
  • a label may represent a result of verifying the truth of the statement based on the set of evidence.
  • statements could include "Bob won the 2020 election”
  • evidence sets could include textual descriptions from multiple data sources
  • labels could include "against”.
  • the label indicates that the statement is not true.
  • a verification model 130 may be constructed to complete the above verification process.
  • the verification goal is to predict the following probability distribution: p(y
  • the representation can be divided into multiple phrases (the set of phrases can be denoted as W c , and each phrase can be denoted as w i , w i ⁇ W c ).
  • phrases may be extracted based on heuristic rules, and phrases may include named entities (NE), verb phrases, adjective phrases (AP), and noun phrases (NP), among others.
  • phrase truth can provide an explanation for statement truth, that is, clarify which phrase/phrases are based on which the statement is determined to be true or false. In this way, a verification scheme that is more accurate and has a higher granularity can be provided.
  • FIG. 2 shows a block diagram 200 of a verification model for verifying the authenticity of representations, according to some implementations of the present disclosure.
  • FIG. 2 describes the details of the verification model by taking the training data 112 as a specific example.
  • the expression 120 in the training data 112 is "Bob won the 2020 election”
  • the evidence set 122 in the training data 112 may include one or more evidences 230 .
  • the evidence 230 may be evidence retrieved based on various technical solutions that have been developed and/or will be developed in the future, so as to be used as a basis for authenticity verification.
  • Evidence 230 may include a variety of types, including but not limited to encyclopedias, news, textbooks, papers, and the like.
  • the representation 120 may be parsed based on a variety of natural language processing techniques to divide the representation 120 (denoted by the symbol c) into phrases. For example, each phrase can be represented by the symbol wi . In this example, w 1 means “Bob”, w 2 means “won”, and w 3 means "the 2020 election”.
  • the local premise building block 210 can be utilized to determine the local premise of each phrase respectively. Specifically, the detection question qi can be constructed for each phrase, and the evidence phrase w' i matching the phrase w i can be determined in the evidence set, and then the corresponding local premise c' i can be determined.
  • a probe question q2 can be constructed for this phrase, and an answer w′ 2 to the probe question can be determined in the evidence set 122 . Further, w' 2 can be used to replace the phrase w 2 in the expression, and then generate the corresponding local premise c' 2 . According to an exemplary implementation of the present disclosure, similar processing can be performed on each phrase, so as to obtain the corresponding local premise.
  • the authenticity verification module 220 can be processed by the authenticity verification module 220 based on the original representation c, the evidence set E and the generated local premises c′ 1 , c′ 2 and c′ 3 in order to determine the phrase authenticity zi and the entire The representational truth of the representation y.
  • a training data set 110 including a plurality of training data 112 may be acquired. Further, similar processing can be performed for each training data, so as to iteratively train the verification model with multiple training data. Referring first to Fig. 3 it is described how the local premises are determined from the training data.
  • FIG. 3 shows a block diagram 300 for determining local premises, according to some implementations of the disclosure.
  • a syntax analysis may be performed on the expression 120 to divide the expression into a plurality of phrases w 1 , w 2 , w 3 .
  • various extraction algorithms that are currently known and/or will be developed in the future can be used to determine the number of phrases included in the expression c.
  • Set W c For example, part-of-speech (POS) parsing tools can be used to identify verbs.
  • POS part-of-speech
  • a syntactic component analysis tool may be used to identify noun phrases. In order to refine the recognition granularity of noun phrases, you can further use POS analysis tools and named entity recognition parsers to split noun phrases.
  • various heuristic rules can be assisted, for example, 1) all leaf node noun phrases can be parsed using a syntactic component analysis tool, and all verbs can be identified using a POS parsing tool; 2) can be used The named entity recognition parser splits the leaf node noun phrases to obtain fine-grained noun phrases and corresponding adjective phrases.
  • the verification model may include a phrase verification model and an expression verification model.
  • the phrase verification model includes a relation among representations, evidence sets, and phrase authenticity of each phrase in the representation, and can determine the phrase authenticity of each phrase in the representation.
  • the representation verification model may include associations among representations, evidence sets, and representation truthfulness of representations, and may determine representation truthfulness of representations.
  • the phrase verification model may be trained based on the training data and the plurality of phrases, such that the phrase verification model determines a plurality of phrase authenticities for the plurality of phrases, respectively, based on the evidence set.
  • each phrase may be processed to determine the local premises corresponding to each phrase.
  • the process of determining the local premise c′ 1 where the local premise represents the knowledge used to verify the authenticity of the phrase, will be described taking the phrase w 1 as an example.
  • the probing question generation module 310 and the MRC module 330 may be utilized to determine phrase evidence 340 that matches each phrase in the evidence set 122 .
  • phrase evidence 340 may represent a point of knowledge for determining the authenticity of a phrase. In the following, more details of determining phrase evidence 340 are described with reference to FIG. 3 .
  • a probe question generation module 310 may be used to generate a probe question 320 for the phrase w i .
  • Probing questions can be generated based on various methods, for example, the phrase w i can be removed from the expression c to generate a cloze sentence associated with the expression c as the probing question qi.
  • the phrase "Bob" can be removed from the formulation 120, and the probe question 320 can be expressed as: "[mask] won the 2020 election". That is, the detection question 320 at this time is not a complete sentence, but a cloze sentence including the unknown part "[mask]".
  • an interrogative sentence may be used to express a detection question.
  • an interrogative sentence for the query phrase can be generated and used as a detection question. For example, based on grammatical analysis, it may be determined that the phrase "Bob" is in the position of the subject in the utterance. At this point, the interrogative sentence "Who won the 2020 election?" can be used as a detection question.
  • the relationship between phrases and expressions can be conveniently described by constructing detection questions, and then the meaning of each phrase in the language environment specified by the expressions can be retrieved in the evidence set 122 .
  • the information needed to verify each phrase can be found from evidence set 122 . Further, a set of local premises can be constructed based on these information. Specifically, the above process can be transformed into an MRC task.
  • an evidence phrase 340 associated with a phrase may be determined based on the MRC module 330 .
  • the MRC module 330 may include a pre-trained MRC model. Probe questions 320 and evidence sets 122 may be input to the MRC model, and the MRC model may output answers to probe questions 320 obtained from receiving evidence sets 122 .
  • FIG. 4 shows a block diagram 400 for training an MRC model according to some implementations of the present disclosure.
  • the MRC model 420 may be trained using the training data set 110 .
  • the training data set 110 here may include a large amount of training data, and each training data may include three parts: representation, evidence set and label.
  • the MRC model includes the association relationship between the question and the evidence set, and the answer to the input question can be found from the evidence set.
  • an initial reading comprehension model can be established, the model can be trained, and the answer output by the trained reading comprehension model is consistent with the real answer to the probe question.
  • the same training data set 110 can be used to train each model in the verification model.
  • training data satisfying predetermined conditions can be obtained from the training data set. It will be understood that only the training data labeled "support" are selected here, so that the selected training data is used to train the MRC model.
  • the MRC model can be trained based on a self-supervised manner, at this time, the training data 410 labeled as “support” can be selected from the training data set 110 .
  • the representation 412 of the training data 410 may include "The 2020 Game was held in 2021”
  • the evidence set 414 includes one or more evidences associated with "2020 Game” for validating the representation
  • the label 416 May include "support”. Since the label of the training data 410 is "support”, the MRC model 420 can be trained using the training data.
  • the probe question "The 2020 Game was held in [mask]” can be generated, and "2021" can be directly used as the answer to the probe question.
  • the MRC model 420 can be trained based on the probe questions, the evidence set 414 and the answers.
  • training data set 110 can be reused as much as possible, and various overheads related to preparing training data during the training process can be reduced.
  • the MRC model 420 can be trained based on the method described above. Further, the trained MRC model 420 can be used to determine the answer to each probe question. Specifically, for each phrase w i ( wi ⁇ W c ), the probe question generator 310 described above can be used first to generate probe questions q i , where the set of all probe questions q i can be expressed as Can be input to the MRC model and E, and then the set of answers for all probe questions can be obtained
  • the phrase in the statement may be replaced with the evidence phrase, so as to generate a local premise.
  • the evidence phrase 340 found is w' 1 : "Bob”.
  • the phrase w 1 in the expression c can be replaced by w' 1 to generate a local premise c' 1 .
  • a verification model can provide authenticity verification for expressions at the phrase level.
  • the phrase verification model can determine the truth of the phrase
  • the representation verification model can determine the truth of the representation.
  • an objective function for training a phrase verification model and an expression verification model may be established.
  • the authenticity of the phrase can be expressed as an implicit variable z i that chooses one of three (for example, one of SUP, REF and NEI).
  • W c denotes the number of phrases in expression c.
  • the truth y of the expression c depends on the implicit variable z.
  • the concept of logical rule constraints is proposed. It will be appreciated that there is a logical constraint between the phrase truth of each phrase in the representation and the representation truth of the representation.
  • the logical rule constraints between the authenticity of multiple phrases and the authenticity of representations can be obtained, specifically, 1) for the REF label, if the evidence set is against at least one of the multiple phrases, the representational authenticity is REF; 2) for For the SUP label, if the evidence set supports all of the phrases, the authenticity is expressed as SUP; 3)
  • SUP For the NEI label, if the above two are not satisfied, the verification result is uncertain. Therefore, a verification model can be constructed based on the above three constraints. Specifically, a logical rule constraint as shown in Formula 1 below can be defined:
  • V(c, W c , E) represents three numerical values (i.e., for, against, and indeterminate).
  • V(c, Wc , E) corresponds to one of three predetermined labels y ⁇ SUP, REF, NEI ⁇ .
  • an implicit variable model can be established, and further, the above logical rule constraints can be applied to the implicit variable model.
  • x) can be defined:
  • z i are independent of each other, that is, p(z
  • x) ⁇ ip(z i
  • the objective function can be constructed as follows:
  • the Expectation-maximization (EM) algorithm can be used to optimize the model.
  • EM Expectation-maximization
  • the negative evidence lower bound (negative Evidence Lower BOund, negative ELBO) can be minimized, and formula 4 is used to express the objective function related to variables:
  • NLI Natural Language Inference
  • logical rule constraints can be introduced based on logical knowledge distillation. That is, the objective function can be updated using the logical rule constraint shown in formula 1, so that the relationship between the authenticity of multiple phrases and the authenticity of the expression output by the verification model trained based on the objective function satisfies the logical rule constraint.
  • a knowledge distillation method is proposed, which can use a teacher model and a student model to construct a final objective function.
  • z, x) is the target to be optimized, and the variational distribution q ⁇ (z
  • the subspace y z is a logical aggregation of z
  • the subspace is constrained by the logical rules as expressed in Equation 1 above.
  • the distillation loss can be determined based on the following formula 5:
  • the objective function for jointly training the phrase verification model and the expression verification model can be established, taking into account both the objective function related to the variables in the training data and the objective function related to the logical rule constraints.
  • the objective function makes the relationship between the authenticity of the expression and the label satisfy a predetermined condition, and can make the relationship between the authenticity of the phrase and the authenticity of the expression satisfy the constraints of logical rules.
  • the final objective function can be determined based on the following formula 6:
  • logical rule constraints can be applied to the process of constructing the objective function, so that the objective function not only considers the training data in the training data set, but also considers the logic between the authenticity of phrases and the authenticity of expressions Rules bound. In this way, the verification model can be enabled to provide explanations for the stated verification results at the phrase level.
  • a soft logic scheme may be used during training and latent variable normalization.
  • the aggregation operation can be performed based on the following formula 7:
  • the training data can be encoded, and then the phrase verification model and the expression verification model are used to parameterize p ⁇ (y
  • the phrase verification model and the expression verification model are used to parameterize p ⁇ (y
  • FIG. 5 shows a block diagram 500 for training a phrase verification model and an utterance verification model, according to some implementations of the present disclosure.
  • input data for training the phrase verification model 510 and the utterance verification model 520 may be determined based on the training data 112 .
  • the input 512 to the phrase verification model 510 may include representation authenticity 514, local encoding 516, and global encoding 518.
  • Inputs 522 to representation verification model 520 may include phrase truth 524 , global encoding 518 , and contextual encoding 528 .
  • specific formulas will be used to show how to calculate the above parameters.
  • representation authenticity may be determined 514 based on labels in the training data 112 that indicate representation authenticity (ie, ground-truth data).
  • a pre-trained language model PLM
  • the representation c can be concatenated with each local premise c′ i to obtain a textual representation with respect to the local encoding 516
  • An encoder can then be utilized to map this textual representation to a local encoding 516 (i.e., represents the coding space).
  • various encoders that are currently known and/or will be developed in the future can be used to perform the above encoding process.
  • This textual representation can then be encoded to obtain a global encoding 518 (i.e. ).
  • a self-selection model can be applied in order to obtain the most important parts in the code.
  • suspicious phrase attention models can be designed based on the following experience: effective local premises (i.e., beneficial to the output The partial premise of the truth of the statement according to the actual situation) should be semantically close to the evidence in the evidence set. Therefore, based on and h global to determine the importance ⁇ i of each phrase w i .
  • the context code h local can be determined based on the following formula 8:
  • both p ⁇ ( ) and q ⁇ ( ) can be applied to the two-layer multi-layer perception model.
  • y, x) may include the concatenation of the following three parameters: representation authenticity 514(y * ), local encoding 516 and a global encoding 518 (h global ), and the output 514 may include zi (ie, the probability of phrase truth for each phrase).
  • the objective function shown in the above formula 6 can be used to jointly optimize both q ⁇ ( ⁇ ) and p ⁇ ( ⁇ ) during the training process. That is, the phrase verification model 510 and the expression verification model 520 are jointly trained using the objective function. Iterative optimization can be performed using a variety of optimization techniques that are currently known and/or will be developed in the future. For example, the optimization can be performed using the Gumbel representation algorithm for discrete argmax operations.
  • Equation 6 represent all parameters associated with the distributions p ⁇ and q ⁇ respectively described above.
  • the specific values of each parameter determined based on the training data set for example, y * , h global , z i , h local , etc.
  • the portion of the objective function associated with the phrase verification model 510 may involve labeling, local encoding, and global encoding
  • the portion of the objective function associated with the representation verification model 520 may involve phrase authenticity, global encoding, and context encoding.
  • Each training data in the training data set 110 can be processed one by one in a similar manner so as to obtain various parameters for performing training.
  • the training process may be performed iteratively using the obtained parameters until each model satisfies a predetermined stopping condition.
  • the trained expression verification model 520 can output the expression truth of the expression
  • the trained phrase verification model 510 can output the phrase truth of each phrase, where the phrase truth can provide an explanation for the expression truth.
  • authenticity may be represented in a probability distribution manner. For example, when expressing the probability distribution of authenticity in the order of SUP, REF, and NEI, assuming that the authenticity of the expression is (0.8, 0.1, 0.1), at this time the probability 0.8 associated with SUP is the maximum value, then the expression authenticity Sex is "SUP", which means support. According to an exemplary implementation manner of the present disclosure, phrase authenticity can be represented in a similar manner, which will not be described in detail below.
  • the verification model can not only verify the authenticity of the expression, but also process each phrase in the expression at a finer granularity, and verify the phrase authenticity of each phrase .
  • phrase authenticity can represent the contribution of the corresponding phrase to the ultimate authenticity of the utterance, and thus can provide an explanation for the final verification result.
  • Input data 140 may be input to the trained validation model 130'.
  • input data 140 may include a representation 142 to be verified and a set of evidence 144 to support the verification.
  • FIG. 6 shows a block diagram 600 for performing inference based on a phrase verification model and an expression verification model, according to some implementations of the disclosure.
  • input data 140 may include representations 142 and evidence sets 144 , for example.
  • the representation 142 represents the content to be verified
  • the evidence set 144 includes at least one evidence supporting the verification of the authenticity of the representation.
  • the individual modules in the validation model operate in a manner similar to the training process.
  • representation 142 may be divided into phrases and parameters associated with each phrase determined based on grammatical analysis.
  • phrase verification model 510 may be utilized to determine multiple phrase authenticities for multiple phrases, respectively, based on evidence set 144 .
  • the specific values of the parameters in the inputs 610 and 620 can be respectively determined according to the method described above.
  • Representation verification model 520 may be utilized to determine the representation truthfulness of a representation based on evidence set 144 and a plurality of phrase truths (ie, output 612 ), where the plurality of phrase truths provide an explanation for the representation truthfulness.
  • the phrase authenticity verification model 510 may set the plurality of phrase authenticity respectively to a plurality of predetermined initial values, such as (0.4, 0.3, 0.3) or other numerical values. At this time, the value of the output 612 shown in FIG.
  • This value may be used as an input to the representation authenticity model 520, and together with the global encoding h global and the context encoding h global determined from the input data 140, is used to determine the representation authenticity (ie, output 622). At this point, the first round of inference operation ends.
  • the reasoning process described above may be iteratively performed in multiple rounds.
  • the output 622 can be taken as input y to the phrase truth model 510 in the second round and combined with the local encoding determined from the input data 140 Together with the global encoding h global to obtain the new phrase truth at output 612.
  • the new phrase truth can be input to the representation truth model 520 to obtain the new representation truth at output 622 .
  • the iterative process has been described above with only the first round and the second round as examples.
  • one or more subsequent rounds may be executed until a predetermined stop condition is met.
  • the stop condition may specify, for example: stop when the difference between the output results of the first and last two rounds is less than a predetermined threshold.
  • the stopping condition may also specify: stop when the authenticity indicated by the probability distribution of two or more consecutive rounds no longer changes.
  • the stop condition may also specify: stop the iterative process when the predetermined number of rounds is reached.
  • the verification model can not only output representation authenticity for the overall representation, but also process individual phrases in the representation at a finer granularity. Further, the phrase truth of each phrase can be output to provide an explanation for the truth of the expression.
  • FIG. 7A shows a block diagram 700A for determining the authenticity of a statement, according to some implementations of the disclosure.
  • representations 142 and evidence sets 144 may be received.
  • Representation 142 may be divided into a plurality of phrases 712A, 714A, and 716A, and further, local premises associated with each phrase may be determined respectively.
  • local premise 730A shows the local premise associated with phrases 712A, 714A, and 716A.
  • Premise 1 represents the local premise associated with phrase 712A, and the phrase truth of phrase 712A is "supported” (as indicated by maximum value 732A).
  • Premise 2 represents the local premise associated with phrase 714A, and the phrase truth of phrase 714A is "against" (as indicated by maximum value 734A).
  • Premise 3 represents the local premise associated with phrase 716A, and the phrase truth of phrase 716A is "supported” (as indicated by maximum value 736A).
  • prediction 740A shows a predicted result of authenticity verification of "Disagreement", as indicated by maximum value 742A. This example shows that the prediction result is consistent with the truth value. It can be seen that the phrase authenticity can provide a more fine-grained prediction result and can provide an explanation for the final prediction result.
  • FIG. 7B shows a block diagram 700B for determining the authenticity of a statement, according to some implementations of the disclosure.
  • representation 710B and evidence set 720B may be received.
  • Expression 710B may be divided into a plurality of phrases 712B and 714B.
  • local premises associated with each phrase may be determined respectively.
  • local premise 730B shows the local premise associated with phrases 712B and 714B.
  • Premise 1 represents the local premise associated with phrase 712B, and the phrase truth of phrase 712B is "supported” (as indicated by maximum value 732B).
  • Premise 2 represents the local premise associated with phrase 714B, and the phrase truth of phrase 714B is "against" (as indicated by maximum value 734B).
  • prediction 740B shows a prediction result of authenticity verification of "disagreement", as indicated by maximum value 742B. This example shows a situation where the predicted result does not agree with the true value ("not sure”).
  • the phrase authenticity of the phrase 714B is "oppose" which leads to an error in the prediction result. In this way, the phrase authenticity can reflect the cause of errors to a certain extent, which is beneficial to further optimize the verification model.
  • FIG. 8 shows a flowchart of a method 800 for verifying the authenticity of a representation, according to some implementations of the present disclosure.
  • training data including representations, evidence sets, and labels are obtained
  • the representations represent the content to be verified
  • the evidence set includes at least one evidence for supporting the authenticity of the verification representation
  • the labels represent the content to be verified based on the evidence set.
  • the expression is divided into a plurality of phrases.
  • the phrase verification model is trained based on the training data and the plurality of phrases, such that the phrase verification model determines a plurality of phrase truths for the plurality of phrases, respectively, based on the evidence set.
  • phrases among the plurality of phrases may be processed one by one.
  • local premises associated with a phrase representing knowledge used to verify the authenticity of the phrase, may be determined based on the evidence set.
  • a phrase authenticity verification model can be trained based on local premises and training data.
  • an evidence phrase matching a phrase may be determined based on an evidence set, and the evidence phrase represents a knowledge point for determining the authenticity of the phrase. Further, phrases in statements can be replaced with evidence phrases in order to generate local premises.
  • probing questions associated with the phrase may be generated based on the utterance. Then, the answers to the probe questions are retrieved in the evidence set as evidence phrases.
  • phrases may be removed from the representation to use the cloze sentences associated with the representation as probing questions.
  • interrogative sentences for query phrases may be used as probing questions based on the position of the phrase in the expression.
  • the label includes any one of the following: “support”, “against” and “not sure”.
  • a reading comprehension model may be established that makes the answer consistent with the true answer to the probe question.
  • Another training data including representations, evidence sets and labels may be obtained. If the label of another training data is "support”, another training data can be used to train the reading comprehension model.
  • the representation verification model is trained based on the training data and the plurality of phrases that provide an explanation for the representation truth such that the representation verification model determines representation truthfulness of the representation based on the evidence set.
  • an objective function for jointly training the phrase verification model and the expression verification model can be established, the objective function makes the relationship between the authenticity of the expression and the label satisfy a predetermined condition.
  • the objective function can be used to jointly train the phrase verification model and the expression verification model.
  • a plurality of logical rule constraints between the authenticity of phrases and the authenticity of expressions can be obtained.
  • the objective function can be updated based on the logical rule constraints, and the objective function makes the relationship between the plurality of phrase authenticity and the expression authenticity satisfy the logical rule constraints.
  • multiple local codes respectively associated with multiple phrases may be determined based on the representation and multiple local premises.
  • a global encoding of representations can be determined based on representations and evidence sets.
  • the objective function can be determined using labels, multiple local codes and global codes as parameters.
  • a plurality of importances of a plurality of phrases may be respectively determined based on a comparison of a plurality of local premises and expressions.
  • the context encoding of the representation can be determined based on the multiple importances and the multiple local codes. Multiple phrase authenticity, context encoding and global encoding can be utilized as parameters.
  • FIG. 9 shows a flowchart of a method 900 for verifying the authenticity of a representation, according to some implementations of the present disclosure.
  • a statement representing what is to be verified and a set of evidence associated with the statement may be obtained, the statement representing content to be verified, and the set of evidence including at least one piece of evidence supporting verification of the authenticity of the statement.
  • the expression may be divided into phrases based on the grammatical analysis of the expression.
  • a plurality of phrase authenticities for the plurality of phrases, respectively, may be determined based on the evidence set utilizing the phrase verification model.
  • multiple phrase authenticities may be set to multiple predetermined initial values, respectively.
  • a plurality of phrase authenticity is determined based on the evidence set and the representation authenticity.
  • a representation verification model may be utilized to determine a representational truth of the representation based on the evidence set and a plurality of phrases that provide an explanation for the representational truthfulness.
  • the method 900 may be iteratively executed until the relationships between the authenticity of multiple phrases and the authenticity of expressions satisfy a predetermined stop condition.
  • FIG. 10A shows a block diagram of an apparatus 1000A for verifying the authenticity of an expression according to some implementations of the present disclosure.
  • an apparatus 1000A includes an acquisition module 1010A, a division module 1020A, a phrase verification module 1030A, and an expression verification module 1040A.
  • the acquisition module 1010A is configured to acquire training data including expressions, evidence sets and labels, the expressions represent the content to be verified, and the evidence sets include at least one The evidence, and the label represents the result of verifying the authenticity of the expression based on the evidence set;
  • the segmentation module 1020A is configured to divide the expression into a plurality of phrases based on the grammatical analysis of the expression;
  • the phrase verification module 1030A is configured to be based on the training data and a plurality of phrases to train a phrase verification model, so that the phrase verification model respectively determines a plurality of phrase authenticity of a plurality of phrases based on an evidence set;
  • a representation verification module 1040A configured to train a representation based on training data and a plurality of phrases The model is validated such that the representation verification model determines a representational truth of the representation based on the evidence set, wherein the plurality of phrase truths provide an explanation for the representational truthfulness.
  • the phrase verification module is further configured to: for a phrase in the plurality of phrases, determine a local premise associated with the phrase based on the evidence set, the local premise represents the knowledge used to verify the authenticity of the phrase ; and training a phrase authenticity verification model based on local premises and training data.
  • the phrase verification module is further configured to: determine an evidence phrase matching the phrase based on the evidence set, where the evidence phrase represents a knowledge point for determining the authenticity of the phrase; and replace the expression with the evidence phrase phrases in , in order to generate local premises.
  • the phrase verification module is further configured to: generate a probing question associated with the phrase based on the expression; and retrieve an answer to the probing question in the evidence set as the evidence phrase.
  • the phrase verification module is further configured to: remove the phrase from the expression to use the cloze sentence associated with the expression as a detection question; and based on the position of the phrase in the expression, use Interrogative sentences for query phrases serve as probing questions.
  • the label includes any one of the following: “support”, “against” and “not sure”.
  • the phrase verification module is further configured to: build a reading comprehension model that reconciles the answer with the true answer to the probe question; obtain another training data including statements, evidence sets, and labels; and label in response to the other training data For "support”, another training data is used to train the reading comprehension model.
  • an establishment module configured to establish an objective function for jointly training the phrase verification model and the expression verification model, the objective function makes the relationship between the authenticity of the expression and the label satisfy a predetermined condition .
  • the phrase verification module and the representation verification module are further configured to jointly train the phrase verification model and the representation verification model using an objective function.
  • the establishment module is further configured to: obtain the logical rule constraints between the authenticity of multiple phrases and the authenticity of the expression; and update the objective function based on the logical rule constraints, the objective function makes the multiple phrases real The relationship between sex and representational authenticity satisfies logical rule constraints.
  • the establishment module is further configured to: determine a plurality of local codes respectively associated with a plurality of phrases based on the representation and a plurality of local premises; determine a global code of the representation based on the representation and the evidence set; And using labels, multiple local codes and global codes as parameters to determine the objective function.
  • the establishment module is further configured to: determine multiple importances of multiple phrases based on comparisons between multiple local premises and expressions; determine multiple importances of multiple phrases based on multiple importances and multiple local codes, determine Contextual encoding of representations; multiple phrase truths, contextual encodings, and global encodings are used as parameters to determine an objective function.
  • Fig. 10B shows a block diagram of an apparatus 1000B for verifying the authenticity of an expression according to some implementations of the present disclosure.
  • the apparatus 1000B includes an acquisition module 1010B, a segmentation module 1020B, a phrase verification module 1030B and an expression verification module 1040B.
  • the obtaining module 1010B is configured to obtain training data including a statement, an evidence set and a label, the statement represents the content to be verified, and the evidence set includes information used to support the verification of the statement at least one evidence of the authenticity of the expression, and the label represents a result of verifying the authenticity of the expression based on the set of evidence;
  • the segmentation module 1020B is configured to, based on the syntax analysis of the expression, The expression is divided into a plurality of phrases;
  • the phrase verification module 1030B is configured to train a phrase verification model based on the training data and the plurality of phrases, so that the phrase verification model determines the a plurality of phrase authenticity of the plurality of phrases;
  • a representation verification module 1040B configured to train a representation verification model based on the training data and the plurality of phrases such that the representation verification model is based on the evidence set
  • a statement truth of the statement is determined, wherein the plurality of phrase truths provide an explanation for the statement truth.
  • the phrase verification module 1030B is further configured to set a plurality of phrase authenticity to a plurality of predetermined initial values in the initial stage; and in a subsequent stage after the initial stage, based on the evidence set and Statement authenticity to determine multiple phrase authenticity.
  • the phrase verification module 1030B and the expression verification module 1040B are invoked iteratively until a plurality of relationships between phrase authenticity and expression authenticity satisfy a predetermined stop condition.
  • FIG. 11 shows a block diagram of a device 1100 capable of implementing various implementations of the present disclosure. It should be understood that the computing device 1100 shown in FIG. 11 is exemplary only, and should not constitute any limitation as to the functionality and scope of the implementations described herein. The computing device 1100 shown in FIG. 11 can be used to implement the model training system 150 shown in FIG. 1 , and can also be used to implement the model application system 152 shown in FIG. 1 .
  • computing device 1100 is in the form of a general-purpose computing device.
  • Components of computing device 1100 may include, but are not limited to, one or more processors or processing units 1110, memory 1120, storage devices 1130, one or more communication units 1140, one or more input devices 1150, and one or more output devices 1160.
  • the processing unit 1110 may be an actual or virtual processor and is capable of performing various processes according to programs stored in the memory 1120 . In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capability of the computing device 1100 .
  • Computing device 1100 typically includes a plurality of computer storage media. Such media can be any available media that is accessible to computing device 1100, including but not limited to, volatile and nonvolatile media, removable and non-removable media.
  • Memory 1120 can be volatile memory (e.g., registers, cache, random access memory (RAM), nonvolatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM) , flash memory) or some combination of them.
  • Storage device 1130 may be removable or non-removable media, and may include machine-readable media, such as flash drives, magnetic disks, or any other media that may be capable of storing information and/or data (e.g., training data for training ) and can be accessed within computing device 1100.
  • machine-readable media such as flash drives, magnetic disks, or any other media that may be capable of storing information and/or data (e.g., training data for training ) and can be accessed within computing device 1100.
  • the computing device 1100 may further include additional removable/non-removable, volatile/nonvolatile storage media.
  • a disk drive for reading from or writing to a removable, nonvolatile disk such as a "floppy disk"
  • a disk drive for reading from a removable, nonvolatile disk may be provided.
  • CD-ROM drive for reading or writing.
  • each drive may be connected to the bus (not shown) by one or more data media interfaces.
  • Memory 1120 may include a computer program product 1125 having one or more program modules configured to perform various methods or actions of various implementations of the present disclosure.
  • the communication unit 1140 enables communication with other computing devices through communication media. Additionally, the functionality of the components of computing device 1100 may be implemented in a single computing cluster or as a plurality of computing machines capable of communicating via communication links. Accordingly, computing device 1100 may operate in a networked environment using logical connections to one or more other servers, a network personal computer (PC), or another network node.
  • PC network personal computer
  • the input device 1150 may be one or more input devices, such as a mouse, keyboard, trackball, and the like.
  • Output device 1160 may be one or more output devices, such as a display, speakers, printer, or the like.
  • the computing device 1100 can also communicate with one or more external devices (not shown) through the communication unit 1140 as needed, such as storage devices, display devices, etc., and one or more devices that enable the user to interact with the computing device 1100 communicate with, or communicate with, any device (eg, network card, modem, etc.) that enables computing device 1100 to communicate with one or more other computing devices. Such communication may be performed via an input/output (I/O) interface (not shown).
  • I/O input/output
  • a computer-readable storage medium on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the methods described above.
  • a computer program product tangibly stored on a non-transitory computer-readable medium and comprising computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.
  • a computer program product on which a computer program is stored, the program implementing the method described above when executed by a processor.
  • These computer-readable program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processing unit of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices and/or other devices to work in a specific way, so that the computer-readable medium storing instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in flowcharts and/or block diagrams.
  • computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other equipment, so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process, Instructions executed on computers, other programmable data processing devices, or other devices can thus implement the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.
  • each block in a flowchart or block diagram may represent a module, a program segment, or a portion of an instruction that contains one or more executable instruction.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Machine Translation (AREA)

Abstract

根据本公开的实现方式，提供了用于验证表述的真实性的方法、设备、装置和介质。在一种方法中，获取包括表述、证据集和标签的训练数据，表述表示被验证的内容，证据集包括用于支持验证表述的真实性的至少一个证据，以及标签表示基于证据集来验证表述的真实性的结果。基于对表述的语法分析，将表述划分为多个短语。基于训练数据和多个短语来训练短语验证模型，以使得短语验证模型基于证据集来分别确定多个短语的多个短语真实性。基于训练数据和多个短语来训练表述验证模型，以使得表述验证模型基于证据集来确定表述的表述真实性，其中多个短语真实性对表述真实性提供解释。以此方式，以更为精细的粒度处理表述并且为表述真实性提供更多解释。

Description

用于验证表述的真实性的方法、设备、装置和介质
本申请要求于2021年11月16日提交中国专利局，申请号为202111356625.8，发明名称为“用于验证表述的真实性的方法、设备、装置和介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开的示例性实现方式总体涉及计算机领域，特别地涉及用于验证表述的真实性的方法、设备、装置和计算机可读存储介。
背景技术
随着自然语言处理技术的发展，目前已经提出了用于验证自然语言形式的表述的真实性的技术方案。然而，已有技术方案难以提供关于验证结果的解释，并且验证结果的准确性并不令人满意。因而，期望能够以更为有效和准确的方式来执行真实性验证。
发明内容
根据本公开的示例性实现方式，提供了一种用于验证表述的真实性的方案。
在本公开的第一方面，提供了一种用于验证表述的真实性的方法。在该方法中，获取包括表述、证据集和标签的训练数据，表述表示被验证的内容，证据集包括用于支持验证表述的真实性的至少一个证据，以及标签表示基于证据集来验证表述的真实性的结果。基于对表述的语法分析，将表述划分为多个短语。基于训练数据和多个短语来训练短语验证模型，以使得短语验证模型基于证据集来分别确定多个短语的多个短语真实性。基于训练数据和多个短语来训练表述验证模型，以使得表述验证模型基于证据集来确定表述的表述真实性，其中多个短语真实性对表述真实性提供解释。
在本公开的第二方面，提供了一种电子设备，包括：至少一个处理 单元；以及至少一个存储器，至少一个存储器被耦合到至少一个处理单元并且存储用于由至少一个处理单元执行的指令，指令在由至少一个处理单元执行时使设备执行动作。该动作包括：获取包括表述、证据集和标签的训练数据，表述表示被验证的内容，证据集包括用于支持验证表述的真实性的至少一个证据，以及标签表示基于证据集来验证表述的真实性的结果；基于对表述的语法分析，将表述划分为多个短语；基于训练数据和多个短语来训练短语验证模型，以使得短语验证模型基于证据集来分别确定多个短语的多个短语真实性；以及基于训练数据和多个短语来训练表述验证模型，以使得表述验证模型基于证据集来确定表述的表述真实性，其中多个短语真实性对表述真实性提供解释。
在本公开的第三方面，提供了一种用于验证表述的真实性的方法。在该方法中，获取表述和与表述相关联的证据集，表述表示将被验证的内容，以及证据集包括用于支持验证表述的真实性的至少一个证据；基于对表述的语法分析，将表述划分为多个短语；利用短语验证模型，基于证据集来分别确定多个短语的多个短语真实性；以及利用表述验证模型，基于证据集和多个短语来确定表述的表述真实性，其中多个短语真实性对表述真实性提供解释。
在本公开的第四方面，提供了一种电子设备，包括：至少一个处理单元；以及至少一个存储器，至少一个存储器被耦合到至少一个处理单元并且存储用于由至少一个处理单元执行的指令，指令在由至少一个处理单元执行时使设备执行动作。该动作包括：获取表述和与表述相关联的证据集，表述表示将被验证的内容，以及证据集包括用于支持验证表述的真实性的至少一个证据；基于对表述的语法分析，将表述划分为多个短语；利用短语验证模型，基于证据集来分别确定多个短语的多个短语真实性；以及利用表述验证模型，基于证据集和多个短语来确定表述的表述真实性，其中多个短语真实性对表述真实性提供解释。
在本公开的第五方面，提供了一种用于验证表述的真实性的装置。该装置包括：获取模块，配置用于获取包括表述、证据集和标签的训练数据，表述表示被验证的内容，证据集包括用于支持验证表述的真实性 的至少一个证据，以及标签表示基于证据集来验证表述的真实性的结果；划分模块，配置用于基于对表述的语法分析，将表述划分为多个短语；短语验证模块，配置用于基于训练数据和多个短语来训练短语验证模型，以使得短语验证模型基于证据集来分别确定多个短语的多个短语真实性；以及表述验证模块，配置用于基于训练数据和多个短语来训练表述验证模型，以使得表述验证模型基于证据集来确定表述的表述真实性，其中多个短语真实性对表述真实性提供解释。
在本公开的第六方面，提供了一种用于验证表述的真实性的装置。该装置包括：获取模块，配置用于获取表述和与表述相关联的证据集，表述表示将被验证的内容，以及证据集包括用于支持验证表述的真实性的至少一个证据；划分模块，配置用于基于对表述的语法分析，将表述划分为多个短语；短语验证模块，配置用于利用短语验证模型，基于证据集来分别确定多个短语的多个短语真实性；以及表述验证模块，配置用于利用表述验证模型，基于证据集和多个短语来确定表述的表述真实性，其中多个短语真实性对表述真实性提供解释。
在本公开的第七方面，提供了一种计算机可读存储介质。介质上存储有计算机程序，程序被处理器执行时实现第一方面的方法。
在本公开的第八方面，提供了一种计算机可读存储介质。介质上存储有计算机程序，程序被处理器执行时实现第三方面的方法。
应当理解，本发明内容部分中所描述的内容并非旨在限定本公开的实现方式的关键特征或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的描述而变得容易理解。
附图说明
在下文中，结合附图并参考以下详细说明，本公开各实现方式的上述和其他特征、优点及方面将变得更加明显。在附图中，相同或相似的附图标记表示相同或相似的元素，其中：
图1示出了本公开的实现方式能够在其中实现的示例环境的框图；
图2示出了根据本公开的一些实现方式的用于验证表述的真实性的 验证模型的框图；
图3示出了根据本公开的一些实现方式的用于确定局部前提的框图；
图4示出了根据本公开的一些实现方式的用于训练机器阅读理解(Machine Reading Comprehension，MRC)模型的框图；
图5示出了根据本公开的一些实现方式的用于训练短语验证模型和表述验证模型的框图；
图6示出了根据本公开的一些实现方式的用于基于短语验证模型和表述验证模型来执行推理的框图；
图7A示出了根据本公开的一些实现方式的用于确定表述的真实性的框图；
图7B示出了根据本公开的一些实现方式的用于确定表述的真实性的框图；
图8示出了根据本公开的一些实现方式的用于验证表述的真实性的方法的流程图；
图9示出了根据本公开的一些实现方式的用于验证表述的真实性的方法的流程图；
图10A示出了根据本公开的一些实现方式的用于验证表述的真实性的装置的框图；
图10B示出了根据本公开的一些实现方式的用于验证表述的真实性的装置的框图；以及
图11示出了能够实施本公开的多个实现方式的设备的框图。
具体实施方式
下面将参照附图更详细地描述本公开的实现方式。虽然附图中示出了本公开的某些实现方式，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实现方式，相反，提供这些实现方式是为了更加透彻和完整地理解本公开。应当理解的是，本公开的附图及实现方式仅用于示例性作用，并非用于限制本公开的保护范围。
在本公开的实现方式的描述中，术语“包括”及其类似用语应当理解为开放性包含，即“包括但不限于”。术语“基于”应当理解为“至少部分地基于”。术语“一个实现方式”或“该实现方式”应当理解为“至少一个实现方式”。术语“一些实现方式”应当理解为“至少一些实现方式”。下文还可能包括其他明确的和隐含的定义。
如本文中所使用的，术语“模型”可以从训练数据中学习到相应的输入与输出之间的关联，从而在训练完成后可以针对给定的输入，生成对应的输出。模型的生成可以基于机器学习技术。深度学习是一种机器学习算法，通过使用多层处理单元来处理输入和提供相应输出。神经网络模型是基于深度学习的模型的一个示例。在本文中，“模型”也可以被称为“机器学习模型”、“学习模型”、“机器学习网络”或“学习网络”，这些术语在本文中可互换地使用。
“神经网络”是一种基于深度学习的机器学习网络。神经网络能够处理输入并且提供相应输出，其通常包括输入层和输出层以及在输入层与输出层之间的一个或多个隐式层。在深度学习应用中使用的神经网络通常包括许多隐式层，从而增加网络的深度。神经网络的各个层按顺序相连，从而前一层的输出被提供作为后一层的输入，其中输入层接收神经网络的输入，而输出层的输出作为神经网络的最终输出。神经网络的每个层包括一个或多个节点(也称为处理节点或神经元)，每个节点处理来自上一层的输入。
通常，机器学习大致可以包括三个阶段，即训练阶段、测试阶段和应用阶段(也称为推理阶段)。在训练阶段，给定的模型可以使用大量的训练数据进行训练，不断迭代更新参数值，直到模型能够从训练数据中获取一致的满足预期目标的推理。通过训练，模型可以被认为能够从训练数据中学习从输入到输出之间的关联(也称为输入到输出的映射)。训练后的模型的参数值被确定。在测试阶段，将测试输入应用到训练后的模型，测试模型是否能够提供正确的输出，从而确定模型的性能。在应用阶段，模型可以被用于基于训练得到的参数值，对实际的输入进行处理，确定对应的输出。
在表述验证领域，可以利用包括表述、证据集和标签的大量训练数据来训练验证模型，进而使得验证模型可以基于输入的证据集来验证表述的真实性。
示例环境
图1示出了本公开的实现方式能够在其中实现的示例环境100的框图。在图1的环境100中，期望训练和使用这样的模型(即，验证模型130)，该模型被配置用于接收表述120和证据集122，并且基于证据集122来验证表述120的真实性。如图1所示，环境100包括模型训练系统150和模型应用系统152，可以使用机器学习技术来实现验证模型130。图1上部示出了模型训练阶段的过程，并且下部示出模型应用阶段的过程。在训练前，验证模型130的参数值可以具有初始值，或者可以具有通过预训练过程获得经预训练的参数值。经过训练过程，验证模型130的参数值可以被更新和调整。在训练完成后可以获得验证模型130’。此时，验证模型130’的参数值已经被更新，并且基于已更新的参数值，验证模型130’在应用阶段可以被用于实现验证任务。
在模型训练阶段，可以基于包括多个训练数据112的训练数据集110，并利用模型训练系统150来训练验证模型130。在此，每个训练数据112可以涉及三元组格式，例如包括表述120、证据集122(例如，包括来自于百科和新闻等资源的证据)和标签124。在本公开的上下文中，表述和证据集可以以一种或者多种自然语言来表示。在下文中，将仅以英文作为自然语言的示例，来描述有关验证过程的具体细节。根据本公开的一个示例性实现方式，表述120和证据集122还可以以任一语言来表示，包括但不限于汉语、日语、法语、俄语、西班牙语，等等。
在本公开的上下文中，表述120可以以自然语言表示，例如表述120可以包括“Bob won the 2020 election”，证据集122可以包括来自于百科和新闻等资源的一个或多个证据，并且标签124可以包括用于指示表述120的内容是否真实可靠的标签。例如，“支持”、“反对”或者“不确定”。可以利用包括表述120、证据集122和标签124的训练数据112 来训练验证模型130。具体地，可以利用大量训练数据迭代地执行训练过程。在训练完成之后，验证模型130’可以将以基于输入数据中的表述和证据集，来验证表述的内容是否是真实的。在模型应用阶段，可以利用模型应用系统152来调用验证模型130’(此时的验证模型130’具有训练后的参数值)，并且可以执行上述验证任务。例如，可以接收输入140(包括表述142和证据集144)，并且输出验证结果146。
在图1中，模型训练系统150和模型应用系统152可以包括具有计算能力的任何计算系统，例如各种计算设备/系统、终端设备、服务器等。终端设备可以涉及任意类型的移动终端、固定终端或便携式终端，包括移动手机、台式计算机、膝上型计算机、笔记本计算机、上网本计算机、平板计算机、媒体计算机、多媒体平板、或者前述各项的任意组合，包括这些设备的配件和外设或者其任意组合。服务器包括但不限于大型机、边缘计算节点、云环境中的计算设备，等等。
应当理解，图1示出的环境100中的部件和布置仅仅是示例，适于用于实现本公开所描述的示例性实现方式的计算系统可以包括一个或多个不同的部件、其他部件和/或不同的布置方式。例如，虽然被示出为是分离的，但模型训练系统150和模型应用系统152可以集成在相同系统或设备中。本公开的实现方式在此方面不受限制。以下将继续参考附图，分别描述模型训练和模型应用的示例性实现方式。
验证模型的架构
目前已经基于机器学习技术实现了多种验证方案。根据一种技术方案，可以接收以自然语言表示的表述，并且针对表述的内容是否真实给出“支持”、“反对”或者“不确定”的验证结果。然而，该技术方案并不能验证表述中的各个部分的真实性，也不能针对为何给出某个验证结果进行解释。根据另一技术方案，可以将表述中的某个部分进行高亮显示，以便指示基于该部分来得出了当前的验证结果。然而，上述技术方案仅能将表述作为一个整体给出验证结果，并不能单独地评价表述中的各个部分的真实性。因而，期望可以提供具有更为精细粒度的验证方 案。
根据本公开的一个示例性实现方式，提供了一种用于验证表述的真实性的技术方案。具体地，可以接收以自然语言表示的表述(例如，表示为c)，并且获取用于支持真实性验证过程的证据集(例如，表示为E)。可以基于目前已知的和/或将在未来开发的多种技术来获取证据集，例如，可以使用KGAT算法来获取证据集。在下文中将详细描述如何基于获取的证据集来验证表述的真实性。
在本公开的上下文中，表述可以表示被验证的内容，证据集可以包括用于支持验证表述的真实性的至少一个证据，以及标签可以表示基于证据集来验证表述的真实性的结果。例如，在一个训练数据中，表述可以包括“Bob won the 2020 election”，证据集可以包括来自多种数据来源的文字描述，并且标签可以包括“反对”。在此训练数据中，标签表示表述内容不是真实的。
根据本公开的一个示例性实现方式，可以构建验证模型130来完成上述验证过程。在此，验证目标是预测如下概率分布：p(y|c，E)，y∈{SUP，REF，NEI}。也即，基于E来将c分类为SUP、REF或者NEI，其中SUP表示“支持”、REF表示“反对”、并且“NEI”表示即不是“支持”也不是“反对”的不可确定状态。基于该验证模型，可以将表述划分为多个短语(短语集合可以表示为W c，并且每个短语可以表示为w i，w i∈W c)。例如，可以基于启发式规则来提取短语，并且短语可以包括命名实体(NE)、动词短语、形容词短语(AP)和名词短语(NP)，等等。
例如，在表述“Bob won the 2020 election”中，“Bob”表示NE，“won”表示动词短语，并且“the 2020 election”表示NP。进一步，可以在短语级别执行预测。例如，可以利用p(z i|c，w i，E)(其中z i∈{SUP，REF，NEI})来表示各个短语的真实性。利用本公开的示例性实现方式，不但可以确定整个表述的表述真实性，还可以分别确定每个短语的短语真实性。在此，短语真实性可以为表述真实性提供解释，也即，阐明基于哪个/哪些短语来确定表述的真假。以此方式，可以提供更加准 确并且具有更高精细粒度的验证方案。
在下文中，将首先参见图2描述根据本公开的一个示例性实现方式的验证模型的概要。图2示出了根据本公开的一些实现方式的用于验证表述的真实性的验证模型的框图200。图2以训练数据112作为具体示例来描述了验证模型的细节。如图2所示，训练数据112中的表述120为“Bob won the 2020 election”，并且训练数据112中的证据集122可以包括一个或多个证据230。在此，证据230可以是基于目前已经开发的和/或将在未来开发的多种技术方案检索到的证据，以便用作真实性验证的基础。证据230可以包括多种类型，包括但不限于百科、新闻、教科书、论文，等等。
可以基于多种自然语言处理技术来对表述120进行语法分析，以便将表述120(利用符号c表示)划分为多个短语。例如，可以利用符号w i来表示每个短语。在此示例中，w 1表示“Bob”，w 2表示“won”，并且w 3表示“the 2020 election”。可以利用局部前提构建模块210来分别确定各个短语的局部前提。具体地，可以为每个短语构造探测问题qi，在证据集中确定与短语w i相匹配的证据短语w′ i，进而确定相应的局部前提c′ i。以短语w 2为示例，可以针对该短语来构造探测问题q2，并且在证据集122中确定该探测问题的答案w′ 2。进一步，可以利用w′ 2来替换表述中的短语w 2，进而生成相应的局部前提c′ 2。根据本公开的一个示例性实现方式，可以针对每个短语进行类似的处理，以便获取相应的局部前提。
继而，可以由真实性验证模块220来基于原始表述c、证据集E和生成的局部前提c′ 1、c′ 2和c′ 3进行处理，以便确定每个短语的短语真实性z i和整个表述的表述真实性y。
利用本公开的示例性实现方式，不但可以确定整个表述的表述真实性，还可以以更为精细的粒度分别确定每个短语的短语真实性。在此，短语真实性可以针对为何给出“支持”、“反对”或者“不确定”的验证结果提供解释。以此方式，可以以更加精细的粒度确定表述内的各个部分的真假，从而提高真实性验证的准确性。已经参见图2描述了验证模型的概要，在下文中，将分别结合训练阶段和应用阶段描述有关该验 证模型的更多细节。
模型训练过程
在下文中，将参见附图描述有关训练过程的更多细节。根据本公开的一个示例性实现方式，可以获取包括多个训练数据112的训练数据集110。进一步，可以针对每个训练数据进行类似的处理，以便利用多个训练数据来迭代地训练验证模型。首先参见图3描述如何从训练数据中确定局部前提。图3示出了根据本公开的一些实现方式的用于确定局部前提的框图300。
如图3所示，可以针对表述120执行语法分析，以便将表述划分为多个短语w 1、w 2、w 3。根据本公开的一个示例性实现方式，为了将表述120划分为多个短语，可以使用目前已知的和/或将在未来开发的多种提取算法，来确定表述c中包括的多个短语的集合W c。例如，可以使用词性(part-of-speech，POS)解析工具来标识动词。又例如，可以使用句法成分分析工具来识别名词短语。为了细化名词短语的识别粒度，可以进一步使用POS解析工具和命名实体识别解析器对名词短语做拆分。根据本公开的一个示例性实现方式，可以辅助使用多种启发式规则，例如，1)可以利用句法成分分析工具解析全部叶节点名词短语，并且利用POS解析工具来识别全部动词；2)可以利用命名实体识别解析器来对叶节点名词短语进行拆分，从而得到细粒度名词短语以及对应的形容词短语。
根据本公开的一个示例性实现方式，验证模型可以包括短语验证模型和表述验证模型。短语验证模型包括表述、证据集、和表述中的各个短语的短语真实性之间的关联关系，并且可以确定表述中的各个短语的短语真实性。表述验证模型可以包括表述、证据集和表述的表述真实性之间的关联关系，并且可以确定表述的表述真实性。
在已经获得多个短语之后，可以基于训练数据和多个短语来训练短语验证模型，以使得短语验证模型基于证据集来分别确定多个短语的多个短语真实性。首先，可以对每个短语进行处理以便确定与每个短语相对应的局部前提。在下文中，将仅以短语w 1为示例描述确定局部前提c′ 1 的过程，在此局部前提表示用于验证短语的真实性的知识。根据本公开的一个示例性实现方式，可以利用探测问题生成模块310和MRC模块330来在证据集122中确定与各个短语相匹配的短语证据340。在此，短语证据340可以表示用于确定短语的真实性的知识点。在下文中，参见图3描述确定短语证据340的更多细节。
根据本公开的一个示例性实现方式，可以使用探测问题生成模块310来为短语w i生成探测问题320。可以基于多种方式来生成探测问题，例如，可以从表述c中移除短语w i，以生成与表述c相关联的完形填空语句来作为探测问题qi。对于短语w 1而言，可以将短语“Bob”从表述120中移除，此时探测问题320可以表示为：“[mask]won the 2020 election”。也即，此时探测问题320并不是完整的句子，而是包括未知部分“[mask]”的完形填空语句。
又例如，可以利用疑问句来表示探测问题。具体地，可以基于短语在表述中的位置，生成用于查询短语的疑问句并将其作为探测问题。例如，可以基于语法分析，确定短语“Bob”位于表述中主语的位置。此时，可以利用疑问句“Who won the 2020 election？”来作为探测问题。利用本公开的示例性实现方式，通过构造探测问题可以方便地描述短语与表述之间的关系，进而便于在证据集122中检索各个短语在由表述所指定的语言环境下的含义。
根据本公开的一个示例性实现方式，可以从证据集122中找到用于验证每个短语所需的信息。进一步，可以基于这些信息来构造局部前提的集合。具体地，上述过程可以转化为MRC任务。根据本公开的一个示例性实现方式，可以基于MRC模块330来确定与短语相关联的证据短语340。在此，MRC模块330可以包括预先训练好的MRC模型。可以向该MRC模型输入探测问题320和证据集122，并且MRC模型可以输出从接收证据集122获取的针对探测问题320的答案。
将会理解，可以基于目前已知的和/或将被开发的多种方式获取MRC模型。根据本公开的一个示例性实现方式，提出了一种基于自监督方式来训练MRC模型的方法。在下文中，参见图4描述有关训练过 程的更多细节。图4示出了根据本公开的一些实现方式的用于训练MRC模型的框图400。如图4所示，可以使用训练数据集110来训练MRC模型420。将会理解，在此的训练数据集110可以包括大量训练数据，并且每个训练数据可以包括表述、证据集和标签三部分。在此，MRC模型包括问题和证据集之间的关联关系，并且可以从证据集找到输入问题的答案。
根据本公开的一个示例性实现方式，可以建立初始阅读理解模型，可以训练该模型并且使得训练好的阅读理解模型输出的答案与探测问题的真实答案相一致。为了不增加获取训练数据的工作量，可以使用相同的训练数据集110来训练验证模型中的各个模型。为了训练MRC模型，可以从训练数据集中获取满足预定条件的训练数据。将会理解，在此仅选择标签为“支持”的训练数据，以便利用选择的训练数据来训练MRC模型。
具体地，可以基于自监督方式来训练MRC模型，此时可以从训练数据集110中选择标签为“支持”的训练数据410。如图4所示，训练数据410的表述412可以包括“The 2020 Game was held in 2021”，证据集414包括用于验证表述的与“2020 Game”相关联的一个或多个证据，并且标签416可以包括“支持”。由于训练数据410的标签为“支持”，因而可以利用训练数据来训练MRC模型420。在训练过程中，可以生成探测问题“The 2020 Game was held in[mask]”，并且将“2021”直接作为该探测问题的答案。进一步，可以基于探测问题、证据集414和答案来训练MRC模型420。
将会理解，对于标签为“反对”或者“不确定”的训练数据而言，难以从相应的证据集中找到真实的正确答案，因而在训练MRC模型420时，可以放弃这些训练数据并且仅使用标签为“支持”的训练数据。以此方式，可以尽量重用训练数据集110，并且降低训练过程准备训练数据的各种相关开销。
根据本公开的一个示例性实现方式，可以基于上文描述的方法来训练MRC模型420。进一步，可以使用训练好的MRC模型420来确定每 个探测问题的答案。具体地，对于每个短语w i(w i∈W c)，可以首先利用上文描述的探测问题生成器310来生成探测问题q i，在此可以将全部探测问题q i的集合表示为
Figure PCTCN2022132139-appb-000001
可以向MRC模型输入
Figure PCTCN2022132139-appb-000002
和E，进而可以获得针对全部探测问题的答案集合
Figure PCTCN2022132139-appb-000003
根据本公开的一个示例性实现方式，在已经基于证据集确定与短语相匹配的证据短语之后，可以利用证据短语替换表述中的短语，以便生成局部前提。返回图3，对于短语w 1“Bob”，找到的证据短语340为w′ 1：“Bob”。此时，可以利用w′ 1来替换表述c中的短语w 1，以便生成局部前提c′ 1。
将会理解，上文仅以短语w 1为示例描述了生成局部前提c′ 1的过程。在本公开的上下文中，可以针对每个短语w 2和w 3进行类似处理，以便生成相应的局部前提c′ 2和c′ 3。例如，对于第二个短语w 2“won”而言，此时w′ 2＝“lost”，局部前提c′ 2可以表示为“Bob lost the 2020 election”。局部前提的集合可以表示为
Figure PCTCN2022132139-appb-000004
进一步，可以利用训练数据和所确定的多个局部前提，来训练短语验证模型和表述验证模型。
在下文中，首先描述与验证模型相关的数学原理。在本公开的上下文中，验证模型可以在短语等级针对表述提供真实性验证。具体地，短语验证模型可以确定短语的真实性，并且表述验证模型可以确定表述的真实性。根据本公开的一个示例性实现方式，可以建立用于训练短语验证模型和表述验证模型的目标函数。具体地，对于每个短语w i∈W c，可以将短语的真实性表示为三选一(例如，SUP、REF和NEI之一)的隐式变量z i。此时，可以定义
Figure PCTCN2022132139-appb-000005
|W c|表示表述c中的短语的数量。此时表述c的真实性y依赖于隐式变量z。
根据本公开的一个示例性实现方式，提出了逻辑规则约束的概念。将会理解，表述中的各个短语的短语真实性与表述的表述真实性之间存在逻辑约束。可以获取多个短语真实性与表述真实性之间的逻辑规则约束，具体地，1)对于REF标签，如果证据集反对多个短语中的至少一个短语，则表述真实性为REF；2)对于SUP标签，如果证据集支持多个短语中的全部短语，则表述真实性为SUP；3)对于NEI标签，如果 以上两者都不满足，则验证结果不确定。因而，可以基于上述3个约束关系来构建验证模型。具体地，可以定义如下文公式1所示的逻辑规则约束：
Figure PCTCN2022132139-appb-000006
其中c表示表述，E表示用于执行验证过程所基于的证据集，W c表示c中包括的短语的集合，V(c，W c，E)表示三个数值
Figure PCTCN2022132139-appb-000007
(也即，支持、反对和不确定)中的一个。对于W c和E而言，V(c，W c，E)对应于三个预定标签y∈{SUP，REF，NEI}中的一个。
根据本公开的一个示例性实现方式，可以建立隐式变量模型，进一步可以将上文的逻辑规则约束应用于隐式变量模型。对于表述c和获取的证据集E，可以定义目标分布p θ(y|x)：
Figure PCTCN2022132139-appb-000008
在此，p(z|x)表示基于输入x(x＝(c，E))在隐式变量z之上的先验分布，p θ表示在x和z条件下y的概率。在此，假设z i彼此独立，也即p(z|x)＝Πip(z i|x，w i)。对于训练数据中的真值标签y *而言，可以构造如下所示的目标函数：
Figure PCTCN2022132139-appb-000009
理论上，可以使用最大期望(Expectation-maximization，EM)算法来优化模型。然而，在实际操作中由于z的空间巨大导致难以确定p θ(z|y，x)，因而可以基于变分推断算法来确定变分后验分布。具体地，可以最小化负证据下界(negative Evidence Lower BOund，negative ELBO)，并利用公式4来表示与变量相关的目标函数：
Figure PCTCN2022132139-appb-000010
其中q φ(·)表示以y和x为条件的变分后验分布，D KL表示两个分布之间的距离，并且θ和φ分别表示与上文描述的分布p θ和q φ相关的全部参数。进一步，有关其他符号的含义与已有的ELBO算法中的符号含义相同，因而不再赘述。具体地，可以将预先训练的自然语言推理(Natural Language Inference，NLI)模型作为先验分布p(z|x)(其参数是固定的)。在此，NLI模型可以生成在三种类型上的分布：CONTRADICTED、NEUTRAL以及ENTAILMENT，并且三种类型可以分别对应于本公开中的“反对”、“不确定”以及“支持”。
进一步，可以基于逻辑知识蒸馏来引入逻辑规则约束。也即，可以利用公式1示出的逻辑规则约束来更新目标函数，使得基于目标函数训练的验证模型输出的多个短语真实性和表述真实性之间的关系满足逻辑规则约束。根据本公开的一个示例性实现方式，提出了一种知识蒸馏方法，该方法可以采用教师模型和学生模型来构造最终的目标函数。在此，学生模型p θ(y|z，x)是期待优化的目标，可以将变分分布q φ(z|y，x)映射至子空间
Figure PCTCN2022132139-appb-000011
来构造教师模型。在此，由于子空间y z是z的逻辑聚合，因而子空间受到如上文公式1表示的逻辑规则的约束。由此，可以模拟
Figure PCTCN2022132139-appb-000012
的输出，来将逻辑约束指示应用到p θ。具体地，可以基于如下公式5确定蒸馏损失：
Figure PCTCN2022132139-appb-000013
其中
Figure PCTCN2022132139-appb-000014
表示与逻辑规则约束相关的目标函数，D KL表示两个分布之间的距离。进一步，在考虑与训练数据中的变量相关的目标函数和与逻辑规则约束相关的目标函数两者的情况下，可以建立用于联合训练短语验证模型和表述验证模型的目标函数。在此，目标函数使得表述真实性与标签之间的关系满足预定条件，并且可以使得短语真实性与表 述真实性之间的关系满足逻辑规则约束。具体地，可以基于如下公式6来确定最终的目标函数：
Figure PCTCN2022132139-appb-000015
其中
Figure PCTCN2022132139-appb-000016
表示最终目标函数，
Figure PCTCN2022132139-appb-000017
表示训练数据中的变量相关的目标函数，
Figure PCTCN2022132139-appb-000018
表示逻辑规则约束相关的目标函数，并且1-λ和λ分别表示用于两种类型的目标函数的权重。利用本公开的示例性实现方式，可以将逻辑规则约束应用于构建目标函数的过程，进而使得目标函数即考虑了训练数据集中的训练数据，又考虑了短语真实性和表述真实性之间的逻辑规则约束。以此方式，可以使得验证模型可以在短语的等级对于表述的验证结果提供解释。
根据本公开的一个示例性实现方式，在训练和隐式变量规格化过程中，可以使用软逻辑方案。具体地，可以基于如下公式7执行聚合操作：
Figure PCTCN2022132139-appb-000019
其中
Figure PCTCN2022132139-appb-000020
表示与规则规格化的子空间相关的信息，其中y z可以分别等于SUP、REF或者NEI。
Figure PCTCN2022132139-appb-000021
并且
Figure PCTCN2022132139-appb-000022
以此方式，可以为上文公式5所示的
Figure PCTCN2022132139-appb-000023
具体化，进而在满足逻辑规则约束的情况下确定短语的真实性。
根据本公开的一个示例性实现方式，可以针对训练数据进行编码，继而分别利用短语验证模型和表述验证模型来来参数化p θ(y|z，x)和变分分布q φ(z|y，x)，以便基于变分EM算法来迭代地进行优化。在下文中，将参见图5描述有关训练短语验证模型和表述验证模型的更多细节。
图5示出了根据本公开的一些实现方式的用于训练短语验证模型和表述验证模型的框图500。如图5所示，可以基于训练数据112来确定用于训练短语验证模型510和表述验证模型520的输入数据。此时，短 语验证模型510的输入512可以包括表述真实性514、局部编码516和全局编码518。表述验证模型520的输入522可以包括短语真实性524、全局编码518和上下文编码528。在下文中，将以具体公式示出如何计算上述参数。
在训练阶段，可以基于训练数据112中的指示表述真实性的标签(也即真值数据)来确定表述真实性514。进一步，对于给定的c、E和针对每个短语的局部前提，可以利用预训练的语言模型(PLM)来计算文本表示。例如，可以将表述c和各个局部前提c′ i进行连接，以获得相关于局部编码516的文本表示
Figure PCTCN2022132139-appb-000024
继而，可以利用编码器来将该文本表示映射至局部编码516(即，
Figure PCTCN2022132139-appb-000025
表示编码空间)。根据本公开的一个示例性实现方式，可以使用目前已知的和/或将在未来开发的多种编码器来执行上述编码过程。
类似地，可以将表述c和证据集E进行连接以得到与全局编码518相关联的文本表示x global＝(c，E)。继而，可以将该文本表示进行编码以获得全局编码518(即，
Figure PCTCN2022132139-appb-000026
)。进一步，可以应用自选择模型以便获得编码中的最为重要的部分。
将会理解，多个短语中可能会存在可疑短语(也即，影响表述真实性的短语)，因而，可以基于如下经验来设计可疑短语注意力模型：有效的局部前提(也即，有益于输出符合实际情况的表述真实性的局部前提)在语义上应当接近证据集中的证据。由此，可以基于
Figure PCTCN2022132139-appb-000027
和h global之间的相似性来确定每个短语w i的重要性α i。具体地，可以基于如下公式8来确定上下文编码h local：
Figure PCTCN2022132139-appb-000028
其中
Figure PCTCN2022132139-appb-000029
表示参数，并且σ表示softmax函数。在计算上述参数之后，可以将p θ(·)和q φ(·)两者应用于两层多层感知模型。如图5所示，q φ(z i|y，x)的输入可以包括以下三个参数的连接：表述真实性514(y *)、局部编码516
Figure PCTCN2022132139-appb-000030
以及全局编码518(h global)，并且输出514可 以包括z i(也即，每个短语的短语真实性的概率)。在此，
Figure PCTCN2022132139-appb-000031
进一步，p θ(y|z，x)的输入522可以包括以下三个参数的连接：短语真实性524(z＝(z 1，z 2，...，z max)，此时max表示填充的最大长度)、全局编码518(h global)、以及上下文编码528(h local)，并且输出524可以包括y(也即，表述真实性)。
根据本公开的一个示例性实现方式，在训练过程中可以利用上文公式6所示的目标函数来将q φ(·)和p θ(·)两者进行联合优化。也即，利用目标函数来联合训练短语验证模型510和表述验证模型520。可以使用目前已知的和/或将在未来开发的多种优化技术，来执行迭代优化。例如，可以使用Gumbel表示算法来进行离散argmax运算，来执行优化。
将会理解，公式6中表示的θ和φ分别表示与上文描述的分布p θ和q φ相关的全部参数。在具体训练过程中，可以分别将基于训练数据集确定的各个参数的具体数值(例如，y *、
Figure PCTCN2022132139-appb-000032
h global、z i、h local，等等)带入公式6，进而进行训练过程。此时，目标函数中与短语验证模型510相关联的部分可以涉及标签、局部编码和全局编码，并且目标函数中与表述验证模型520相关联的部分可以涉及短语真实性、全局编码和上下文编码。通过设置统一的目标函数，一方面可以考虑两个模型之间的内在逻辑依赖关系，另一方面还可以降低训练涉及的多种开销。
可以按照类似的方式来逐一处理训练数据集110中的每个训练数据，以便获得用于执行训练的各个参数。可以利用获得的参数来迭代地执行训练过程，直到各个模型满足预定的停止条件。此时，训练后的表述验证模型520可以输出表述的表述真实性，并且训练后的短语验证模型510可以输出各个短语的短语真实性，在此短语真实性可以对表述真实性提供解释。
根据本公开的一个示例性实现方式，可以以概率分布方式表示真实性。例如，当按照SUP、REF和NEI的顺序表示真实性的概率分布时，假设表述真实性为(0.8，0.1，0.1)，此时与SUP相关联的概率0.8为最大值，则此时表述真实性为“SUP”，也即支持。根据本公开的一个示例性实现方式，可以以类似的方式来表示短语真实性，在下文中将不 再赘述。
上文已经描述了用于训练验证模型的过程，此时的验证模型不但可以验证表述的真假，还可以以更为精细的粒度处理表述中的各个短语，并且验证每个短语的短语真实性。以此方式，短语真实性可以表示相应短语对于表述的最终真实性的贡献，因而可以为最终验证结果提供解释。
模型应用过程
上文已经描述对验证模型130的训练，训练后的验证模型130’可以被提供到如图1所示的模型应用系统152中使用，以用于对输入数据140进行处理。具体地，在已经完成模型训练阶段之后，可以使用已经训练好的、具有训练后的参数值的验证模型130’来处理接收到的输入。返回图1描述有关模型应用过程的更多信息。可以向训练后的验证模型130’输入输入数据140。在此，输入数据140可以包括将要被验证的表述142以及用于为验证提供支持的证据集144。充分训练之后的验证模型130’可以接收输入数据140，并且基于证据集144中的一个或多个证据来验证表述142的真实性。
在下文中，参见图6描述有关推理过程的更多细节。图6示出了根据本公开的一些实现方式的用于基于短语验证模型和表述验证模型来执行推理的框图600。如图6所示，输入数据140例如可以包括表述142和证据集144。在此，表述142表示将被验证的内容，以及证据集144包括用于支持验证表述的真实性的至少一个证据。在推理阶段，验证模型中的各个模块以类似于训练过程的方式操作。例如，可以基于语法分析，将表述142划分为多个短语，并且确定与每个短语相关联的参数。
进一步，可以利用短语验证模型510，基于证据集144来分别确定多个短语的多个短语真实性。可以按照上文描述的方法来分别确定输入610和620中的各个参数的具体数值。可以利用表述验证模型520，基于证据集144和多个短语真实性(也即，输出612)来确定表述的表述真实性，在此多个短语真实性对表述真实性提供解释。将会理解，在初始阶段，短语真实性验证模型510可以将多个短语真实性分别设置为多 个预定初始值，例如(0.4，0.3，0.3)或者其他数值。此时，图6所示的输出612的取值即为(0.4，0.3，0.3)。该取值可以作为表述真实性模型520的输入，并且与从输入数据140确定的全局编码h global和上下文编码h global一起，用于确定表述真实性(也即输出622)。此时，第一轮次的推理操作结束。
根据本公开的一个示例性实现方式，可以在多个轮次中迭代地执行上文描述的推理过程。具体地，在第二轮次中可以将输出622作为短语真实性模型510的输入y，并且将其与从输入数据140确定的局部编码
Figure PCTCN2022132139-appb-000033
和全局编码h global一起，以便在输出612处获取新的短语真实性。继而，可以将新的短语真实性输入至表述真实性模型520，以便在输出622处获得新的表述真实性。将会理解，上文仅以第一轮次和第二轮次作为示例来描述了迭代过程。根据本公开的一个示例性实现方式，在第二轮次之后还可以执行一个或多个后续的轮次，直到满足预定的停止条件。
根据本公开的一个示例性实现方式，停止条件例如可以指定：当前后两个轮次的输出结果之间的差异小于预定阈值时停止。又例如，停止条件还可以指定：当两个或者更多连续轮次的概率分布所指示的真实性不再变化时停止。又例如，停止条件还可以指定：当达到预定轮次数量时停止迭代过程。利用本公开的示例性实现方式，验证模型不但可以针对整体表述输出表述真实性，还可以以更为精细的粒度来处理表述中的各个短语。进一步，可以输出各个短语的短语真实性，以便为表述真实性提供解释。
上文已经描述了利用验证模型来处理输入数据的概要。在下文中，将分别参见图7A和图7B利用验证模型处理输入数据的具体示例。图7A示出了根据本公开的一些实现方式的用于确定表述的真实性的框图700A。如图7A所示，可以接收表述142和证据集144。可以将表述142划分为多个短语712A、714A和716A，进一步，可以分别确定与各个短语相关联局部前提。具体地，局部前提730A示出了与短语712A、714A和716A相关联的局部前提。
如图7A所示，前提1表示与短语712A相关联的局部前提，并且短语712A的短语真实性为“支持”(如最大值732A所示)。前提2表示与短语714A相关联的局部前提，并且短语714A的短语真实性为“反对”(如最大值734A所示)。前提3表示与短语716A相关联的局部前提，并且短语716A的短语真实性为“支持”(如最大值736A所示)。进一步，如最大值742A所示，预测740A示出了真实性验证的预测结果“反对”。该示例示出了预测结果与真值相一致的情况，可见，短语真实性可以提供更加精细粒度的预测结果，并且可以为最终预测结果提供解释。
图7B示出了根据本公开的一些实现方式的用于确定表述的真实性的框图700B。如图7B所示，可以接收表述710B和证据集720B。可以将表述710B划分为多个短语712B和714B。进一步，可以分别确定与各个短语相关联局部前提。具体地，局部前提730B示出了与短语712B和714B相关联的局部前提。
如图7B所示，前提1表示与短语712B相关联的局部前提，并且短语712B的短语真实性为“支持”(如最大值732B所示)。前提2表示与短语714B相关联的局部前提，并且短语714B的短语真实性为“反对”(如最大值734B所示)。进一步，如最大值742B所示，预测740B示出了真实性验证的预测结果“反对”。该示例示出了预测结果与真值(“不确定”)不一致的情况。此时，通过分析各个短语真实性可以发现，短语714B的短语真实度“反对”导致了预测结果错误。以此方式，短语真实度可以在一定程度上反映错误原因，进而有益于进一步优化验证模型。
示例过程
图8示出了根据本公开的一些实现方式的用于验证表述的真实性的方法800的流程图。具体地，在框810处，获取包括表述、证据集和标签的训练数据，表述表示被验证的内容，证据集包括用于支持验证表述的真实性的至少一个证据，以及标签表示基于证据集来验证表述的真实 性的结果。在框820处，基于对表述的语法分析，将表述划分为多个短语。在框830处，基于训练数据和多个短语来训练短语验证模型，以使得短语验证模型基于证据集来分别确定多个短语的多个短语真实性。
根据本公开的一个示例性实现方式，为了训练短语验证模型，可以逐一处理多个短语中的短语。具体地，可以基于证据集确定与短语相关联的局部前提，局部前提表示用于验证短语的真实性的知识。可以基于局部前提和训练数据，训练短语真实性验证模型。
根据本公开的一个示例性实现方式，为了确定局部前提，可以基于证据集确定与短语相匹配的证据短语，证据短语表示用于确定短语的真实性的知识点。进一步，可以利用证据短语替换表述中的短语，以便生成局部前提。
根据本公开的一个示例性实现方式，为了确定证据短语，可以基于表述生成与短语相关联的探测问题。继而，在证据集中检索探测问题的答案，以作为证据短语。
根据本公开的一个示例性实现方式，可以从表述中移除短语，以将与表述相关联的完形填空语句作为探测问题。
根据本公开的一个示例性实现方式，可以基于短语在表述中的位置，将用于查询短语的疑问句作为探测问题。
根据本公开的一个示例性实现方式，标签包括以下任一项：“支持”、“反对”以及“不确定”。根据本公开的一个示例性实现方式，为了检索答案，可以建立阅读理解模型，阅读理解模型使得答案与探测问题的真实答案相一致。可以获取包括表述、证据集和标签的另一训练数据。如果另一训练数据的标签为“支持”，可以利用另一训练数据来训练阅读理解模型。
在框840处，基于训练数据和多个短语来训练表述验证模型，以使得表述验证模型基于证据集来确定表述的表述真实性，其中多个短语真实性对表述真实性提供解释。
根据本公开的一个示例性实现方式，可以建立用于联合训练短语验证模型和表述验证模型的目标函数，目标函数使得表述真实性与标签之 间的关系满足预定条件。根据本公开的一个示例性实现方式，可以利用目标函数来联合训练短语验证模型和表述验证模型。
根据本公开的一个示例性实现方式，可以获取多个短语真实性与表述真实性之间的逻辑规则约束。可以基于逻辑规则约束更新目标函数，目标函数使得多个短语真实性和表述真实性之间的关系满足逻辑规则约束。
根据本公开的一个示例性实现方式，为了建立目标函数，可以基于表述和多个局部前提，确定分别与多个短语相关联的多个局部编码。可以基于表述和证据集确定表述的全局编码。可以利用标签、多个局部编码和全局编码作为参数，来确定目标函数。
根据本公开的一个示例性实现方式，为了建立目标函数，可以基于多个局部前提与表述的比较，分别确定多个短语的多个重要性。可以基于多个重要性和多个局部编码，确定表述的上下文编码。可以利用多个短语真实性、上下文编码和全局编码作为参数。
图9示出了根据本公开的一些实现方式的用于验证表述的真实性的方法900的流程图。在框910处，可以获取表述和与表述相关联的证据集，表述表示将被验证的内容，以及证据集包括用于支持验证表述的真实性的至少一个证据。在框920处，可以基于对表述的语法分析，将表述划分为多个短语。在框930处，可以利用短语验证模型，基于证据集来分别确定多个短语的多个短语真实性。
根据本公开的一个示例性实现方式，在初始阶段，可以将多个短语真实性分别设置为多个预定初始值。根据本公开的一个示例性实现方式，在初始阶段之后的后续阶段，基于证据集和表述真实性来确定多个短语真实性。
在框940处，可以利用表述验证模型，基于证据集和多个短语来确定表述的表述真实性，其中多个短语真实性对表述真实性提供解释。
根据本公开的一个示例性实现方式，可以迭代地执行方法900，直到多个短语真实性和表述真实性之间的关系满足预定停止条件。
示例装置和设备
图10A示出了根据本公开的一些实现方式的用于验证表述的真实性的装置1000A的框图。如图10A所示，装置1000A包括获取模块1010A、划分模块1020A、短语验证模块1030A和表述验证模块1040A。
根据本公开的一个示例性实现方式，获取模块1010A，配置用于获取包括表述、证据集和标签的训练数据，表述表示被验证的内容，证据集包括用于支持验证表述的真实性的至少一个证据，以及标签表示基于证据集来验证表述的真实性的结果；划分模块1020A，配置用于基于对表述的语法分析，将表述划分为多个短语；短语验证模块1030A，配置用于基于训练数据和多个短语来训练短语验证模型，以使得短语验证模型基于证据集来分别确定多个短语的多个短语真实性；以及表述验证模块1040A，配置用于基于训练数据和多个短语来训练表述验证模型，以使得表述验证模型基于证据集来确定表述的表述真实性，其中多个短语真实性对表述真实性提供解释。
根据本公开的一个示例性实现方式，短语验证模块进一步配置用于：针对多个短语中的短语，基于证据集确定与短语相关联的局部前提，局部前提表示用于验证短语的真实性的知识；以及基于局部前提和训练数据，训练短语真实性验证模型。
根据本公开的一个示例性实现方式，短语验证模块进一步配置用于：基于证据集确定与短语相匹配的证据短语，证据短语表示用于确定短语的真实性的知识点；以及利用证据短语替换表述中的短语，以便生成局部前提。
根据本公开的一个示例性实现方式，短语验证模块进一步配置用于：基于表述生成与短语相关联的探测问题；以及在证据集中检索探测问题的答案，以作为证据短语。
根据本公开的一个示例性实现方式，短语验证模块进一步配置用于：从表述中移除短语，以将与表述相关联的完形填空语句作为探测问题；以及基于短语在表述中的位置，将用于查询短语的疑问句作为探测问题。
根据本公开的一个示例性实现方式，标签包括以下任一项：“支持”、 “反对”以及“不确定”。短语验证模块进一步配置用于：建立阅读理解模型，阅读理解模型使得答案与探测问题的真实答案相一致；获取包括表述、证据集和标签的另一训练数据；以及响应于另一训练数据的标签为“支持”，利用另一训练数据来训练阅读理解模型。
根据本公开的一个示例性实现方式，进一步包括：建立模块，配置用于建立用于联合训练短语验证模型和表述验证模型的目标函数，目标函数使得表述真实性与标签之间的关系满足预定条件。短语验证模块和表述验证模块进一步配置用于：利用目标函数来联合训练短语验证模型和表述验证模型。
根据本公开的一个示例性实现方式，建立模块进一步配置用于：获取多个短语真实性与表述真实性之间的逻辑规则约束；以及基于逻辑规则约束更新目标函数，目标函数使得多个短语真实性和表述真实性之间的关系满足逻辑规则约束。
根据本公开的一个示例性实现方式，建立模块进一步配置用于：基于表述和多个局部前提，确定分别与多个短语相关联的多个局部编码；基于表述和证据集确定表述的全局编码；以及利用标签、多个局部编码和全局编码作为参数，来确定目标函数。
根据本公开的一个示例性实现方式，建立模块进一步配置用于：基于多个局部前提与表述的比较，分别确定多个短语的多个重要性；基于多个重要性和多个局部编码，确定表述的上下文编码；利用多个短语真实性、上下文编码和全局编码作为参数，来确定目标函数。
图10B示出了根据本公开的一些实现方式的用于验证表述的真实性的装置1000B的框图。如图10B所示，装置1000B包括获取模块1010B、划分模块1020B、短语验证模块1030B和表述验证模块1040B。
根据本公开的一个示例性实现方式，获取模块1010B，配置用于获取包括表述、证据集和标签的训练数据，所述表述表示被验证的内容，所述证据集包括用于支持验证所述表述的所述真实性的至少一个证据，以及所述标签表示基于所述证据集来验证所述表述的所述真实性的结果；划分模块1020B，配置用于基于对所述表述的语法分析，将所述表 述划分为多个短语；短语验证模块1030B，配置用于基于所述训练数据和所述多个短语来训练短语验证模型，以使得所述短语验证模型基于所述证据集来分别确定所述多个短语的多个短语真实性；以及表述验证模块1040B，配置用于基于所述训练数据和所述多个短语来训练表述验证模型，以使得所述表述验证模型基于所述证据集来确定所述表述的表述真实性，其中所述多个短语真实性对所述表述真实性提供解释。
根据本公开的一个示例性实现方式，短语验证模块1030B进一步配置用于在初始阶段，将多个短语真实性分别设置为多个预定初始值；以及在初始阶段之后的后续阶段，基于证据集和表述真实性来确定多个短语真实性。
根据本公开的一个示例性实现方式，短语验证模块1030B和表述验证模块1040B被迭代地调用，直到多个短语真实性和表述真实性之间的关系满足预定停止条件。
图11示出了能够实施本公开的多个实现方式的设备1100的框图。应当理解，图11所示出的计算设备1100仅仅是示例性的，而不应当构成对本文所描述的实现方式的功能和范围的任何限制。图11所示出的计算设备1100可以用于实现如图1所示的模型训练系统150，也可以实现用于如图1所示的模型应用系统152。
如图11所示，计算设备1100是通用计算设备的形式。计算设备1100的组件可以包括但不限于一个或多个处理器或处理单元1110、存储器1120、存储设备1130、一个或多个通信单元1140、一个或多个输入设备1150以及一个或多个输出设备1160。处理单元1110可以是实际或虚拟处理器并且能够根据存储器1120中存储的程序来执行各种处理。在多处理器系统中，多个处理单元并行执行计算机可执行指令，以提高计算设备1100的并行处理能力。
计算设备1100通常包括多个计算机存储介质。这样的介质可以是计算设备1100可访问的任何可以获得的介质，包括但不限于易失性和非易失性介质、可拆卸和不可拆卸介质。存储器1120可以是易失性存储器(例如寄存器、高速缓存、随机访问存储器(RAM))、非易失性 存储器(例如，只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、闪存)或它们的某种组合。存储设备1130可以是可拆卸或不可拆卸的介质，并且可以包括机器可读介质，诸如闪存驱动、磁盘或者任何其他介质，其可以能够用于存储信息和/或数据(例如用于训练的训练数据)并且可以在计算设备1100内被访问。
计算设备1100可以进一步包括另外的可拆卸/不可拆卸、易失性/非易失性存储介质。尽管未在图11中示出，可以提供用于从可拆卸、非易失性磁盘(例如“软盘”)进行读取或写入的磁盘驱动和用于从可拆卸、非易失性光盘进行读取或写入的光盘驱动。在这些情况中，每个驱动可以由一个或多个数据介质接口被连接至总线(未示出)。存储器1120可以包括计算机程序产品1125，其具有一个或多个程序模块，这些程序模块被配置为执行本公开的各种实现方式的各种方法或动作。
通信单元1140实现通过通信介质与其他计算设备进行通信。附加地，计算设备1100的组件的功能可以以单个计算集群或多个计算机器来实现，这些计算机器能够通过通信连接进行通信。因此，计算设备1100可以使用与一个或多个其他服务器、网络个人计算机(PC)或者另一个网络节点的逻辑连接来在联网环境中进行操作。
输入设备1150可以是一个或多个输入设备，例如鼠标、键盘、追踪球等。输出设备1160可以是一个或多个输出设备，例如显示器、扬声器、打印机等。计算设备1100还可以根据需要通过通信单元1140与一个或多个外部设备(未示出)进行通信，外部设备诸如存储设备、显示设备等，与一个或多个使得用户与计算设备1100交互的设备进行通信，或者与使得计算设备1100与一个或多个其他计算设备通信的任何设备(例如，网卡、调制解调器等)进行通信。这样的通信可以经由输入/输出(I/O)接口(未示出)来执行。
根据本公开的示例性实现方式，提供了一种计算机可读存储介质，其上存储有计算机可执行指令，其中计算机可执行指令被处理器执行以实现上文描述的方法。根据本公开的示例性实现方式，还提供了一种计算机程序产品，计算机程序产品被有形地存储在非瞬态计算机可读介质 上并且包括计算机可执行指令，而计算机可执行指令被处理器执行以实现上文描述的方法。根据本公开的示例性实现方式，提供了一种计算机程序产品，其上存储有计算机程序，所述程序被处理器执行时实现上文描述的方法。
这里参照根据本公开实现的方法、装置、设备和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理单元，从而生产出一种机器，使得这些指令在通过计算机或其他可编程数据处理装置的处理单元执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
可以把计算机可读程序指令加载到计算机、其他可编程数据处理装置、或其他设备上，使得在计算机、其他可编程数据处理装置或其他设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其他可编程数据处理装置、或其他设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实现的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/ 或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
以上已经描述了本公开的各实现，上述说明是示例性的，并非穷尽性的，并且也不限于所公开的各实现。在不偏离所说明的各实现的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实现的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其他普通技术人员能理解本文公开的各个实现方式。

Claims (32)

  1. 一种用于验证表述的真实性的方法，包括：
    获取包括表述、证据集和标签的训练数据，所述表述表示被验证的内容，所述证据集包括用于支持验证所述表述的所述真实性的至少一个证据，以及所述标签表示基于所述证据集来验证所述表述的所述真实性的结果；
    基于对所述表述的语法分析，将所述表述划分为多个短语；
    基于所述训练数据和所述多个短语来训练短语验证模型，以使得所述短语验证模型基于所述证据集来分别确定所述多个短语的多个短语真实性；以及
    基于所述训练数据和所述多个短语来训练表述验证模型，以使得所述表述验证模型基于所述证据集来确定所述表述的表述真实性，其中所述多个短语真实性对所述表述真实性提供解释。
  2. 根据权利要求1所述的方法，其中训练所述短语验证模型包括：针对所述多个短语中的短语，
    基于所述证据集确定与所述短语相关联的局部前提，所述局部前提表示用于验证所述短语的真实性的知识；以及
    基于所述局部前提和所述训练数据，训练所述短语真实性验证模型。
  3. 根据权利要求2所述的方法，其中确定所述局部前提包括：
    基于所述证据集确定与所述短语相匹配的证据短语，所述证据短语表示用于确定所述短语的所述真实性的知识点；以及
    利用所述证据短语替换所述表述中的所述短语，以便生成所述局部前提。
  4. 根据权利要求3所述的方法，其中确定所述证据短语包括：
    基于所述表述生成与所述短语相关联的探测问题；以及
    在所述证据集中检索所述探测问题的答案，以作为所述证据短语。
  5. 根据权利要求4所述的方法，其中生成所述探测问题包括以下至少任一项：
    从所述表述中移除所述短语，以将与所述表述相关联的完形填空语句作为所述探测问题；以及
    基于所述短语在所述表述中的位置，将用于查询所述短语的疑问句作为所述探测问题。
  6. 根据权利要求4所述的方法，其中所述标签包括以下任一项：“支持”、“反对”以及“不确定”，其中检索所述答案包括：
    建立阅读理解模型，所述阅读理解模型使得所述答案与所述探测问题的真实答案相一致；
    获取包括表述、证据集和标签的另一训练数据；以及
    响应于所述另一训练数据的标签为“支持”，利用所述另一训练数据来训练所述阅读理解模型。
  7. 根据权利要求2所述的方法，进一步包括：
    建立用于联合训练所述短语验证模型和所述表述验证模型的目标函数，所述目标函数使得所述表述真实性与所述标签之间的关系满足预定条件；以及
    训练所述短语验证模型和所述表述验证模型包括：利用所述目标函数来联合训练所述短语验证模型和所述表述验证模型。
  8. 根据权利要求7所述的方法，进一步包括：
    获取所述多个短语真实性与所述表述真实性之间的逻辑规则约束；以及
    基于所述逻辑规则约束更新所述目标函数，所述目标函数使得所述多个短语真实性和所述表述真实性之间的关系满足所述逻辑规则约束。
  9. 根据权利要求8所述的方法，其中建立所述目标函数包括：
    基于所述表述和所述多个局部前提，确定分别与所述多个短语相关联的多个局部编码；
    基于所述表述和所述证据集确定所述表述的全局编码；以及
    利用所述标签、所述多个局部编码和所述全局编码作为参数，来确定所述目标函数。
  10. 根据权利要求9所述的方法，其中建立所述目标函数进一步包括：
    基于所述多个局部前提与所述表述的比较，分别确定所述多个短语的多个重要性；
    基于所述多个重要性和所述多个局部编码，确定所述表述的上下文编码；
    利用所述多个短语真实性、所述上下文编码和所述全局编码作为参数，来确定所述目标函数。
  11. 一种电子设备，包括：
    至少一个处理单元；以及
    至少一个存储器，所述至少一个存储器被耦合到所述至少一个处理单元并且存储用于由所述至少一个处理单元执行的指令，所述指令在由所述至少一个处理单元执行时使所述设备执行以下动作：
    获取包括表述、证据集和标签的训练数据，所述表述表示被验证的内容，所述证据集包括用于支持验证所述表述的所述真实性的至少一个证据，以及所述标签表示基于所述证据集来验证所述表述的所述真实性的结果；
    基于对所述表述的语法分析，将所述表述划分为多个短语；
    基于所述训练数据和所述多个短语来训练短语验证模型，以使得所述短语验证模型基于所述证据集来分别确定所述多个短语的多个短语真实性；以及
    基于所述训练数据和所述多个短语来训练表述验证模型，以使得所述表述验证模型基于所述证据集来确定所述表述的表述真实性，其中所述多个短语真实性对所述表述真实性提供解释。
  12. 根据权利要求11所述的设备，其中训练所述短语验证模型包括：针对所述多个短语中的短语，
    基于所述证据集确定与所述短语相关联的局部前提，所述局部前提表示用于验证所述短语的真实性的知识；以及
    基于所述局部前提和所述训练数据，训练所述短语真实性验证模型。
  13. 根据权利要求12所述的设备，其中确定所述局部前提包括：
    基于所述证据集确定与所述短语相匹配的证据短语，所述证据短语表示用于确定所述短语的所述真实性的知识点；以及
    利用所述证据短语替换所述表述中的所述短语，以便生成所述局部前提。
  14. 根据权利要求13所述的设备，其中确定所述证据短语包括：
    基于所述表述生成与所述短语相关联的探测问题；以及
    在所述证据集中检索所述探测问题的答案，以作为所述证据短语。
  15. 根据权利要求14所述的设备，其中生成所述探测问题包括以下至少任一项：
    从所述表述中移除所述短语，以将与所述表述相关联的完形填空语句作为所述探测问题；以及
    基于所述短语在所述表述中的位置，将用于查询所述短语的疑问句作为所述探测问题。
  16. 根据权利要求14所述的设备，其中所述标签包括以下任一项：“支持”、“反对”以及“不确定”，其中检索所述答案包括：
    建立阅读理解模型，所述阅读理解模型使得所述答案与所述探测问题的真实答案相一致；
    获取包括表述、证据集和标签的另一训练数据；以及
    响应于所述另一训练数据的标签为“支持”，利用所述另一训练数据来训练所述阅读理解模型。
  17. 根据权利要求12所述的设备，进一步包括：
    建立用于联合训练所述短语验证模型和所述表述验证模型的目标函数，所述目标函数使得所述表述真实性与所述标签之间的关系满足预定条件；以及
    训练所述短语验证模型和所述表述验证模型包括：利用所述目标函数来联合训练所述短语验证模型和所述表述验证模型。
  18. 根据权利要求17所述的设备，进一步包括：
    获取所述多个短语真实性与所述表述真实性之间的逻辑规则约束；以及
    基于所述逻辑规则约束更新所述目标函数，所述目标函数使得所述多个短语真实性和所述表述真实性之间的关系满足所述逻辑规则约束。
  19. 根据权利要求18所述的设备，其中建立所述目标函数包括：
    基于所述表述和所述多个局部前提，确定分别与所述多个短语相关联 的多个局部编码；
    基于所述表述和所述证据集确定所述表述的全局编码；以及
    利用所述标签、所述多个局部编码和所述全局编码作为参数，来确定所述目标函数。
  20. 根据权利要求19所述的设备，其中建立所述目标函数进一步包括：
    基于所述多个局部前提与所述表述的比较，分别确定所述多个短语的多个重要性；
    基于所述多个重要性和所述多个局部编码，确定所述表述的上下文编码；
    利用所述多个短语真实性、所述上下文编码和所述全局编码作为参数，来确定所述目标函数。
  21. 一种用于验证表述的真实性的方法，包括：
    获取表述和与所述表述相关联的证据集，所述表述表示将被验证的内容，以及所述证据集包括用于支持验证所述表述的所述真实性的至少一个证据；
    基于对所述表述的语法分析，将所述表述划分为多个短语；
    利用短语验证模型，基于所述证据集来分别确定所述多个短语的多个短语真实性；以及
    利用表述验证模型，基于所述证据集和所述多个短语来确定所述表述的表述真实性，其中所述多个短语真实性对所述表述真实性提供解释。
  22. 根据权利要求21所述的方法，其中确定所述多个短语真实性包括以下至少任一项：
    在初始阶段，将所述多个短语真实性分别设置为多个预定初始值；以及
    在所述初始阶段之后的后续阶段，基于所述证据集和所述表述真实性来确定所述多个短语真实性。
  23. 根据权利要求22所述的方法，其中所述方法被迭代地执行，直到所述多个短语真实性和所述表述真实性之间的关系满足预定停止条件。
  24. 一种电子设备，包括：
    至少一个处理单元；以及
    至少一个存储器，所述至少一个存储器被耦合到所述至少一个处理单元并且存储用于由所述至少一个处理单元执行的指令，所述指令在由所述至少一个处理单元执行时使所述设备执行以下动作：
    获取表述和与所述表述相关联的证据集，所述表述表示将被验证的内容，以及所述证据集包括用于支持验证所述表述的所述真实性的至少一个证据；
    基于对所述表述的语法分析，将所述表述划分为多个短语；
    利用短语验证模型，基于所述证据集来分别确定所述多个短语的多个短语真实性；以及
    利用表述验证模型，基于所述证据集和所述多个短语来确定所述表述的表述真实性，其中所述多个短语真实性对所述表述真实性提供解释。
  25. 根据权利要求24所述的设备，其中确定所述多个短语真实性包括以下至少任一项：
    在初始阶段，将所述多个短语真实性分别设置为多个预定初始值；以及
    在所述初始阶段之后的后续阶段，基于所述证据集和所述表述真实性来确定所述多个短语真实性。
  26. 根据权利要求25所述的设备，其中所述方法被迭代地执行，直到所述多个短语真实性和所述表述真实性之间的关系满足预定停止条件。
  27. 一种用于验证表述的真实性的装置，包括：
    获取模块，配置用于获取包括表述、证据集和标签的训练数据，所述表述表示被验证的内容，所述证据集包括用于支持验证所述表述的所述真实性的至少一个证据，以及所述标签表示基于所述证据集来验证所述表述的所述真实性的结果；
    划分模块，配置用于基于对所述表述的语法分析，将所述表述划分为多个短语；
    短语验证模块，配置用于基于所述训练数据和所述多个短语来训练短语验证模型，以使得所述短语验证模型基于所述证据集来分别确定所述多 个短语的多个短语真实性；以及
    表述验证模块，配置用于基于所述训练数据和所述多个短语来训练表述验证模型，以使得所述表述验证模型基于所述证据集来确定所述表述的表述真实性，其中所述多个短语真实性对所述表述真实性提供解释。
  28. 一种用于验证表述的真实性的装置，包括：
    获取模块，配置用于获取表述和与所述表述相关联的证据集，所述表述表示将被验证的内容，以及所述证据集包括用于支持验证所述表述的所述真实性的至少一个证据；
    划分模块，配置用于基于对所述表述的语法分析，将所述表述划分为多个短语；
    短语验证模块，配置用于利用短语验证模型，基于所述证据集来分别确定所述多个短语的多个短语真实性；以及
    表述验证模块，配置用于利用表述验证模型，基于所述证据集和所述多个短语来确定所述表述的表述真实性，其中所述多个短语真实性对所述表述真实性提供解释。
  29. 一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现根据权利要求1至10中任一项所述的方法。
  30. 一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现根据权利要求21至23中任一项所述的方法。
  31. 一种计算机程序产品，其上存储有计算机程序，所述程序被处理器执行时实现根据权利要求1至10中任一项所述的方法。
  32. 一种计算机程序产品，其上存储有计算机程序，所述程序被处理器执行时实现根据权利要求21至23中任一项所述的方法。
PCT/CN2022/132139 2021-11-16 2022-11-16 用于验证表述的真实性的方法、设备、装置和介质 WO2023088278A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111356625.8A CN114065741B (zh) 2021-11-16 2021-11-16 用于验证表述的真实性的方法、设备、装置和介质
CN202111356625.8 2021-11-16

Publications (1)

Publication Number Publication Date
WO2023088278A1 true WO2023088278A1 (zh) 2023-05-25

Family

ID=80272972

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/132139 WO2023088278A1 (zh) 2021-11-16 2022-11-16 用于验证表述的真实性的方法、设备、装置和介质

Country Status (2)

Country Link
CN (1) CN114065741B (zh)
WO (1) WO2023088278A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114065741B (zh) * 2021-11-16 2023-08-11 北京有竹居网络技术有限公司 用于验证表述的真实性的方法、设备、装置和介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160078018A1 (en) * 2014-09-17 2016-03-17 International Business Machines Corporation Method for Identifying Verifiable Statements in Text
CN110516697A (zh) * 2019-07-15 2019-11-29 清华大学 基于证据图聚合与推理的声明验证方法及系统
CN112396185A (zh) * 2021-01-21 2021-02-23 中国人民解放军国防科技大学 一种事实验证方法、系统、计算机设备和存储介质
CN114065741A (zh) * 2021-11-16 2022-02-18 北京有竹居网络技术有限公司 用于验证表述的真实性的方法、设备、装置和介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI645303B (zh) * 2016-12-21 2018-12-21 財團法人工業技術研究院 字串驗證方法、字串擴充方法與驗證模型訓練方法
CN109635110A (zh) * 2018-11-30 2019-04-16 北京百度网讯科技有限公司 数据处理方法、装置、设备以及计算机可读存储介质
CN112100351A (zh) * 2020-09-11 2020-12-18 陕西师范大学 一种通过问题生成数据集构建智能问答系统的方法及设备
CN112069321B (zh) * 2020-11-11 2021-02-12 震坤行网络技术(南京)有限公司 用于文本层级分类的方法、电子设备和存储介质
CN112163574A (zh) * 2020-11-23 2021-01-01 南京航天工业科技有限公司 一种基于深度残差网络的etc干扰信号发射机的识别方法及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160078018A1 (en) * 2014-09-17 2016-03-17 International Business Machines Corporation Method for Identifying Verifiable Statements in Text
CN110516697A (zh) * 2019-07-15 2019-11-29 清华大学 基于证据图聚合与推理的声明验证方法及系统
CN112396185A (zh) * 2021-01-21 2021-02-23 中国人民解放军国防科技大学 一种事实验证方法、系统、计算机设备和存储介质
CN114065741A (zh) * 2021-11-16 2022-02-18 北京有竹居网络技术有限公司 用于验证表述的真实性的方法、设备、装置和介质

Also Published As

Publication number Publication date
CN114065741B (zh) 2023-08-11
CN114065741A (zh) 2022-02-18

Similar Documents

Publication Publication Date Title
US9922025B2 (en) Generating distributed word embeddings using structured information
CN111611810B (zh) 一种多音字读音消歧装置及方法
Filice et al. Kelp: a kernel-based learning platform for natural language processing
US10719668B2 (en) System for machine translation
CN111832312B (zh) 文本处理方法、装置、设备和存储介质
US11170169B2 (en) System and method for language-independent contextual embedding
Downey et al. Sparse information extraction: Unsupervised language models to the rescue
US11669687B1 (en) Systems and methods for natural language processing (NLP) model robustness determination
US11941361B2 (en) Automatically identifying multi-word expressions
US20220245353A1 (en) System and method for entity labeling in a natural language understanding (nlu) framework
US11669740B2 (en) Graph-based labeling rule augmentation for weakly supervised training of machine-learning-based named entity recognition
WO2018174816A1 (en) Method and apparatus for semantic coherence analysis of texts
WO2018174815A1 (en) Method and apparatus for semantic coherence analysis of texts
Zhang et al. Event recognition based on deep learning in Chinese texts
US20220245361A1 (en) System and method for managing and optimizing lookup source templates in a natural language understanding (nlu) framework
WO2023088278A1 (zh) 用于验证表述的真实性的方法、设备、装置和介质
CN111723583B (zh) 基于意图角色的语句处理方法、装置、设备及存储介质
CN113705207A (zh) 语法错误识别方法及装置
US20220237383A1 (en) Concept system for a natural language understanding (nlu) framework
Han et al. Lexicalized neural unsupervised dependency parsing
US20220229986A1 (en) System and method for compiling and using taxonomy lookup sources in a natural language understanding (nlu) framework
CN113779199B (zh) 用于文档和摘要的一致性检测的方法、设备、装置和介质
US20220229987A1 (en) System and method for repository-aware natural language understanding (nlu) using a lookup source framework
US20220229990A1 (en) System and method for lookup source segmentation scoring in a natural language understanding (nlu) framework
CN113742445B (zh) 文本识别样本获取、文本识别方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22894811

Country of ref document: EP

Kind code of ref document: A1