CN118095493A

CN118095493A - Facts assessment model training method, facts assessment method and device

Info

Publication number: CN118095493A
Application number: CN202410364454.0A
Authority: CN
Inventors: 李庆泉; 都文龙; 刘骏楠; 王学伟; 刘瑾
Original assignee: Ant Fortune Shanghai Financial Information Service Co ltd
Current assignee: Ant Fortune Shanghai Financial Information Service Co ltd
Priority date: 2024-03-27
Filing date: 2024-03-27
Publication date: 2024-05-28

Abstract

The specification discloses a fact evaluation model training method, a fact evaluation method and a device, wherein the fact evaluation model training method comprises the following steps: generating a sample text pair with a factual evaluation tag based on the reference text; inputting a sample text pair into an initial fact evaluation model for model training, determining the fact evaluation probability distribution information of the sample text pair, and outputting a fact evaluation result of the sample text pair based on the fact evaluation probability distribution information; and determining a model loss value based on the facts assessment label and the facts assessment result, and finally carrying out model parameter adjustment on the initial facts assessment model by using the model loss value until the initial facts assessment model completes model training to obtain a target facts assessment model.

Description

Facts assessment model training method, facts assessment method and device

Technical Field

The present disclosure relates to the field of natural language processing technologies, and in particular, to a fact assessment model training method, a fact assessment method, and a device.

Background

In recent years, with the rapid development of large models, the large models are gradually becoming the focus of artificial intelligence, especially in the technical field of natural language processing. However, in the process of generating text contents by using a large model, there are cases where the fact of the text contents generated by the large model is inconsistent with the fact of the reference text input by the large model, and the fact consistency of the text contents generated by the large model with the reference text input by the large model is difficult to evaluate effectively.

Disclosure of Invention

The specification provides a fact evaluation model training method, a fact evaluation method and a device, wherein the technical scheme is as follows:

in a first aspect, the present specification provides a fact assessment model training method, the method comprising:

Creating an initial fact evaluation model, acquiring a reference text, and generating a sample text pair with a fact evaluation label based on the reference text;

Model training the sample text pair to be input into the initial fact evaluation model, determining the fact evaluation probability distribution information of the sample text pair based on the initial fact evaluation model, and outputting a fact evaluation result of the sample text pair based on the fact evaluation probability distribution information;

And determining a model loss value based on the facts assessment label and the facts assessment result, and carrying out model parameter adjustment on the initial facts assessment model by adopting the model loss value until the initial facts assessment model finishes model training to obtain a target facts assessment model.

In a second aspect, the present specification provides a facts assessment method, the method comprising:

Acquiring a reference text and a target generation text aiming at a target large model, wherein the reference text is an input text of the target large model, and the target generation text is an output text of the target large model;

Obtaining a target text pair based on the reference text and the target generation text, inputting the target text pair into a target fact evaluation model, determining target fact evaluation probability distribution information of the target text pair based on the target fact evaluation model, and outputting a target fact evaluation result of the target text pair based on the target fact evaluation probability distribution information.

In a third aspect, the present specification provides a fact assessment model training apparatus, the apparatus comprising:

the creating module is suitable for creating an initial fact evaluation model, acquiring a reference text, and generating a sample text pair with a fact evaluation label based on the reference text;

The model training module is suitable for carrying out model training on the sample text pair input into the initial fact evaluation model, determining the fact evaluation probability distribution information of the sample text pair based on the initial fact evaluation model, and outputting a fact evaluation result of the sample text pair based on the fact evaluation probability distribution information;

And the model generation module is suitable for determining a model loss value based on the facts assessment label and the facts assessment result, and carrying out model parameter adjustment on the initial facts assessment model by adopting the model loss value until the initial facts assessment model finishes model training to obtain a target facts assessment model.

In a fourth aspect, the present specification provides a facts assessment device, the device comprising:

The acquisition module is suitable for acquiring a target reference text and a target generation text aiming at a target large model, wherein the target reference text is an input text of the target large model, and the target generation text is an output text of the target large model;

The facts assessment module is suitable for obtaining a target text pair based on the target reference text and the target generation text, inputting the target text pair into a target facts assessment model, determining target facts assessment probability distribution information of the target text pair based on the target facts assessment model, and outputting a target facts assessment result of the target text pair based on the target facts assessment probability distribution information.

In a fifth aspect, the present description provides a computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the above-described method steps.

In a sixth aspect, the present description provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the above-mentioned method steps.

In a seventh aspect, the present description provides a computer program product storing at least one instruction for loading by a processor and performing the method steps of any one of the above.

The technical scheme provided by some embodiments of the present specification has the following beneficial effects: and generating a sample text pair with a facts assessment label based on the reference text, so that the generated sample text pair is close to text data in a real scene, and then carrying out model training on an initial facts assessment model based on the sample text pair.

Meanwhile, the fact evaluation model training method provided by the specification effectively avoids model training by adopting randomly generated text pairs, and avoids errors such as deviation and potential grammar of a data set in the traditional method.

Drawings

In order to more clearly illustrate the technical solutions of the present specification or the prior art, the following description will briefly introduce the drawings that are required to be used in the embodiments or the prior art descriptions, it is obvious that the drawings in the following description are only some embodiments of the present specification, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic view of a facts assessment model training system provided in the present specification;

FIG. 2 is a schematic flow chart of a training method of a fact assessment model according to an embodiment of the present disclosure;

FIG. 3 is a flow chart of generating sample text pairs with factual evaluation labels based on reference text provided by embodiments of the present disclosure;

FIG. 4 is a flow chart of another training method for a fact assessment model according to an embodiment of the present disclosure;

FIG. 5 is a schematic flow chart of determining factual evaluation probability distribution information of a sample pair according to an embodiment of the present disclosure;

FIG. 6 is a schematic flow chart of determining a factual evaluation result according to an embodiment of the present disclosure;

fig. 7 is a schematic flow chart of a fact assessment method according to an embodiment of the present disclosure;

FIG. 8 is a flow chart of another method for facts assessment according to an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of the fact assessment model training apparatus of the present specification;

Fig. 10 is a schematic structural view of the fact-evaluating device of the present specification;

Fig. 11 is a block diagram of an electronic device according to an embodiment of the present disclosure;

FIG. 12 is a schematic diagram of an operating system and user space according to an embodiment of the present disclosure;

FIG. 13 is a block diagram of the android operating system of FIG. 12;

FIG. 14 is a diagram of an architecture of the IOS system of FIG. 12.

Detailed Description

The following description of the embodiments of the present invention will be made apparent from, and elucidated with reference to, the drawings of the present specification, in which embodiments described are only some, but not all, embodiments of the present specification. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.

In the description of the present specification, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In the description of the present specification, it should be noted that, unless expressly specified and limited otherwise, "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus. The specific meaning of the terms in this specification will be understood by those of ordinary skill in the art in the light of the specific circumstances. In addition, in the description of the present specification, unless otherwise indicated, "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.

The present specification is described in detail below with reference to specific examples.

Referring to fig. 1, fig. 1 is a schematic view of a scenario of a factual assessment model training system provided in the present specification. As shown in fig. 1, the fact assessment model training system may include at least a client cluster and a service platform 100.

The client cluster may include at least one client, as shown in fig. 1, specifically including a client 1 corresponding to a user 1, a client 2 corresponding to a user 2, …, and a client n corresponding to a user n, where n is an integer greater than 0.

Each client in the client cluster may be a communication-enabled electronic device including, but not limited to: wearable devices, handheld devices, personal computers, tablet computers, vehicle-mounted devices, smart phones, computing devices, or other processing devices connected to a wireless modem, etc. Electronic devices in different networks may be called different names, for example: a user equipment, an access terminal, a subscriber unit, a subscriber station, a mobile station, a remote terminal, a mobile device, a user terminal, a wireless communication device, a user agent or user equipment, a cellular telephone, a cordless telephone, an electronic device in a Personal Digital Assistant (PDA), a 5G network, or a future evolution network, etc.

The service platform 100 may be a separate server device, such as: rack-mounted, blade, tower-type or cabinet-type server equipment or hardware equipment with stronger computing capacity such as workstations, mainframe computers and the like is adopted; the server cluster may also be a server cluster formed by a plurality of servers, and each server in the server cluster may be formed in a symmetrical manner, wherein each server is functionally equivalent and functionally equivalent in a transaction link, and each server may independently provide services to the outside, and the independent provision of services may be understood as no assistance of another server is needed.

In one or more embodiments of the present description, the service platform 100 may establish a communication connection with at least one client in the client cluster, and complete interaction of data in a fact assessment model training process, such as online transaction data interaction, based on the communication connection, e.g., the service platform 100 may implement providing a fact assessment service to the client based on a target fact assessment model obtained by the fact assessment model training method of the present description; as another example, the service platform 100 may obtain training data, such as first training data, from a client.

It should be noted that, the service platform 100 establishes a communication connection with at least one client in the client cluster through a network for interactive communication, where the network may be a wireless network, or may be a wired network, where the wireless network includes, but is not limited to, a cellular network, a wireless local area network, an infrared network, or a bluetooth network, and the wired network includes, but is not limited to, an ethernet network, a universal serial bus (universal serial bus, USB), or a controller area network. In one or more embodiments of the specification, techniques and/or formats including HyperText Mark-up Language (HTML), extensible markup Language (Extensible Markup Language, XML), and the like are used to represent data exchanged over a network (e.g., a target compression package). All or some of the links may also be encrypted using conventional encryption techniques such as secure sockets layer (Secure Socket Layer, SSL), transport layer security (Transport Layer Security, TLS), virtual private network (Virtual Private Network, VPN), internet protocol security (Internet Protocol Security, IPsec), etc. In other embodiments, custom and/or dedicated data communication techniques may also be used in place of or in addition to the data communication techniques described above.

The embodiment of the fact evaluation model training system provided in the present specification and the fact evaluation model training method in one or more embodiments belong to the same concept, and an execution subject corresponding to the fact evaluation model training method related in one or more embodiments in the present specification may be the service platform 100 described above; the execution subject corresponding to the fact evaluation model training method related to one or more embodiments of the specification may also be an electronic device corresponding to the client, and specifically determined based on an actual application environment. The implementation process of the fact assessment model training system embodiment may be described in detail in the following method embodiment, which is not described herein.

Based on the schematic view of the scenario shown in fig. 1, a detailed description of a facts assessment model training method provided in one or more embodiments of the present specification is provided below.

Referring to fig. 2, fig. 2 is a schematic flow chart of a training method of a fact evaluation model according to an embodiment of the present disclosure. The method may be implemented in dependence on a computer program, and may be run on a von neumann system-based fact assessment model training device. The computer program may be integrated in the application or may run as a stand-alone tool class application. The fact assessment model training apparatus may be a service platform.

Specifically, the fact assessment model training method comprises the following steps:

S202: an initial fact assessment model is created, reference text is acquired, and sample text pairs with a fact assessment tag are generated based on the reference text.

The initial fact evaluation model can be an untrained basic generation type large model, or a generation type large model obtained through incomplete training. Here, the initial facts assessment model may initially perform a factual assessment of the text content. The factual assessment may be an assessment of the authenticity and accuracy of text content generated by the large model. The fact evaluation has wide application value in more fields. For example, in the field of news stories, the factual evaluation of large model-generated news is a critical task to discern false news and misleading information; in legal litigation, evaluating the authenticity and reliability of evidence obtained by sorting and summarizing based on a large model is beneficial to improving the litigation efficiency; in scientific research, experimental data obtained by sorting based on a large model is subjected to actual evaluation, so that reliability of research results is ensured.

Traditional factual assessment methods often rely on manual auditing and expert judgment, and have certain limitations. On one hand, the manual auditing is high in cost, long in time consumption and difficult to deal with the processing of massive information; on the other hand, expert judgment is susceptible to subjective consciousness and existing cognition, possibly resulting in deviation of the evaluation result.

After the initial fact assessment model is created, reference text may be obtained, including but not limited to news stories, official announcements, scientific data, and various types of documents, etc. After obtaining the reference text, generating a sample text pair based on the reference text, the sample text pair may include the reference text and a sample text generated based on the reference text, and a facts assessment tab for characterizing whether the reference text and the sample text are in fact identical. In particular, the sample text may include positive text having the same facts as the reference text and negative text having facts that are contrary to the reference text.

To avoid deviations of the generated sample text from the text form of the actual scene, deviations from the actual scene mismatch, such as grammar errors, and mismatch of the sample text and the reference text with the actual evaluation tag, etc., are caused. Sample text generation may be performed on the reference text based on the large model and the prompter of the normalized large model. Specifically, the positive text and the negative text can be generated based on the same large model, and the positive text and the negative text can be generated based on prompt words of different large models; of course, positive text and negative text may also be generated based on different large models.

Here, the facts assessment labels include a reference positive facts assessment label and a reference negative facts assessment label, when the facts assessment label is the reference positive facts assessment label, indicating that the facts of the sample text pairs are the same, i.e., the sample text has the same facts as the reference text; when the factual evaluation tab is a reference negative factual evaluation tab, it indicates that the facts of the sample text pair are contradictory, i.e., the sample text has facts that are contradictory to the reference text. It will be readily appreciated that the reference positive facts assessment labels and the reference negative facts assessment labels may be represented by quantified numbers.

S204: and inputting the sample text pair into an initial fact evaluation model for model training, determining the fact evaluation probability distribution information of the sample text pair based on the initial fact evaluation model, and outputting the fact evaluation result of the sample text pair based on the fact evaluation probability distribution information.

After the sample text pair is input into the initial fact evaluation model, the initial fact evaluation model learns the text characteristics, entity characteristics, semantic characteristics and the like of the reference text and the sample text in the sample text pair respectively. Text features may be language features in text, including features in terms of vocabulary, grammar, syntax, and the like. Entity characteristics may be entities that appear in text, such as person names, place names, organization names, time, numbers, and so forth. Semantic features refer to the meaning or meaning of a word or phrase in a particular context, which may include the literal meaning of the word or phrase, as well as the literal meaning of the word or phrase in the context.

The initial fact assessment model compares the text features, entity features and semantic features of the reference text with the text features, entity features and semantic features of the sample text, thereby determining the fact assessment probability distribution information of the sample text pairs.

Specifically, the initial fact assessment model may determine a first text feature vector, a first entity feature vector, and a first semantic feature vector of the reference text based on the text features, entity features, and semantic features of the reference text, and determine a second text feature vector, a second entity feature vector, and a second semantic feature vector of the sample text based on the text features, entity features, and semantic features of the sample text.

Then, a first vector similarity of the first text feature vector and the second text feature vector is calculated, a second vector similarity of the first entity feature vector and the second entity feature vector is calculated, a third vector similarity of the first semantic feature vector and the second semantic feature vector is calculated, and the fact evaluation probability distribution information of the reference text and the sample text in the sample text pair is determined based on the first vector similarity, the second vector similarity and the third vector similarity, namely, the first probability that the reference text and the sample text are the same fact text and the second probability that the reference text and the sample text are opposite to the fact text.

After obtaining the factual evaluation probability distribution information of the sample text pairs, analyzing the factual evaluation probability distribution information to determine the factual evaluation results of the sample text pairs, wherein the factual evaluation results comprise a sample positive factual evaluation label and a sample negative factual evaluation label. It is readily understood that the sample positive facts assessment labels and the sample negative facts assessment labels may be represented by quantified numbers.

The model base of the initial facts assessment model may employ a generative model, such as T5 (Text-to-Text transfer model, text-to-Text Transfer Transformer), for which hinting words may be built for sample Text pairs during model training of the sample Text pairs into the initial facts assessment model, such as the input of the initial facts assessment model: "does the following assertions agree with the context? Assertion of: < generated text > context: the generated text may be a sample text in the sample text pair corresponding to the reference text, and the source text may be the reference text.

S206: and determining a model loss value based on the facts assessment label and the facts assessment result, and performing model parameter adjustment on the initial facts assessment model by using the model loss value until the initial facts assessment model completes model training to obtain a target facts assessment model.

Determining a first parameter corresponding to the facts assessment label and a second parameter corresponding to the facts assessment result, constructing a model loss function based on the first parameter and the second parameter, substituting quantization numbers corresponding to the facts assessment label and the facts assessment result into the model loss function to calculate a model loss value, performing model parameter adjustment on an initial facts assessment model based on the model loss value after obtaining the model loss value, and performing model training on the initial facts assessment model again based on a sample until the model loss value converges, wherein the initial facts assessment model completes model training to obtain a target facts assessment model.

In the specification, a sample text pair with a facts assessment label is generated based on a reference text, so that the generated sample text pair is close to text data in a real scene, then an initial facts assessment model is model trained based on the sample text pair, and the initial facts assessment model can fully learn facts assessment information of the sample text pair to continuously adjust model parameters so as to obtain a target facts assessment model, so that the facts consistency of text content generated by a large model and the reference text input by the large model can be assessed.

Referring to fig. 3, fig. 3 is a schematic flow chart of generating a sample text pair with a factual evaluation tag based on a reference text according to an embodiment of the present disclosure. As shown in fig. 3, generating a sample text pair with a factual evaluation tag based on the reference text in S202 includes:

S302: and carrying out the same factual data generation processing on the reference text to obtain a positive text, and carrying out the opposite factual data generation processing on the reference text to obtain a negative text.

The method comprises the steps of rewriting a reference text through at least three dimensions of comprehensive text features, entity features and semantic features, so that facts of the obtained forward text and the reference text at least in the three dimensions are consistent, and the same fact data generation processing of the reference text is achieved.

The text features can be key text features, importance weights corresponding to language features in the reference text can be calculated, then the importance weights of the language features in the reference text are ordered in a descending order, the language features corresponding to the importance weights of the preset proportion arranged in front are extracted to serve as the key text features, and therefore forward text generation is carried out based on the key text features instead of all the text features, generation constraint conditions of the forward text are reduced, and forward text can be generated as much as possible while facts of the forward text and the reference text are kept consistent as much as possible.

In contrast, in the process of generating the contradictory factual data on the reference text, the reference text is rewritten by at least one of three dimensions of text features, entity features and semantic features, so that the facts of the obtained negative text and the reference text are kept inconsistent at least in at least one of the three dimensions, and the process of generating the contradictory factual data on the reference text is realized.

S304: a positive text pair with a positive factuality evaluation tag is generated based on the positive text and the reference text, and a negative text pair with a negative factuality evaluation tag is generated based on the negative text and the reference text.

Wherein the first text pair is obtained based on the forward text and the reference text, and since the forward text is obtained by performing the same factual data generation processing based on the reference text, the forward factual evaluation tag can be marked on the first text pair, thereby obtaining the forward text pair.

Similarly, the second text pair is obtained based on the negative text and the reference text, and the negative text pair is obtained by performing the contradictory factual data generation processing based on the reference text, so that the negative factual evaluation tag can be marked on the second text pair, and the negative text pair is obtained.

Based on the forward text pairs, the initial fact assessment model can fully learn the fact assessment information of the forward text pairs so as to continuously adjust model parameters in a fact consistency dimension; also, based on the negative text pairs, the initial fact assessment model may learn the factual assessment information of the negative text pairs sufficiently to make constant adjustments to model parameters in the factually contradictory dimensions.

S306: sample text pairs with factual evaluation labels are derived based on the positive text pairs and the negative text pairs.

The fact evaluation labels comprise positive fact evaluation labels and negative fact evaluation labels, the sample text pairs comprise positive text pairs and negative text pairs, the obtained positive text pairs and negative text pairs are put into the same data set, a fact data set is obtained, and each element in the fact data set is the sample text pair.

The initial fact evaluation model is trained based on the sample text, so that the initial fact evaluation model can fully learn the fact evaluation information of the positive text pair and the negative text pair in the dimension of the fact consistency and the dimension of the fact contradiction, the model parameters are continuously adjusted, and the accuracy of the fact evaluation result of the obtained target fact evaluation model is improved.

In the embodiment provided in the present specification, the same factual data generation processing and the opposite factual data generation processing are performed on the reference text, so that a positive text which is in agreement with the facts of the reference text and a negative text which is in contradiction with the facts of the reference text are obtained, then a positive text pair is obtained through the positive text and the reference text, and a negative text pair is obtained through the negative text and the reference text, so that the initial fact evaluation model can fully learn the factual evaluation information of the positive text pair and the negative text pair in the dimension of the facts consistency and the dimension of the facts in contradiction, and the accuracy of the factual evaluation result of the obtained target fact evaluation model is improved.

In one embodiment provided in the present specification, the same factual data generation process is performed on the reference text in S302, to obtain a forward text, including:

Carrying out the same factual overwriting processing on the reference text based on the forward text generation model to obtain a forward text; the forward text generation model is obtained by processing scene adaptation aiming at the same facts based on a first basic large language generation model.

The fact consistency of the generated forward text can be more reasonable based on the generation capacity of the large model corresponding to the forward text generation model, meanwhile, the method is close to a real text application scene, and errors such as deviation and potential grammar of a data set in a traditional method are avoided.

In the process of carrying out the same factual rewrite processing on the reference text through the forward text generation model, the forward text is obtained by not changing the fact consistency of the rewritten text and the reference text. Specifically, the forward text generation model may ensure fact consistency of the generated forward text with the reference text from at least three dimensions, integrated text features, entity features, and semantic features.

Meanwhile, in the process of adapting the first basic large language generation type model to obtain the forward text generation model for the same factual rewrite processing scene, the adaptation can be performed by performing parameter adjustment on the first basic large language generation type model while modifying the corresponding Prompt (Prompt word), so that the forward text consistent with the facts of the reference text can be output for the same factual rewrite processing scene by constructing a specific Prompt for the reference text. Here, the first basic large language generative model may be a GPT (generative pre-training transducer model GENERATIVE PRE-Trained Transformer) -3.5-turbo equi-large model.

In the embodiment provided by the specification, based on the generation capability of the large model corresponding to the forward text generation model, the fact consistency of the generated forward text and the reference text can be more reasonable, so that the generated forward text can be close to a real text application scene, and errors such as deviation, potential grammar and the like existing in a data set in a traditional method are avoided.

In one embodiment provided in the present specification, the performing, in S302, the negative text by performing the contradictory factual data generation processing on the reference text includes:

Performing contradictory factual rewrite processing on the reference text based on the negative text generation model to obtain a negative text; the negative text generation model is obtained by adapting a second basic large language generation model to a contrary-to-reality rewriting processing scene.

The fact paradox of the generated negative text can be more reasonable based on the generation capacity of the large model corresponding to the negative text generation model, and meanwhile, the method is close to a real text application scene, and errors such as deviation and potential grammar of a data set in a traditional method can be avoided.

In the process of carrying out the contradictory factual rewriting processing on the reference text through the negative text generation model, the negative text is obtained by changing the fact consistency between the rewritten text and the reference text. Specifically, the positive text generation model may ensure that the generated negative text is contradictory to the fact of the reference text from at least one of three dimensions, integrated text features, entity features, and semantic features.

Meanwhile, in the process of adapting the negative text generation model for the contradictory factual rewrite processing scene based on the second basic large language generation model, the adaptation can be performed by performing parameter adjustment on the second basic large language generation model while modifying the corresponding Prompt (Prompt word), so that the negative text that is contradictory to the factual rewrite processing scene of the reference text can be output for the contradictory factual rewrite processing scene by constructing a specific Prompt for the reference text. Here again, the second basic large language generative model may be a GPT (generative pre-trained transducer model GENERATIVE PRE-Trained Transformer) -3.5-turbo equi-model.

In the embodiment provided by the specification, based on the generation capability of the large model corresponding to the negative text generation model, the fact paradox of the generated negative text and the reference text can be more reasonable, so that the generated negative text can be close to a real text application scene, and errors such as deviation and potential grammar existing in a data set in a traditional method are avoided.

It should be appreciated that the positive text generation model and the negative text generation model may be different models or the same model.

Referring to fig. 4, fig. 4 is a schematic flow chart of another training method for a fact assessment model according to an embodiment of the present disclosure. As shown in fig. 4, in the embodiment provided in the present specification, when the positive text generation model and the negative text generation model are the same model, the first basic large language generation formula model and the second basic large language generation formula model are the same. At this time, the first basic large language generation type model or the second basic large language generation type model may be adapted to the paradox factual rewrite processing scene at the same time, and adapted to the same factual rewrite processing scene, so that the large model may output negative text that is paradox to the facts of the reference text for the paradox factual rewrite processing scene based on the hint words of the paradox factual rewrite processing scene, and the large model may output positive text that is in agreement with the facts of the reference text for the same factual rewrite processing scene based on the hint words of the same factual rewrite processing scene.

And then, obtaining a second text pair based on the negative text and the reference text, wherein the negative text is obtained by performing opposite factual data generation processing based on the reference text, so that a negative factual evaluation label can be marked on the second text pair, thereby obtaining a negative text pair, putting the obtained positive text pair and negative text into the same data set to obtain a factual data set, and performing model training on an initial factual evaluation model based on sample text pairs in the factual data set to obtain a target factual evaluation model.

In an embodiment provided in the present specification, in the foregoing embodiment, performing the same factual overwriting process on the reference text based on the forward text generation model to obtain the forward text, including:

And rewriting the reference text by adopting at least one of the same factual word order rearrangement processing, the same factual active-passive conversion processing, the same factual sentence pattern fusion processing, the same factual text abbreviation processing and the same factual synonymous substitution processing based on the forward text generation model to obtain the forward text.

The same fact-based word rearrangement process may be to rearrange phrases, sentences or paragraphs in the reference text, to keep the authenticity of the information consistent with the reference text, i.e. to adjust the word order in conformity with logic or expression habits, and care should be taken to keep the fact and information consistent in the text and to avoid changing the meaning or meaning of the text when such a rewrite is performed.

The same factual active-passive conversion processing of the reference text may be processing of active-passive conversion of the factual content in the reference text. Specifically, the active language in the reference text is converted into the passive language, or the passive language is converted into the active language. In doing so, it is necessary to ensure that the converted forward text still accurately conveys the meaning of the reference text.

The same factual sentence pattern fusion processing of the reference text can be that sentence patterns in the reference text are fused, and a plurality of parallel sentences are fused, so that text space can be reduced, the simplicity and the readability of the text are improved, and the method is a commonly used text conversion type in an actual scene. Also, in making such rewrites, care should be taken to maintain consistency of facts and information in the text and avoid changing the meaning or meaning of the text.

The same de facto text abbreviation processing of the reference text may be one or several pieces of text in the reference text that have repeated content, requiring abbreviation processing to reduce space or increase efficiency. In performing the abbreviation processing, some methods such as deleting duplicate sentences, simplifying sentence patterns, merging similar concepts, employing canonical abbreviation names, etc. may be employed to retain main information and reduce redundancy. The abbreviated text should retain the same factual content as the original text while remaining as clear and easy to understand as possible.

The same factual synonymous substitution process for the reference text may be the same, factual synonymous substitution process for some content in the reference text. In particular, this process may involve replacing certain words or phrases in text, such as verbs, adverbs, nouns, etc., to better express the same meaning or concept. Also, in making such rewrites, care should be taken to maintain consistency of facts and information in the text and avoid changing the meaning or meaning of the text.

In addition, in the process of rewriting the reference text, part of detail information can be omitted, and the fact consistency of the reference text and the forward text is not affected by the loss of part of detail information. It should be noted that, the fact that the reference text is consistent with the forward text means that the text content in the forward text may find a basis in the reference text, instead of the content of the reference text being identical to the content of the forward text, the content of the reference text may include the content of the forward text.

In the embodiment provided in the present specification, the reference text is rewritten by the forward text generation model, which essentially means that the reference text is text-interpreted by the forward text generation model, so that the forward text consistent with the facts of the reference text is obtained.

In an embodiment provided in the present specification, in the foregoing embodiment, performing, based on a negative text generation model, a process of countering a factual rewrite of a reference text to obtain a negative text includes:

and rewriting the reference text based on at least one of the negative text generation model, the negative text is obtained by adopting at least one of the negative facts numerical conversion processing, the negative facts antisense replacement processing and the negative facts host-guest exchange processing.

Wherein the process of countering the fact that the reference text is rewritten may be to modify or change the fact in the reference text to be opposite or contradictory to the original fact in the reference text. The countering factual overwriting process may be a distortion, exaggeration, shrinkage, or illusion of facts in the reference text to change the meaning or effect of the reference text.

The contrary to the actual value transformation processing of the reference text is to edit the data of the reference text contrary to the value of the reference text, such as transforming and rewriting the value (such as date) of the reference text, for example, modifying 1 month and 1 day to 1 month and 2 days.

The process of performing the contradictory de-sense substitution on the reference text can be understood as performing the de-sense substitution on the verb, the adverb or the noun in the reference text, i.e. performing the part-of-speech de-sense editing. The contrary to the fact that the host-guest swap processing is performed on the reference text may be that the host and the guest in the reference text are swapped.

In the embodiment provided in the present specification, the reference text is rewritten by the negative text generation model, which is essentially a negative text that is in contradiction to the fact that the reference text is rewritten by the negative text generation model, so as to obtain the negative text that is in contradiction to the fact of the reference text.

Referring to fig. 5, fig. 5 is a schematic flow chart for determining the probability distribution information of the factual evaluation of the sample pairs according to the embodiment of the present disclosure. Specifically, determining the factual evaluation probability distribution information of the sample pairs based on the initial factual evaluation model in S204 includes:

S502: a first probability that a pair of samples is identical fact text is determined based on an initial fact assessment model.

The initial fact evaluation model may determine a first text feature vector, a first entity feature vector, and a first semantic feature vector of the reference text based on the text features, the entity features, and the semantic features of the reference text, and determine a second text feature vector, a second entity feature vector, and a second semantic feature vector of the sample text based on the text features, the entity features, and the semantic features of the sample text.

Calculating first vector similarity of the first text feature vector and the second text feature vector, calculating second vector similarity of the first entity feature vector and the second entity feature vector, calculating third vector similarity of the first semantic feature vector and the second semantic feature vector, and determining a first probability that the pair of samples are identical fact text based on the first vector similarity, the second vector similarity and the third vector similarity.

S504: a second probability that the pairs of samples are contradictory facts is determined based on the initial facts assessment model.

Wherein the initial fact assessment model may likewise determine a second probability that the pair of sample texts is paradox to the fact text based on the first vector similarity, the second vector similarity, and the third vector similarity; alternatively, a second probability that a sample pair is a counterintuitive factual text may be calculated based on a first probability that the sample pair is the same factual text, where the sum of the first probability and the second probability may be 1.

S506: the factual evaluation probability distribution information is obtained based on the first probability and the second probability.

After the first probability and the second probability are obtained, the first probability is used for representing the probability that the reference text and the sample text in the sample text pair are the same factual text; the second probability is used to characterize the probability that the reference text and the sample text in the sample text pair are mutually contradictory to the factual text.

The first probability that the reference text and the sample text in the sample text pair are the same as each other and the second probability that the reference text and the sample text in the sample text pair are opposite to each other are recorded in the factual evaluation probability distribution information.

In the embodiment provided in the present specification, a first probability that a pair of sample texts is the same factual text and a second probability that a pair of sample texts is the opposite factual text are determined based on the initial factual evaluation model, so that factual evaluation probability distribution information is obtained based on the first probability and the second probability, thereby facilitating determination of a factual evaluation result of the pair of sample texts based on the factual evaluation probability distribution information.

Referring to fig. 6, fig. 6 is a schematic flow chart of determining a factual evaluation result according to an embodiment of the present disclosure. As shown in fig. 6, outputting a result of the factual evaluation of the sample text pair based on the factual evaluation probability distribution information in S204 includes:

S602: the first probability and the second probability are compared.

Wherein, since the first probability is used to characterize the probability that the reference text and the sample text in the sample text pair are the same factual text with each other; the second probability is used to characterize the probability that the reference text and the sample text in the sample text pair are mutually contradictory to the factual text.

Thus, by comparing the magnitudes of the first probability and the second probability, the tendency of the reference text and the sample text in the sample text pair to be the same as each other and the tendency to be the opposite of each other can be determined.

S604: and outputting the forward facts assessment result of the sample when the first probability is greater than or equal to the second probability.

When the first probability is greater than or equal to the second probability, the tendency that the reference text and the sample text in the sample text pair are the same factual text is greater than or equal to the tendency that the reference text and the sample text are opposite factual text, and the factual evaluation results corresponding to the reference text and the sample text in the sample text pair can be considered as sample forward factual evaluation results.

S606: and when the first probability is smaller than the second probability, outputting a sample negative-going facts evaluation result.

When the first probability is smaller than the second probability, the tendency that the reference text and the sample text in the sample text pair are the same factual text is smaller than the tendency that the reference text and the sample text are opposite to the factual text, and the factual evaluation result corresponding to the reference text and the sample text in the sample text pair can be considered as a negative factual evaluation result of the sample.

In the embodiment provided in the present specification, by comparing the first probability and the second probability, when the first probability is greater than or equal to the second probability, indicating that the tendency of the reference text and the sample text in the sample text pair to be the same factual text as each other is greater than or equal to the tendency of the reference text and the sample text to be the opposite factual text to each other, the corresponding factual evaluation result is the sample forward factual evaluation result; when the first probability is smaller than the second probability, indicating that the tendency of the reference text and the sample text in the sample text pair to be the same factual text with each other is smaller than the tendency of the reference text and the sample text to be the opposite factual text with each other, the corresponding factual evaluation result is a sample negative factual evaluation result.

Referring to fig. 7, fig. 7 is a schematic flow chart of a fact evaluation method according to an embodiment of the present disclosure. The method may be implemented in dependence on a computer program, and may be run on a fact assessment device based on von neumann systems. The computer program may be integrated in the application or may run as a stand-alone tool class application.

Specifically, the method comprises the following steps:

S702: and acquiring a target reference text and a target generation text aiming at the target large model, wherein the target reference text is an input text of the target large model, and the target generation text is an output text of the target large model.

After receiving the target reference text, the target large model outputs the generated target generation text based on the target reference text. The target big model is used for generating target generation text based on the target reference text based on instructions of a user, for example, the target big model can refine text content of the target reference text to generate target generation text; or the target big model can answer the questions of the user based on the target reference text aiming at the questions of the user to obtain target generation text. Here, the text processing type of the target large model is not limited.

S704: and obtaining a target text pair based on the target reference text and the target generation text, inputting the target text pair into a target fact evaluation model, determining target fact evaluation probability distribution information of the target text pair based on the target fact evaluation model, and outputting a target fact evaluation result of the target text pair based on the target fact evaluation probability distribution information.

The input text of the target large model and the output text of the target large model are combined to form a target text pair, that is, the target text pair is obtained based on the target reference text and the target generation text, and after the target text pair is obtained, the target text pair is input into the target fact evaluation model, and the training process of the target fact evaluation model may refer to steps S202 to S206, which are not repeated herein.

After inputting the target text pair into the target fact assessment model, the target fact assessment model may determine a first text feature vector, a first entity feature vector, and a first semantic feature vector of the target reference text based on the text features, entity features, and semantic features of the target reference text, and determine a second text feature vector, a second entity feature vector, and a second semantic feature vector of the target generated text based on the text features, entity features, and semantic features of the target generated text.

Then, a first vector similarity of the first text feature vector and the second text feature vector is calculated, a second vector similarity of the first entity feature vector and the second entity feature vector is calculated, a third vector similarity of the first semantic feature vector and the second semantic feature vector is calculated, and fact evaluation probability distribution information of the target reference text and the target generated text in the target text pair is determined based on the first vector similarity, the second vector similarity and the third vector similarity, namely a first probability that the target reference text and the target generated text are the same fact text and a second probability that the target reference text and the target generated text are opposite fact text.

After the target facts evaluation probability distribution information of the target text pair is obtained, the target facts evaluation probability distribution information is analyzed to determine a target facts evaluation result of the target text pair, the target facts evaluation result comprises a target positive facts evaluation label and a target negative facts evaluation label, the target positive facts evaluation label indicates that the target reference text and the target generation text are the same facts text, and the target negative facts evaluation label indicates that the target reference text and the target generation text are opposite facts text.

In the embodiment provided in the present specification, the target fact assessment model is obtained by fully learning the fact assessment information of the sample text pairs based on the initial fact assessment model so as to continuously adjust the model parameters, so that the target fact assessment model can evaluate the fact consistency of the large model output text and the large model input text. Thus, the target facts assessment probability distribution information of the sample text pairs can be accurately determined based on the target facts assessment model, and then the target facts assessment results of the target text pairs can be determined.

Referring to fig. 8, fig. 8 is a flow chart of another fact evaluation method according to an embodiment of the present disclosure. In the embodiment provided in the present specification, the fact evaluation method may further include:

S802: a first probability that the target text pair is the same factual text is determined based on the target factual evaluation probability distribution information.

Wherein the factual evaluation probability distribution information includes a first probability that the target reference text and the target generated text are the same factual text and a second probability that the target reference text and the target generated text are opposite factual text.

Thus, a first probability that the target text pair is the same factual text may be obtained based on the target factual evaluation probability distribution information, the first probability being used to characterize a probability that the target reference text and the target generated text in the target text pair are the same factual text as each other.

S804: a fact consistency score for the target text pair is determined based on the first probability.

The first probability is used for representing the probability that the target reference text and the target generation text in the target text pair are the same factual text, so that the first probability can be subjected to score conversion processing, and the fact consistency score of the target text pair is obtained.

It is easy to understand that the greater the fact consistency score of a target text pair, the higher the probability that the target reference text and the target generated text in the target text pair are the same fact text as each other; the lower the fact consistency score of a target text pair, the lower the probability that the target reference text and the target generated text in the target text pair are the same fact text as each other.

In the embodiments provided herein, a fact consistency score for a target text pair is determined based on a first probability, thereby quantifying the fact consistency for the target text pair based on the fact consistency score.

The fact assessment model training apparatus provided in the embodiment of the present specification will be described in detail with reference to fig. 9. The fact-assessment model training apparatus shown in fig. 9 is used to perform the method of the embodiment shown in fig. 1 to 6 of the present specification, and for convenience of explanation, only the portion relevant to the present specification is shown, and specific technical details are not disclosed, please refer to the embodiment shown in fig. 1 to 6 of the present specification.

Referring to fig. 9, fig. 9 is a schematic structural diagram of a training device for a fact evaluation model in the present specification. The fact assessment model training apparatus 1 may be implemented as all or part of the user terminal by software, hardware or a combination of both. According to some embodiments, the fact assessment model training apparatus 1 comprises a creation module 11, a model training module 12 and a model generation module 13, in particular for:

a creation module 11 adapted to create an initial facts assessment model, obtain a reference text, generate a sample text pair with a facts assessment tag based on the reference text;

The model training module 12 is adapted to input a sample text pair into the initial fact evaluation model for model training, determine the actual evaluation probability distribution information of the sample text pair based on the initial fact evaluation model, and output the actual evaluation result of the sample text pair based on the actual evaluation probability distribution information;

The model generation module 13 is adapted to determine a model loss value based on the facts assessment label and the facts assessment result, and to perform model parameter adjustment on the initial facts assessment model by using the model loss value until the initial facts assessment model completes model training to obtain a target facts assessment model.

Optionally, the creation module 11 includes:

The first generation unit is suitable for carrying out the same factual data generation processing on the reference text to obtain a positive text, and carrying out the opposite factual data generation processing on the reference text to obtain a negative text;

a second generation unit adapted to generate a positive text pair with a positive facts assessment tag based on the positive text and the reference text, and to generate a negative text pair with a negative facts assessment tag based on the negative text and the reference text;

The sample text pair determination unit is adapted to obtain a sample text pair with a factual evaluation tag based on the positive text pair and the negative text pair.

Optionally, the first generating unit is further adapted to perform the same factual overwriting process on the reference text based on the forward text generating model to obtain a forward text; the forward text generation model is obtained by processing scene adaptation aiming at the same facts based on a first basic large language generation model.

Optionally, the first generating unit is further adapted to rewrite the reference text based on the forward text generating model by using at least one of the same factual word order rearrangement process, the same factual active-passive conversion process, the same factual sentence pattern fusion process, the same factual text abbreviation process, and the same factual synonymous substitution process, to obtain the forward text.

Optionally, the first generating unit is further adapted to perform a contradictory factual rewrite process on the reference text based on the negative text generation model, to obtain a negative text; the negative text generation model is obtained by adapting a second basic large language generation model to a contrary-to-reality rewriting processing scene.

Optionally, the first generating unit is further adapted to rewrite the reference text based on the negative text generation model with at least one of a negative-going facts numerical transformation process, a negative-going facts antisense replacement process, and a negative-going facts host-guest exchange process, to obtain the negative text.

Optionally, the model training module 12 includes:

a first probability determination unit adapted to determine a first probability that the pair of sample texts is the same factual text based on the initial factual evaluation model;

A second probability determination unit adapted to determine a second probability that the pair of sample texts is contradictory to the factual text based on the initial fact assessment model;

the probability distribution determining unit is adapted to derive a factual evaluation probability distribution information based on the first probability and the second probability.

Optionally, the model training module 12 includes:

a comparison unit adapted to compare the first probability with the second probability;

The first judging unit is suitable for outputting a sample forward factual evaluation result when the first probability is greater than or equal to the second probability;

And the second judging unit outputs a sample negative-direction facts evaluation result when the first probability is smaller than the second probability.

It should be noted that, when the fact evaluation model training apparatus provided in the foregoing embodiment performs the fact evaluation model training method, only the division of the foregoing functional modules is used as an example, in practical application, the foregoing functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the fact evaluation model training device and the fact evaluation model training method provided in the foregoing embodiments belong to the same concept, which embody detailed implementation procedures in the method embodiments, and are not described herein again.

The fact assessment device provided in the embodiment of the present specification will be described in detail with reference to fig. 10. The fact evaluation device shown in fig. 10 is used to execute the method of the embodiment shown in fig. 7 to 8 of the present specification, and for convenience of explanation, only the portion relevant to the present specification is shown, and specific technical details are not disclosed, please refer to the embodiment shown in fig. 7 to 8 of the present specification.

Referring to fig. 10, fig. 10 is a schematic structural view of the fact assessment apparatus of the present specification. The fact assessment means 2 may be implemented as all or part of the user terminal by software, hardware or a combination of both. According to some embodiments, the fact assessment device 2 comprises an acquisition module 21 and a facts assessment module 22, in particular for:

The acquisition module 21 is adapted to acquire a target reference text for the target large model and a target generation text, wherein the target reference text is an input text of the target large model, and the target generation text is an output text of the target large model;

the facts assessment module 22 is adapted to derive a target text pair based on the target reference text and the target generation text, input the target text pair into a target facts assessment model, determine target facts assessment probability distribution information for the target text pair based on the target facts assessment model, and output a target facts assessment result for the target text pair based on the target facts assessment probability distribution information.

Optionally, the fact assessment apparatus 2 further includes:

a first probability determination module adapted to determine a first probability that the target text pair is the same factual text based on the target factual evaluation probability distribution information;

The fact consistency score determination module is adapted to determine a fact consistency score for the target text pair based on the first probability.

It should be noted that, in the fact evaluation apparatus provided in the foregoing embodiment, when the fact evaluation method is executed, only the division of the foregoing functional modules is used as an example, in practical application, the foregoing functional allocation may be performed by different functional modules, that is, the internal structure of the device is divided into different functional modules, so as to perform all or part of the functions described above. In addition, the fact evaluation device and the fact evaluation method provided in the foregoing embodiments belong to the same concept, which embody the detailed implementation process in the method embodiment, and are not described herein again.

The foregoing description is provided for the purpose of illustration only and does not represent the advantages or disadvantages of the embodiments.

The present disclosure further provides a computer storage medium, where a plurality of instructions may be stored, where the instructions are adapted to be loaded by a processor and executed by the processor to implement the fact assessment model training method or the fact assessment method according to the embodiments shown in fig. 1 to 8, and the specific implementation process may refer to the specific description of the embodiments shown in fig. 1 to 8, which is not repeated herein.

The present disclosure further provides a computer program product, where at least one instruction is stored, where the at least one instruction is loaded by the processor and executed by the processor to implement the fact assessment model training method or the fact assessment method according to the embodiments shown in fig. 1 to 8, and the specific implementation process may refer to the specific description of the embodiments shown in fig. 1 to 8, which is not repeated herein.

Referring to fig. 11, fig. 11 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present disclosure. The electronic device in this specification may include one or more of the following: processor 110, memory 120, input device 130, output device 140, and bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected by a bus 150.

Processor 110 may include one or more processing cores. The processor 110 utilizes various interfaces and lines to connect various portions of the overall electronic device, perform various functions of the electronic device 100, and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and invoking data stored in the memory 120. Alternatively, the processor 110 may be implemented in at least one hardware form of Digital Signal Processing (DSP), field-programmable gate array (FPGA), programmable logic array (programmable logic Array, PLA). The processor 110 may integrate one or a combination of several of a central processor (central processing unit, CPU), an image processor (graphics processing unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for being responsible for rendering and drawing of display content; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 110 and may be implemented solely by a single communication chip.

The memory 120 may include a random access memory (random Access Memory, RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 120 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, which may be an Android (Android) system, including an Android system-based deep development system, an IOS system developed by apple corporation, including an IOS system-based deep development system, or other systems, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, and the like. The storage data area may also store data created by the electronic device in use, such as phonebooks, audiovisual data, chat log data, and the like.

Referring to fig. 12, fig. 12 is a schematic structural diagram of an operating system and a user space provided in the embodiment of the present disclosure, where the memory 120 may be divided into an operating system space and a user space, and an operating system may be run in the operating system space, and a native and a third party application may be run in the user space. In order to ensure that different third party application programs can achieve better operation effects, the operating system allocates corresponding system resources for the different third party application programs. However, the requirements of different application scenarios in the same third party application program on system resources are different, for example, under the local resource loading scenario, the third party application program has higher requirement on the disk reading speed; in the animation rendering scene, the third party application program has higher requirements on the GPU performance. The operating system and the third party application program are mutually independent, and the operating system often cannot timely sense the current application scene of the third party application program, so that the operating system cannot perform targeted system resource adaptation according to the specific application scene of the third party application program.

In order to enable the operating system to distinguish specific application scenes of the third-party application program, data communication between the third-party application program and the operating system needs to be communicated, so that the operating system can acquire current scene information of the third-party application program at any time, and targeted system resource adaptation is performed based on the current scene.

Referring to fig. 13, fig. 13 is a diagram illustrating an architecture of the android operating system in fig. 12. Taking an operating system as an Android system as an example, as shown in fig. 13, a program and data stored in the memory 120 may be stored in the memory 120 with a Linux kernel layer 320, a system runtime library layer 340, an application framework layer 360 and an application layer 380, where the Linux kernel layer 320, the system runtime library layer 340 and the application framework layer 360 belong to an operating system space, and the application layer 380 belongs to a user space. The Linux kernel layer 320 provides the underlying drivers for various hardware of the electronic device, such as display drivers, audio drivers, camera drivers, bluetooth drivers, wi-Fi drivers, power management, and the like. The system runtime layer 340 provides the main feature support for the Android system through some C/c++ libraries. For example, the SQLite library provides support for databases, the OpenGL/ES library provides support for 3D graphics, the Webkit library provides support for browser kernels, and the like. Also provided in the system runtime library layer 340 is An Zhuoyun runtime library (Android runtime), which primarily provides some core libraries that can allow developers to write Android applications using the Java language. The application framework layer 360 provides various APIs that may be used in building applications, which developers can also build their own applications by using, for example, campaign management, window management, view management, notification management, content provider, package management, call management, resource management, location management. At least one application program is running in the application layer 380, and these application programs may be native application programs of the operating system, such as a contact program, a short message program, a clock program, a camera application, etc.; and may also be a third party application developed by a third party developer, such as a game-like application, instant messaging program, photo beautification program, etc.

Referring to FIG. 14, FIG. 14 is an architecture diagram of the IOS system of FIG. 12. Taking an operating system as an IOS system as an example, the programs and data stored in the memory 120 are shown in fig. 14, the IOS system includes: core operating system layer 420 (Core OS layer), core services layer 440 (Core SERVICES LAYER), media layer 460 (MEDIA LAYER), and touchable layer 480 (Cocoa Touch Layer). The core operating system layer 420 includes an operating system kernel, drivers, and underlying program frameworks that provide more hardware-like functionality for use by the program frameworks at the core services layer 440. The core services layer 440 provides system services and/or program frameworks required by the application, such as a Foundation (Foundation) framework, an account framework, an advertisement framework, a data storage framework, a network connection framework, a geographic location framework, a sports framework, and the like. The media layer 460 provides an interface for applications related to audiovisual aspects, such as a graphics-image related interface, an audio technology related interface, a video technology related interface, an audio video transmission technology wireless play (AirPlay) interface, and so forth. The touchable layer 480 provides various commonly used interface-related frameworks for application development, with the touchable layer 480 being responsible for user touch interactions on the electronic device. Such as a local notification service, a remote push service, an advertisement framework, a game tool framework, a message User Interface (UI) framework, a User Interface UIKit framework, a map framework, and so forth.

Among the frameworks shown in fig. 14, frameworks related to most applications include, but are not limited to: a base framework in core services layer 440 and UIKit frameworks in touchable layer 480. The infrastructure provides many basic object classes and data types, providing the most basic system services for all applications, independent of the UI. While the class provided by the UIKit framework is a base UI class library for creating touch-based user interfaces, iOS applications can provide UIs based on the UIKit framework, so it provides the application's infrastructure for building user interfaces, drawing, handling and user interaction events, responding to gestures, and so on.

The manner and principle of implementing data communication between the third party application program and the operating system in the IOS system may refer to the Android system, and this description is not repeated here.

The input device 130 is configured to receive input instructions or data, and the input device 130 includes, but is not limited to, a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is used to output instructions or data, and the output device 140 includes, but is not limited to, a display device, a speaker, and the like. In one example, the input device 130 and the output device 140 may be combined, and the input device 130 and the output device 140 are a touch display screen for receiving a touch operation thereon or thereabout by a user using a finger, a touch pen, or any other suitable object, and displaying a user interface of each application program. Touch display screens are typically provided on the front panel of an electronic device. The touch display screen may be designed as a full screen, a curved screen, or a contoured screen. The touch display screen can also be designed to be a combination of a full screen and a curved screen, and a combination of a special-shaped screen and a curved screen is not limited in this specification.

In addition, those skilled in the art will appreciate that the configuration of the electronic device shown in the above-described figures does not constitute a limitation of the electronic device, and the electronic device may include more or less components than illustrated, or may combine certain components, or may have a different arrangement of components. For example, the electronic device further includes components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WIRELESS FIDELITY, wiFi) module, a power supply, and a bluetooth module, which are not described herein.

In this specification, the execution subject of each step may be the electronic device described above. Optionally, the execution subject of each step is an operating system of the electronic device. The operating system may be an android system, an IOS system, or other operating systems, which is not limited in this specification.

The electronic device of the present specification may further have a display device mounted thereon, and the display device may be various devices capable of realizing a display function, for example: cathode ray tube displays (cathode ray tubedisplay, CR), light-emitting diode displays (light-emitting diode display, LED), electronic ink screens, liquid Crystal Displays (LCD), plasma display panels (PLASMA DISPLAY PANEL, PDP), and the like. A user may utilize a display device on electronic device 101 to view displayed text, images, video, etc. The electronic device may be a smart phone, a tablet computer, a gaming device, an AR (Augmented Reality ) device, an automobile, a data storage, an audio playing device, a video playing device, a notebook, a desktop computing device, a wearable device such as an electronic watch, electronic glasses, an electronic helmet, an electronic bracelet, an electronic necklace, an electronic article of clothing, etc.

In the electronic device shown in fig. 11, where the electronic device may be a terminal, the processor 110 may be configured to invoke the fact assessment model training program stored in the memory 120 and to specifically perform the following operations:

Inputting the sample text pair into an initial fact evaluation model for model training, determining the fact evaluation probability distribution information of the sample text pair based on the initial fact evaluation model, and outputting the fact evaluation result of the sample text pair based on the fact evaluation probability distribution information;

And determining a model loss value based on the facts assessment label and the facts assessment result, and performing model parameter adjustment on the initial facts assessment model by using the model loss value until the initial facts assessment model completes model training to obtain a target facts assessment model.

Optionally, the processor 110 executes a sample text pair with a factual evaluation tag generated based on the reference text, specifically performs:

the same factual data generation processing is carried out on the reference text to obtain a positive text, and the opposite factual data generation processing is carried out on the reference text to obtain a negative text;

generating a positive text pair with a positive facts assessment tag based on the positive text and the reference text, and generating a negative text pair with a negative facts assessment tag based on the negative text and the reference text;

sample text pairs with factual evaluation labels are derived based on the positive text pairs and the negative text pairs.

Optionally, the processor 110 performs the same factual data generation processing on the reference text, and specifically performs when the forward text is obtained:

Optionally, when the processor 110 performs the same factual overwriting process on the reference text based on the forward text generation model to obtain the forward text, the specific implementation is as follows:

Optionally, when the processor 110 performs the process of generating the negative-going text by performing the process of generating the negative-going data on the reference text, the specific steps are as follows:

Alternatively, when the processor 110 performs the process of countering the factual overwriting on the reference text based on the negative text generation model, and obtains the negative text, the specific implementation is as follows:

Optionally, when the processor 110 determines the factual evaluation probability distribution information of the sample pairs based on the initial factual evaluation model, specific execution is performed:

Determining a first probability that the pairs of sample texts are the same factual text based on the initial factual assessment model;

determining a second probability that the pair of sample texts is the contradictory fact text based on the initial fact assessment model;

the factual evaluation probability distribution information is obtained based on the first probability and the second probability.

Alternatively, when the processor 110 executes the output of the factual evaluation result of the sample text pair based on the factual evaluation probability distribution information, specific execution is performed:

Comparing the first probability with the second probability, and outputting a sample forward factual evaluation result when the first probability is greater than or equal to the second probability;

and when the first probability is smaller than the second probability, outputting a sample negative-going facts evaluation result.

Furthermore, the processor 110 may also be configured to invoke a fact evaluation program stored in the memory 120 and specifically perform the following operations:

And obtaining a target text pair based on the reference text and the target generation text, inputting the target text pair into a target fact evaluation model, determining target fact evaluation probability distribution information of the target text pair based on the target fact evaluation model, and outputting a target fact evaluation result of the target text pair based on the target fact evaluation probability distribution information.

Optionally, the processor 110 is further adapted to perform determining a first probability that the target text pair is the same factual text based on the target factual evaluation probability distribution information;

a fact consistency score for the target text pair is determined based on the first probability.

Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, or the like.

It should be noted that, information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, presented data, etc.), and signals according to the embodiments of the present disclosure are all authorized by the user or are fully authorized by the parties, and the collection, use, and processing of relevant data is required to comply with relevant laws and regulations and standards of relevant countries and regions. For example, object features, interactive behavior features, user information, and the like referred to in this specification are all acquired with sufficient authorization.

The foregoing disclosure is only illustrative of the preferred embodiments of the present invention and is not to be construed as limiting the scope of the claims, which follow the meaning of the claims of the present invention.

Claims

1. A facts assessment model training method, the method comprising:

2. The method of claim 1, the generating a sample text pair with a factual evaluation tag based on the reference text, comprising:

Generating a positive text pair with a positive factuality evaluation tag based on the positive text and the reference text, and generating a negative text pair with a negative factuality evaluation tag based on the negative text and the reference text;

3. The method of claim 2, wherein the performing the same factual data generation processing on the reference text to obtain a forward text includes:

4. A method according to claim 3, wherein said subjecting said reference text to the same factual overwriting process based on the forward text generation model to obtain forward text comprises:

and rewriting the reference text based on at least one of the same factual word order rearrangement processing, the same factual active-passive conversion processing, the same factual sentence pattern fusion processing, the same factual text abbreviation processing and the same factual synonymous substitution processing of the forward text generation model to obtain the forward text.

5. The method of claim 2, the performing the contradictory factual data generation processing on the reference text to obtain negative text, comprising:

Performing contradictory factual rewrite processing on the reference text based on a negative text generation model to obtain a negative text; the negative text generation model is obtained by adapting a scene to the contrary to the actual rewriting process based on a second basic large language generation model.

6. The method of claim 5, wherein the performing the negative text-based generation model on the reference text with the negative text-based generation model to perform the negative-going de-facts rewrite process includes:

7. The method of claim 1, the determining factual assessment probability distribution information for the sample pairs based on the initial factual assessment model, comprising:

determining a second probability that the pairs of sample text are contradictory fact text based on the initial fact assessment model;

And obtaining the factual evaluation probability distribution information based on the first probability and the second probability.

8. The method of claim 7, the outputting the factual evaluation results of the sample text pairs based on the factual evaluation probability distribution information, comprising:

Comparing the first probability with the second probability, and outputting a sample forward facts assessment result when the first probability is greater than or equal to the second probability;

And outputting a sample negative-going facts evaluation result when the first probability is smaller than the second probability.

9. A facts assessment method, the method comprising:

10. The method of claim 9, the method further comprising:

determining a first probability that the target text pair is the same factual text based on the target factual evaluation probability distribution information;

11. A factual assessment model training apparatus, the apparatus comprising:

12. A facts assessment device, the device comprising:

13. A computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the method steps of any of claims 1 to 8 or 9 or 10.

14. A computer program product storing at least one instruction for loading by a processor and performing the method steps of any of claims 1-8 or 9 or 10.

15. An electronic device, comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method steps of any of claims 1-8 or 9 or 10.