CN116011570A

CN116011570A - XAI model consistency training method, device, equipment and storage medium

Info

Publication number: CN116011570A
Application number: CN202211685059.XA
Authority: CN
Inventors: 夏正勋; 刘士菖; 汪科; 杨一帆
Original assignee: Nanjing Xinghuan Intelligent Technology Co ltd
Current assignee: Nanjing Xinghuan Intelligent Technology Co ltd
Priority date: 2022-12-27
Filing date: 2022-12-27
Publication date: 2023-04-25

Abstract

The invention discloses an XAI model consistency training method, device, equipment and storage medium. The method comprises the following steps: after Work Model training is completed, inputting a first user data sample of the XAI Model into the Work Model to obtain a prediction decision result; determining a first parameter according to the prediction decision result; after XAI Model training is completed, inputting a decision result of a second user data sample output by the Work Model into the XAI Model to obtain a prediction interpretation result; determining a second parameter according to the prediction interpretation result; determining a target parameter according to the first parameter and the second parameter; and if the target parameter is larger than a parameter threshold, optimizing the gradient direction of the parameter of the XAI Model. The technical scheme of the embodiment of the invention can solve the problem of inconsistent interpretation results of the XAI Model, and can realize optimization of the XAI Model and practical application of the booster XAI technology.

Description

XAI model consistency training method, device, equipment and storage medium

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to an XAI model consistency training method, device, equipment and storage medium.

Background

With the penetration of artificial intelligence applications, people are focusing not only on the capabilities of artificial intelligence, but also on the security credibility of artificial intelligence. XAI (Explainable AI) is a technology capable of providing interpretation for a black box model, and can greatly improve transparency, reliability and controllability of a machine learning model.

However, the inconsistent problem of the XAI model in practical application is to be solved, for example, in a financial trust scene, when the consistency result between the financial trust model result and the trust result considered by the user, the model result for explaining the cause of the trust result and the cause considered by the user is inconsistent, the model result is difficult for the user to trust; when the consistency results of the financial trust model result and the trust result which the user self considers, the model result for explaining the cause of the trust result and the result reference standard of the enterprise are inconsistent, the model result does not accord with the established standard of the enterprise; and when the two final consistency results are inconsistent, namely the consistency results of the user and the enterprise are inconsistent, the financial credit giving system is proved to have 'paradox', and the product standard is not met generally. The essence of the inconsistency problem is that the XAI model results are inconsistent with the real results, the working model results are inconsistent with the real results, and the XAI model results are inconsistent with the reference standard, and the XAI model needs to be optimized continuously.

As can be seen, the inconsistency problem is a big impediment to the implementation of XAI technology in the floor, but there is currently no in-depth research and specific solution to this problem.

Disclosure of Invention

The invention provides an XAI Model consistency training method, device, equipment and storage medium, which are used for solving the problem of inconsistent interpretation results of the XAI Model and realizing optimization of an XAI Model and practical application of a booster XAI technology.

According to an aspect of the present invention, there is provided a method for training consistency of an XAI model, the method comprising:

after Work Model training is completed, inputting a first user data sample of the XAI Model into the Work Model to obtain a prediction decision result;

determining a first parameter according to the prediction decision result;

after XAI Model training is completed, inputting a decision result of a second user data sample output by the Work Model into the XAI Model to obtain a prediction interpretation result;

determining a second parameter according to the prediction interpretation result;

determining a target parameter according to the first parameter and the second parameter;

and if the target parameter is larger than a parameter threshold, optimizing the gradient direction of the parameter of the XAI Model.

According to another aspect of the present invention, there is provided an apparatus for training consistency of XAI model, the apparatus comprising:

The first input module is used for inputting a first user data sample of the XAI Model into the Work Model after the Work Model training is completed, so as to obtain a prediction decision result;

the first determining module is used for determining a first parameter according to the prediction decision result;

the second input module is used for inputting the decision result of the second user data sample output by the Work Model into the XAI Model after the XAI Model training is finished, so as to obtain a prediction interpretation result;

the second determining module is used for determining a second parameter according to the prediction interpretation result;

a third determining module, configured to determine a target parameter according to the first parameter and the second parameter;

and the optimization module is used for optimizing the gradient direction of the parameters of the XAI Model if the target parameters are larger than a parameter threshold.

According to another aspect of the present invention, there is provided an electronic apparatus including:

at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the XAI model consistency training method according to any of the embodiments of the present invention.

According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement the XAI model consistency training method according to any of the embodiments of the present invention when executed.

According to the technical scheme, after Work Model training is completed, a first user data sample of the XAI Model is input into the Work Model to obtain a prediction decision result, a first parameter is determined according to the prediction decision result, after the XAI Model training is completed, a decision result of a second user data sample output by the Work Model is input into the XAI Model to obtain a prediction interpretation result, a second parameter is determined according to the prediction interpretation result, a target parameter is determined according to the first parameter and the second parameter, and if the target parameter is larger than a parameter threshold, the parameters of the XAI Model are optimized along the gradient direction. The technical scheme of the embodiment of the invention can solve the problem of inconsistent interpretation results of the XAI Model, and can realize optimization of the XAI Model and practical application of the booster XAI technology.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a method for consistency training of XAI models provided in accordance with a first embodiment of the invention;

FIG. 2 is a flow chart of another method for XAI model consistency training provided in accordance with a first embodiment of the invention;

FIG. 3 is a schematic structural diagram of an apparatus for training consistency of XAI model according to a second embodiment of the present invention;

fig. 4 is a schematic structural diagram of an electronic device implementing the XAI model consistency training method according to an embodiment of the present invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

It should be noted that the terms "first," "target," and the like in the description and claims of the present invention and in the above-described figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Example 1

Fig. 1 is a flowchart of an XAI model consistency training method according to a first embodiment of the present invention, where the method may be applied to an XAI model consistency training case, and the method may be performed by an XAI model consistency training device, where the XAI model consistency training device may be implemented in hardware and/or software, and where the XAI model consistency training device may be integrated into any electronic device that provides an XAI model consistency training function. As shown in fig. 1, the method includes:

S101, after Work Model training is completed, inputting a first user data sample of the XAI Model into the Work Model to obtain a prediction decision result.

In this embodiment, the Work Model may be a working Model. Specifically, the working model may be any model capable of meeting the requirement of outputting the decision result in the actual application scene, and the specific type of the working model is not limited in this embodiment. For example, in an intelligent customer service scenario, a working model may exist in an intelligent customer service robot system for making decisions on questions posed by a user and outputting the decision results. For example, the user may ask the intelligent customer service: "which product is more durable? The intelligent customer service can output a decision result after making a decision: the "#" product is more durable. "

In this embodiment, the XAI Model may be an interpretable Model. Wherein XAI is an abbreviation of displayable AI, i.e. an AI can be explained. Specifically, the interpretable model may be any model capable of meeting the requirement of outputting an interpretation result in an actual application scenario, and the specific type of the interpretable model is not limited in this embodiment. For example, in an intelligent customer service scenario, an interpretable model may exist in an intelligent customer service robot system for interpreting the problem output interpretation results presented by the user. For example, the user may ask the intelligent customer service: why is the "#" product more durable? The intelligent customer service can output a plurality of interpretation results for interpretation: "because the # product is made of durable material", "because the # product has a perfect after-market service", … …, "because the # product periodically changes parts", etc.

It should be noted that the first user data sample may be sample data in a training set, and may be represented by sample data D.

The predicted decision result may be a Work Model decision result.

Specifically, if the Work Model learning process is not completed (may be determined by using the LOSS or the accuracy as a threshold, which is not described herein in detail), the training of the Work Model is continuously completed. After the Work Model training is completed, if the training stage is the current XAI Model training stage, the sample data D in the training set is used as the input of the Work Model, the label corresponding to the sample data D is marked as T, and the prediction decision result of the Work Model is marked as T'. If the inference process is implemented on the floor after the Model is released, the input of the Work Model may be an input sample S, the predicted decision result of the Work Model is denoted as R, the condition for triggering the XAI Model to continue to optimize in this scenario is usually the interactive feedback of the user, specifically, the interactive feedback of the user may be actively provided by the user, or may also be passively collected by the system, which is not limited in this embodiment, and the feedback of the actual decision result of the user on the input sample S is denoted as R'.

S102, determining a first parameter according to a prediction decision result.

The first parameter may be a parameter for detecting Work Model consistency.

Specifically, after the Work Model training is completed, if the training stage is the current XAI Model training stage, the consistency between T and T 'is checked, and the specific method may be a first parameter ID 1-result=distance (T, T'), where Distance is a Distance function, and may be an existing method, such as L1, L2, a semantic Distance, and the like, and the specific method is selected depending on different types of prediction decision results. If the reasoning process is currently implemented for the floor after the model is released, the consistency between R and R 'can be continuously checked, and the specific method can be that the first parameter loss1=id1-result=distance (R, R'), wherein the Distance method is the same as the above. The final output of this step is ID1-Result.

S103, after XAI Model training is completed, inputting a decision result of the second user data sample output by the Work Model into the XAI Model to obtain a prediction interpretation result.

The second user data sample may be sample data output by the Work Model, and may be represented by sample data S.

The predictive interpretation result may be an interpretation result of the XAI Model.

Specifically, if the XAI Model learning process is not completed (the XAI Model in the present invention is a calculation-based XAI method, such as a linear Model, a tree Model, a deep learning Model, etc., and the optimization and improvement thereof are mainly improved by means of parameter optimization of a function, that is, learning of the XAI Model is completed based on the sample data D, the Work Model output T, and the tag T', which is an existing method, and is not described herein again), training of the XAI Model is continuously completed. When the XAI Model training is completed, the result R of the predictive decision of the Work Model on the sample data S is taken as the input of the XAI Model, and the result of the predictive interpretation of the output of the XAI Model is marked as P.

S104, determining a second parameter according to the prediction interpretation result.

Wherein the second parameter may be a parameter for detecting intrinsic consistency of the XAI Model.

Specifically, the structural comparison learning component a compares P with a reference standard (Rule base) corresponding to the interpretation Result, checks whether P meets the reference standard, and determines a second parameter ID2-Result from a Result 1 or 0 of "whether P meets Rule" (wherein 1 represents compliance and 0 represents non-compliance). In the step, rule base is a reference standard, the form of Rule base can be an experience library or a knowledge graph, the Rule base is an existing system, the system can generally provide a query interface, an interpretation result P 'of the Rule base is input and output or a decision result R' of the Rule base is input and output, and in the step, the decision result R can be input to compare the interpretation results P and P 'or the interpretation results P can be input to compare the decision results R and R' due to the fact that the structure comparison learning component A judges whether P accords with Rule. The final output of this step is the second parameter loss2=id2-result=distance (P, P') or Distance (R, R ").

S105, determining a target parameter according to the first parameter and the second parameter.

In this embodiment, the target parameter may be a loss value of the XAI Model. Specifically, the target parameter may be expressed as LOSS-xai.

In the actual operation process, LOSS value loss12=loss (ID 1-Result, ID 2-Result) between the first parameter and the second parameter, and in this step, the LOSS function may select an implementation of the existing LOSS function according to actual needs, such as MAE LOSS, KL LOSS, and so on. Specifically, the specific calculation mode of the target parameter can be expressed as: target parameter LOSS-xai =loss1+loss2+loss12. In this step, the calculation of LOSS-xai may be performed by adding a predetermined coefficient before LOSS1, 2 and 12, respectively, and adjusting the optimization force by the coefficient or completely shielding the influence of a certain LOSS, which is a common means, which will not be described herein.

And S106, if the target parameter is larger than the parameter threshold, optimizing the gradient direction of the XAI Model parameter.

The parameter threshold may be preset by a user according to actual situations, and the specific numerical value of the parameter threshold is not limited in this embodiment.

Specifically, if the target parameter LOSS-XAI is greater than the parameter threshold, performing inverse calculation, and performing optimization updating of the gradient direction on the parameters of the XAI Model. The reverse calculation in this step is implemented by the existing method, and is not described in detail herein. If the target parameter LOSS-XAI is less than or equal to the parameter threshold, then no reverse calculation is required and no optimization of the parameters of the XAI Model is required.

In addition, for XAI Model that cannot be optimized by gradient, a reinforcement learning framework can be used, and the basic idea is to use a reinforcement learning Actor-Critic framework, which is specifically implemented as follows: optimization of the XAI Model was accomplished according to LOSS-XAI with the XAI Model as the Actor and the Work Model as the Critic, and the optimization method and strategy can use existing techniques.

Optionally, determining the first parameter according to the prediction decision result includes:

And determining a first parameter according to the prediction decision result and the label carried by the first user data sample.

Specifically, after the Work Model training is completed, if the training stage is the current XAI Model training stage, the sample data D in the training set is used as the input of the Work Model, the label corresponding to D is T, the decision Result of the Work Model is T ', the consistency between T and T ' is checked, the specific method is a first parameter ID 1-result=distance (T, T '), wherein the Distance is a Distance function, and the specific method can be the existing methods, such as L1, L2, semantic Distance and the like, and the specific method selects different types depending on the prediction decision Result.

and acquiring decision result feedback information input by a user.

In this embodiment, the decision result feedback information may be actual decision result feedback of the user to the input sample in the input Work Model, which may be denoted as R'.

Specifically, the actual decision result feedback R' of the user to the input sample S in the input Work Model is obtained.

And determining a first parameter according to the predicted decision result and the decision result feedback information.

Specifically, if the current floor implementation reasoning process after Model release is performed, the Work Model input is an input sample S, the decision Result of the Work Model is R, the condition for triggering the XAI Model to continue optimization in this scenario is generally specific to the interactive feedback of the user, the interactive feedback of the user may be provided actively by the user or may also be collected passively by the system, which is not limited in this embodiment, the user may continue to check the consistency between R and R ' if the actual decision Result feedback of the input sample S is R ', and the specific method is loss1=id 1-result=distance (R, R '), where Distance is a Distance function, and may be an existing method, such as L1, L2, semantic Distance, etc., and the specific method selects different types depending on the predicted decision Result.

Optionally, determining the second parameter according to the prediction interpretation result includes:

and acquiring an interpretation result corresponding to the decision result of the second user data sample.

In this embodiment, the interpretation result may be an interpretation result obtained after inputting the decision result of the second user data sample into the XAI Model.

Specifically, after the XAI Model training is completed, the result R of the Work Model decision on the sample data S is taken as the input of the XAI Model, and the output of the XAI Model is the interpretation result P'.

And determining a second parameter according to the interpretation result and the prediction interpretation result corresponding to the decision result of the second user data sample.

Specifically, the second parameter loss2=id2—result=distance (P, P ') may be determined by an interpretation Result P' corresponding to the decision Result of the second user data sample and the prediction interpretation Result P, where Distance is a Distance function, and may be an existing method, such as L1, L2, a semantic Distance, and so on, and a specific method selects different types depending on the prediction decision Result.

and obtaining a decision result corresponding to the prediction interpretation result.

In this embodiment, the decision result may be a Rule base decision result, which may be represented by r″.

Specifically, a decision result R of Rule base is obtained.

And determining a second parameter according to the decision result corresponding to the prediction interpretation result and the decision result of the second user data sample.

Specifically, the second parameter loss2=id2—result=distance (R, R ") may be determined by predicting the decision Result r″ corresponding to the interpretation Result and the decision Result R of the second user data sample, where Distance is a Distance function, and may be an existing method, such as L1, L2, semantic Distance, etc., and a specific method selects different types depending on the predicted decision Result.

Optionally, the XAI model consistency training method further includes:

and acquiring the reason feedback information corresponding to the decision result of the second user data sample.

In this embodiment, the cause feedback information may be cause feedback of the decision result, and may be represented by p″.

Specifically, the cause feedback information P corresponding to the decision result of the second user data sample is obtained.

And determining a third parameter according to the prediction interpretation result and the reason feedback information.

Wherein the third parameter may be a parameter for detecting XAI Model extrinsic consistency.

Specifically, the above steps have completed training the XAI Model, so this stage occurs during the floor-based reasoning process after Model release. The decision result R of the Work Model on the sample data S is taken as input of the XAI Model, and the output of the XAI Model is P, and in this scenario, the condition for triggering the XAI Model to continue to optimize is generally interactive feedback of a user, specifically, the interactive feedback of the user may be provided actively by the user or may be collected passively by the system, which is not limited in this embodiment. The actual decision result feedback to the input sample S is R', the reason feedback to the decision result is P ", and the external consistency includes consistency in two aspects: the decision result and the explanation reasons, wherein the external consistency of the decision result is measured by LOSS1, so that the structure comparison learning component B only needs to check the consistency between P and P' in the step. Specifically, the third parameter loss3=id3—result=distance (P, P "), where Distance is a Distance function, may be an existing method, such as L1, L2, semantic Distance, etc., and the specific method selects different types depending on the prediction decision Result. The final output of this step is ID3-Result.

Optionally, determining the target parameter according to the first parameter and the second parameter includes:

and determining the target parameter according to the first parameter, the second parameter and the third parameter.

Specifically, LOSS value loss12=loss (ID 1-Result, ID 2-Result) between the first parameter and the second parameter, LOSS value loss13=loss (ID 1-Result, ID 3-Result) between the first parameter and the third parameter, and in this step, the LOSS function may select implementation of existing LOSS functions, such as MAE LOSS, KL LOSS, and so on, according to actual needs.

Specifically, the specific calculation mode of the target parameter can be expressed as: target parameter LOSS-xai = loss1+loss2+loss3+loss12+loss13. In this step, the calculation of LOSS-xai may be performed by adding a predetermined coefficient before LOSS1, 2, 3, 12 and 13, respectively, and adjusting the optimizing force by the coefficient or completely shielding the influence of a certain LOSS, which are common means not described herein.

As an exemplary description of an embodiment of the present invention, fig. 2 is a flowchart of another XAI model consistency training method provided according to an embodiment of the present invention. The XAI model consistency training method will now be described in connection with a specific embodiment.

In this embodiment, the enterprise a wants to build a reliable and self-learning XAI system for realizing the intellectualization of the financial trust service, so as to improve the service efficiency and reduce the labor cost. Because the reliability of the rating result is difficult to ensure by the traditional XAI credit rating system, risks of inconsistency between the XAI model result and the real result interpretation, between the working model result and the real result and between the XAI model result and the reference standard exist, and therefore, the enterprise A adopts the XAI model consistency training method to realize financial trust service.

The sample data set Dataset-a is a basic information data set containing a plurality of residents, and is 8000 groups of user data in total, each sample data contains eight parts, d= { index, name, age, generator, income, consume, status, health }, as follows: index (INT 32 integer), name (STRING STRING type, maximum length 20), age (SHORT 16 integer), gender (Boolean type, values 0, 1 represent "men" and "women", respectively), annual income (ten thousand-element) income (INT 32 integer), monthly average consumption count (ten thousand-element) content (INT 32 integer), job stability level stability (SHORT 16 integer), total of 4 values: 1 very stable, 2 relatively stable, 3 relatively unstable, 4 very unstable), body health level health (SHORT 16 integer), total of 4 values: 1 very healthy, 2 relatively healthy, 3 relatively unhealthy, 4 very unhealthy; the label T corresponding to the sample data comprises a credit rating part and a credit rating interpretation part, and is stored in a data set Dataset-B, and 5000 groups of label data are formed. The task targets are financial trust of different citizens, wherein ten layers of credit ratings (SHORT 16 integer 1-10 are shared, and the higher the credit rating is, the higher the layer number is); the FC-Work Model is the FC-Work Model under the scene, and the FC-XAI Model is the FC-XAI Model under the scene. The user interactive input is obtained through an FC-XAI-Sever, the FC-XAI-Sever is an XAI model online learning server, and a user can provide or obtain content to the server through the Internet. DELL EqualLogic PS6100 is used to store the data.

In the implementation, the Work Model and the XAI Model can be models which meet the requirements at will in practical application, and in practical application scenes, the invention has universality and is not limited by specific software and hardware frames or specific Model structures. The method specifically comprises the following steps:

FC-Work Model training. The age, include, consume, status, health variables in the single sample data are selected as inputs to the FC-Work Model. For simplicity of description, a linear regression model y=θ is used in this example ₀ +θ ₁ x ₁ +θ ₂ x ₂ +θ ₃ x ₃ +θ ₄ x ₄ +θ ₅ x ₅ As an FC-Work Model. The parameters are iteratively adjusted by gradient descent to minimize MSE (Mean Square Error ), and training of the FC-Work Model is accomplished by training 1000 sets of sample data each time through multiple iterations (this is a prior art method and will not be described in detail herein).

FC-Work Model consistency detection. When the FC-Work Model training is completed, if the training stage is the current FC-XAI Model training stage, the sample data used for Model training in the stage needs to find the corresponding label data in the database-B. Taking a certain male user data D as an example, taking { age=33, income=50, consume=1, status=2, health=3 } as an input of the FC-Work Model, obtaining a well-trained linear regression Model in the above steps, and calculating a decision result of the FC-Work Model as T' =5. D corresponds to the label t= { T1, T2}, where t1=5 represents the true credit rating and T2 represents the interpretation of the credit rating. The consistency between T and T' is checked below, specifically by calculating the Distance using a Distance function in this example, and normalizing the Result to obtain ID 1-result=0.0. If the inference process is currently implemented for the floor after the Model is released, a certain female user data S is selected as an example, where { age=29, income=18, consume=0.3, status=2, health=1 } in S is used as an input of the FC-Work Model, and the credit rating result of the FC-Work Model is r=6. In this stage, the condition for triggering the FC-XAI Model to continue optimization is usually user interaction feedback, and if the female user logs into the XAI Model online learning server FC-XAI-server of enterprise a through the internet environment, after carefully reading the rating standard, the consistency between R and R ' is continuously checked if the personal credit rating is R ' =5, and the specific method is ID 1-result=distance (R ', R), where the Distance method is the same as above, and the first parameter loss1=id 1-result=1.0 is obtained.

Because the consistency detection method and the loss measurement method of the FC-XAI Model training stage and the FC-XAI Model reasoning stage are the same in the following steps, only the difference of different data sources exists (the training stage is the comparison of a Model result and a data label, and the reasoning stage is the comparison of a Model result and a user interactive feedback), the online learning process of the XAI Model is embodied only by taking the FC-XAI Model reasoning stage as an example.

FC-XAI Model intrinsic consistency detection. This stage is mainly used for consistency measurement between the interpretation result of the FC-XAI Model and the Model design reference formulated by the enterprise A. As a result of the above steps, the FC-XAI Model outputs a credit rating decision R "=5 for a certain female user data S inputted, the user ' S credit rating decision R ' =6, and the 1 st LOSS value is loss1=loss (R ', R") =0.2 by forward calculation. Meanwhile, the FC-XAI Model outputs an interpretation result C [ ] = { C1, C2, C3, C4, C5}, for the credit rating result R "=5, wherein the interpretation factor C1 represents" age-reduced (< 30) ", C2 represents" annual income level >5 ten thousand yuan & <20 ten thousand yuan income=18, ", C3 represents" monthly consumption amount is lower than 5% of annual income, risk resistance is strong ", C4 represents" job-stable medium upper stability=2 ", and C5 represents" health-good health=1 ". Rule Base is a Rule Base, which stores 520 Rule records related to credit rating judgment, and similar to sample data, the Rule Base is stored in a database DELL EqualLogic PS6100. Inputting C [ ] into Rule Base to execute basic Rule reasoning, and according to Rule record { "25 th item: age <30, credit rating <9"," item 26: age >18, credit rating >1"," 128 th item: job stabilization level=2, credit rating <7& >5"," 227: annual revenue <20, credit rating <6"," 229 th item: annual revenue >5, credit rating >1"," item 350: the monthly consumption credit is less than 5% of annual revenue, credit rating >3}, "item 489: health = 1, credit rating >4"} yields an output R '" = 5, which in turn yields a 2 nd LOSS value for FC-XAI Model of loss2=loss (R, R' ") =0.1. Whereas the internal consistency of the FC-XAI Model is calculated by ID 2-result=distance (R ', R' ") to obtain the calculation Result of the second parameter loss2=id2-result=0.8.

FC-XAI Model extrinsic consistency detection. This stage mainly carries out consistency measurement between the interpretation result of the FC-XAI Model and label content or user interaction feedback. As can be seen from the above steps, the FC-XAI Model input is data S, the decision output is R "=5, and the user interactive feedback decision result is R' =6. The female user logs into an XAI model online learning server FC-XAI-Sever of enterprise A through an Internet environment, and feedback on personal credit ratings after carefully reading the rating standards is interpreted as C ' [ ] = { C '1, C '2, C '3, C '4, C '5}, wherein an interpretation factor C '1 represents "age slightly smaller (< 30)", C '2 represents "annual income level 18 ten thousand yuan", C '3 represents "month consumption less than 5% of annual income", C '4 represents "work very stable", and C '5 represents "health degree very good". The external consistency of the FC-XAI Model decision output C' [ ] and C [ ] is calculated as follows: the third parameter loss3=id3-result=distance (C [ ], C' [ ]) =0.4. Extrinsic consistency includes consistency in 2: decision results and explanation reasons in practical implementation, more variations and processing can be performed according to the consistency calculation results of the 2 aspect without departing from the core innovation of the invention.

FC-XAI Model LOSS calculation, i.e., target parameter calculation. For optimization of FC-XAI Model, the interpretation of XAI Model should be consistent in 3 dimensions on user side, model side, rule side, calculated as loss12=loss (ID 1-Result, ID 2-Result) =0.1, loss13=loss (ID 1-Result, ID 3-Reuslt) =0.1. The LOSS calculation for the FC-XAI Model is therefore: target parameter LOSS xai =loss1+loss2+loss3+loss12+loss13=0.9.

The FC-XAI Model compares the output target parameter LOSS-XAI from the previous step to a parameter threshold. In this example, the parameter threshold may be set to 0.0001 in advance according to the model training experience. Since the target parameter LOSS-XAI =0.9 is greater than the parameter threshold value 0.0001, the inverse calculation is performed, and the optimization update along the gradient direction is performed on the parameter of FC-XAI Mode. In other instances, where LOSS-XAI is less than or equal to the parameter threshold, then interpretation of the XAI is illustrated as achieving internal consistency versus external consistency, and no inverse calculations need to be performed, and no optimization of the model is required.

In the actual operation process, different Work Model and XAI Model consistency training methods can be selected according to the form of the decision result in the actual application scene, and the method has strong applicability. Of course, the XAI Model in this embodiment may be applied to various types of user data samples, such as image, text, and table data, by transforming the Work Model and the XAI Model, and the XAI Model in this embodiment has strong versatility.

According to the technical scheme, in the Work Model consistency detection, interactive feedback of a user is obtained based on an online learning server, and a specific calculation method of the Work Model consistency detection is designed according to a form of a decision result. In the step of XAI Model internal consistency detection and XAI Model external consistency detection, a factor structure comparison component A is developed for comparing interpretation of results output by the XAI Model with a corresponding reference standard (Rule base) of interpretation results to determine the XAI Model internal consistency ID2-Result, and a factor structure comparison learning component B is developed for comparing interpretation of results output by the XAI Model with a sample tag or user interactive input to determine the XAI Model external consistency ID3-Result. An XAI online learning management component is developed in the calculation of the target parameters and is used for calculating the overall error loss value of the system, comparing the overall error loss value with a parameter threshold value and determining whether to trigger online learning of different types according to the type of the XAI model. The invention provides a method for solving the interpretation consistency of an XAI model based on online comparison learning.

Example two

Fig. 3 is a schematic structural diagram of an XAI model consistency training device according to a second embodiment of the present invention. As shown in fig. 3, the apparatus includes: a first input module 201, a first determination module 202, a second input module 203, a second determination module 204, a third determination module 205, and an optimization module 206.

The first input module 201 is configured to input a first user data sample of the XAI Model into the Work Model after the Work Model training is completed, so as to obtain a prediction decision result;

a first determining module 202, configured to determine a first parameter according to the prediction decision result;

the second input module 203 is configured to input, after the XAI Model training is completed, a decision result of the second user data sample output by the Work Model into the XAI Model to obtain a prediction interpretation result;

a second determining module 204, configured to determine a second parameter according to the prediction interpretation result;

a third determining module 205, configured to determine a target parameter according to the first parameter and the second parameter;

and an optimization module 206, configured to optimize the gradient direction of the XAI Model parameter if the target parameter is greater than a parameter threshold.

Optionally, the first determining module 202 includes:

And the first determining unit is used for determining a first parameter according to the prediction decision result and the label carried by the first user data sample.

Optionally, the first determining module 202 includes:

the first acquisition unit is used for acquiring decision result feedback information input by a user;

and the second determining unit is used for determining the first parameter according to the predicted decision result and the decision result feedback information.

Optionally, the second determining module 204 includes:

the second acquisition unit is used for acquiring an interpretation result corresponding to the decision result of the second user data sample;

and the third determining unit is used for determining a second parameter according to the interpretation result and the prediction interpretation result corresponding to the decision result of the second user data sample.

Optionally, the second determining module 204 includes:

the third acquisition unit is used for acquiring a decision result corresponding to the prediction interpretation result;

and the fourth determining unit is used for determining a second parameter according to the decision result corresponding to the prediction interpretation result and the decision result of the second user data sample.

Optionally, the XAI model consistency training device further includes:

the acquisition module is used for acquiring the reason feedback information corresponding to the decision result of the second user data sample;

And the fourth determining module is used for determining a third parameter according to the prediction interpretation result and the reason feedback information.

Optionally, the third determining module 205 includes:

and a fifth determining unit configured to determine a target parameter according to the first parameter, the second parameter, and the third parameter.

The XAI model consistency training device provided by the embodiment of the invention can execute the XAI model consistency training method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

Example III

Fig. 4 shows a schematic diagram of an electronic device 30 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 4, the electronic device 30 includes at least one processor 31, and a memory, such as a Read Only Memory (ROM) 32, a Random Access Memory (RAM) 33, etc., communicatively connected to the at least one processor 31, wherein the memory stores a computer program executable by the at least one processor, and the processor 31 can perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM) 32 or the computer program loaded from the storage unit 38 into the Random Access Memory (RAM) 33. In the RAM 33, various programs and data required for the operation of the electronic device 30 may also be stored. The processor 31, the ROM 32 and the RAM 33 are connected to each other via a bus 34. An input/output (I/O) interface 35 is also connected to bus 34.

Various components in electronic device 30 are connected to I/O interface 35, including: an input unit 36 such as a keyboard, a mouse, etc.; an output unit 37 such as various types of displays, speakers, and the like; a storage unit 38 such as a magnetic disk, an optical disk, or the like; and a communication unit 39 such as a network card, modem, wireless communication transceiver, etc. The communication unit 39 allows the electronic device 30 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

The processor 31 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 31 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 31 performs the various methods and processes described above, such as the XAI model consistency training method:

determining a first parameter according to the prediction decision result;

In some embodiments, the XAI model consistency training method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 30 via the ROM 32 and/or the communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the XAI model consistency training method described above may be performed. Alternatively, in other embodiments, the processor 31 may be configured to perform the XAI model consistency training method in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.

The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims

1. A method for consistency training of an XAI model, comprising:

after Work Model training is completed, inputting a first user data sample of the XAIodel into the Work Model to obtain a prediction decision result;

determining a first parameter according to the prediction decision result;

2. The method of claim 1, wherein determining a first parameter based on the predictive decision result comprises:

3. The method of claim 1, wherein determining a first parameter based on the predictive decision result comprises:

acquiring decision result feedback information input by a user;

4. The method of claim 1, wherein determining a second parameter based on the predictive interpretation comprises:

acquiring an interpretation result corresponding to the decision result of the second user data sample;

5. The method of claim 1, wherein determining a second parameter based on the predictive interpretation comprises:

Obtaining a decision result corresponding to the prediction interpretation result;

6. The method according to claim 4 or 5, further comprising:

acquiring reason feedback information corresponding to a decision result of the second user data sample;

7. The method of claim 6, wherein determining a target parameter from the first parameter and the second parameter comprises:

and determining a target parameter according to the first parameter, the second parameter and the third parameter.

8. An apparatus for training consistency of XAI models, comprising:

9. An electronic device, the electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the XAI model consistency training method of any of claims 1-7.

10. A computer readable storage medium storing computer instructions for causing a processor to implement the XAI model consistency training method of any of claims 1-7 when executed.