WO2022171037A1 - Method and apparatus for interpreting artificial intelligence model, and system - Google Patents

Abstract

A method and apparatus for interpreting an artificial intelligence model, and a system. The method comprises: acquiring a first interpretation result of an artificial intelligence model, wherein the first interpretation result is an interpretation result represented on the basis of a feature field of the artificial intelligence model; acquiring an association relationship between a feature field and a service meaning; replacing a feature field in the first interpretation result with an associated service meaning on the basis of the association relationship, so as to obtain a second interpretation result of the artificial intelligence model; and generating an interpretation report of the artificial intelligence model on the basis of the second interpretation result.

Description

Method, apparatus and system for interpreting artificial intelligence models

This disclosure claims the priority of the Chinese patent application with the application number 202110176552.8 and the application title "Method, Apparatus and System for Interpreting Artificial Intelligence Models" filed with the China Patent Office on February 09, 2021, the entire contents of which are by reference Incorporated in this disclosure.

technical field

The present disclosure relates to the technical field of artificial intelligence, and more particularly, to a method, apparatus and system for interpreting an artificial intelligence model.

Background technique

With the continuous development of big data and artificial intelligence technology, artificial intelligence models have been widely used, but the depth and complexity of the models are far beyond the scope of human understanding, or black boxes. When an AI model generalizes well, it can be cross-validated to verify its accuracy and apply it in a production environment. However, it is difficult for users to perceive which important features are considered by this model when making decisions, and what judgment logic is based on which to make the final decision.

In many scenarios, users who use artificial intelligence models need to clarify the decision-making basis of the artificial intelligence models. In addition, explaining the artificial intelligence model can also help users to further improve the artificial intelligence model, optimize the characteristics of the artificial intelligence model, and improve the generalization of the artificial intelligence model.

Therefore, it is very valuable to propose a scheme that can explain the artificial intelligence model.

SUMMARY OF THE INVENTION

An object of the present disclosure is to provide a new technical solution for interpreting artificial intelligence models.

According to a first aspect of the present disclosure, there is provided a method for interpreting an artificial intelligence model, comprising: obtaining a first interpretation result of the artificial intelligence model; the first interpretation result is obtained based on a feature field of the artificial intelligence model The interpretation result of the representation; obtain the association relationship between the feature field and the business meaning; based on the association relationship, replace the feature field in the first interpretation result with the associated business meaning, and obtain the artificial intelligence model. A second interpretation result; based on the second interpretation result, an interpretation report of the artificial intelligence model is generated.

According to a second aspect of the present disclosure, there is provided an apparatus for interpreting an artificial intelligence model, comprising: a first interpretation result obtaining module configured to obtain a first interpretation result of the artificial intelligence model; the first interpretation result is Based on the interpretation results represented by the feature fields of the artificial intelligence model; an association relationship acquisition module is configured to obtain the association relationship between the feature fields and business meanings; a second interpretation result generation module is configured based on the association relationship. , replace the feature field in the first interpretation result with the associated business meaning, and obtain the second interpretation result of the artificial intelligence model; the interpretation report generation module is configured to generate, based on the second interpretation result, Interpretation report of the artificial intelligence model.

According to a third aspect of the present disclosure, there is provided a system comprising at least one computing device and at least one storage device, wherein the at least one storage device is configured to store instructions that are stored by the at least one computing device At runtime, the at least one computing device is caused to perform the method of the first aspect of the present disclosure.

According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the method of the first aspect of the present disclosure.

Through the method of the present disclosure, the association relationship between the feature field and the business meaning in the first interpretation result of the artificial intelligence model is obtained; based on the obtained association relationship, the feature field that the user cannot understand in the first interpretation result is replaced with an association The second interpretation result based on the business meaning is obtained; the interpretation report of the artificial intelligence model is generated based on the second interpretation result based on the business meaning and presented to the user, and the user can clearly understand the artificial intelligence model through the explanation report. The basis of various prediction results improves the user's ability to understand the artificial intelligence model products, and at the same time, it can also optimize and adjust the business process based on the decision rules of the artificial intelligence model in the interpretation report.

Other features of the present disclosure and advantages thereof will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure.

1 is a block diagram of one example of a hardware configuration of an electronic device that can be used to implement embodiments of the present disclosure;

2 is a schematic flowchart of a method for interpreting an artificial intelligence model according to an embodiment of the present disclosure;

3 is a schematic flowchart of an example of a method for explaining an artificial intelligence model according to an embodiment of the present disclosure;

4 is a block schematic diagram of an apparatus for interpreting an artificial intelligence model according to an embodiment of the present disclosure;

5 is a block schematic diagram of a system according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application or uses in any way.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques, methods, and apparatus should be considered part of the specification.

In all examples shown and discussed herein, any specific values should be construed as illustrative only and not limiting. Accordingly, other instances of the exemplary embodiment may have different values.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further discussion in subsequent figures.

Hereinafter, various embodiments and examples according to embodiments of the present disclosure will be described with reference to the accompanying drawings.

<Hardware configuration>

FIG. 1 is a block diagram showing a hardware configuration of an electronic device 1000 that can implement an embodiment of the present disclosure.

The electronic device 1000 may be a laptop computer, a desktop computer, a cell phone, a tablet computer, or the like. As shown in FIG. 1 , the electronic device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and the like. The processor 1100 may be a central processing unit CPU, a microprocessor MCU, or the like. The memory 1200 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a nonvolatile memory such as a hard disk, and the like. The interface device 1300 includes, for example, a USB interface, an earphone interface, and the like. The communication device 1400 is capable of, for example, wired or wireless communication, and may specifically include Wifi communication, Bluetooth communication, 2G/3G/4G/5G communication, and the like. The display device 1500 is, for example, a liquid crystal display, a touch display, or the like. The input device 1600 may include, for example, a touch screen, a keyboard, a somatosensory input, and the like. The user can input/output voice information through the speaker 1700 and the microphone 1800 .

The electronic device shown in FIG. 1 is merely illustrative and in no way imply any limitation on the present disclosure, its application, or use. Applied to the embodiments of the present disclosure, the memory 1200 of the electronic device 1000 is used to store instructions, and the instructions are used to control the processor 1100 to operate to execute any one of the methods provided by the embodiments of the present disclosure. Those skilled in the art should understand that although a plurality of apparatuses are shown for the electronic device 1000 in FIG. 1 , the present disclosure may only relate to some of the apparatuses, for example, the electronic apparatus 1000 only relates to the processor 1100 and the storage apparatus 1200. A skilled person can design instructions according to the solutions disclosed in the present disclosure. How the instruction controls the processor to operate is well known in the art, so it will not be described in detail here.

<Method Example>

In this embodiment, a method for interpreting an artificial intelligence model is provided. The method for interpreting an artificial intelligence model may be implemented by an electronic device. The electronic device may be the electronic device 1000 shown in FIG. 1 .

As shown in FIG. 2 , the method for explaining an artificial intelligence model in this embodiment may include the following steps S2100 to S2400:

Step S2100, obtaining a first interpretation result of the artificial intelligence model; the first interpretation result is an interpretation result represented by a feature field based on the artificial intelligence model.

The artificial intelligence model in this embodiment may be specified by the user according to their own needs. The framework of the artificial intelligence model may be Tensorflow, Spark, Lightgbm, Xgboost, Sklearn, Pytorch, Mxnet, PaddlePaddle, etc., which is not limited in this embodiment.

The first interpretation result in this embodiment may include an analysis result of the importance of the feature field and/or a decision rule. The importance analysis result of the feature field may include the feature field and the reference weight of the feature field when the artificial intelligence model makes a decision.

In the embodiment in which the artificial intelligence model is a credit card fraud determination model, the feature fields of the credit card fraud determination model may include “amt_10d_sum”, “amt_20d_sum”, “amt_30d_sum”, “Income”, “Age”, and “Sex”, etc. Feature fields, which are not listed one by one here. The importance analysis results of some feature fields of the credit card fraud determination model are shown in Table 1.

Table 1:

Feature	Weight
amt_10d_sum	0.004
amt_20d_sum	0.003
amt_30d_sum	0.001
Income	0.5
Age	0.2
Sex	0.2

In an embodiment of the present disclosure, a training sample for training an artificial intelligence model may be obtained, and by analyzing the training sample, a characteristic field of the artificial intelligence model may be extracted, and the characteristic field of the artificial intelligence model may be at least 1, the number of feature fields is not limited here.

When the artificial intelligence model is used for decision analysis of the samples to be explained, the artificial intelligence model will extract the eigenvalues of each feature field from the samples to be explained, and obtain the decision results of the samples to be explained based on the eigenvalues of each feature field.

In an embodiment of the present disclosure, the reference weight of each feature field of the artificial intelligence model may be explained by an interpretability method.

In the embodiment where the first interpretation result includes the importance analysis result of the feature field, the LIME (Local Interpretable Model-Agnostic Explanations) algorithm can be used to interpret the artificial intelligence model, and the importance analysis of each feature field when interpreting the interpretation sample is obtained. result.

The main idea of LIME is to use interpretability models (such as linear models, decision trees) to approximate the predictions of artificial intelligence models locally, rather than globally.

Specifically, acquiring the importance analysis result of the feature field of the artificial intelligence model includes steps S2111-S2118:

Step S2111, acquiring the feature fields of the artificial intelligence model.

Specifically, a training sample for training an artificial intelligence model may be obtained, and feature fields of the artificial intelligence model may be extracted by analyzing the training sample.

Step S2112, select samples to be explained from the acquired training sample set of the artificial intelligence model; the samples to be explained include feature values of feature fields.

In this embodiment, the sample to be explained may be a training sample randomly selected from the training sample set, or may be a training sample selected from the training sample set according to a preset selection rule. The preset selection rules may be set in advance according to application scenarios or specific requirements. For example, the samples to be explained selected from the training sample set according to the preset selection rules may be the newly generated training samples in the training sample set as the samples to be explained, or the training samples in which the feature value of the specified feature field is selected as the specified value. The sample is used as the sample to be explained, and the training sample with the specified label can also be selected as the sample to be explained.

Step S2113, based on the sample to be interpreted, generate a local sample and a sample weight of the local sample; the local sample includes the feature value of the feature field.

In an embodiment of the present disclosure, step S2113 generates local samples and sample weights of local samples based on the samples to be interpreted, including:

Transform the samples to be interpreted according to preset transformation rules to obtain local samples.

The preset transformation rules in this embodiment may be set in advance according to application scenarios or specific requirements. Different types of samples to be interpreted use different sample transformation methods. For the to-be-interpreted samples of the type of text data, the preset transformation rule may be to transform individual words one by one; for the to-be-interpreted samples of the type structured data, the preset transformation rule may be to change the feature value of one feature field individually each time . The number of local samples is at least one, and the number of local samples is not limited here. The local samples are obtained by transforming the samples to be explained. Therefore, the local samples and the samples to be explained have the same feature fields, but the feature values of the feature fields are different.

In this embodiment, the similarity between the local sample and the sample to be explained may be determined as the sample weight of the local sample.

In this embodiment, the local samples and the samples to be explained can also be mapped to the vector space, and the distance between the local samples and the samples to be explained in the vector space is calculated; based on the distance between the local samples and the samples to be explained in the vector space, the sample weights. The closer the vector of the local sample is to the vector of the sample to be explained, the greater the sample weight corresponding to the local sample. Calculate the sample weight of the local samples, so that when training the interpretation model, the importance of the local samples is taken into account, so as to ensure that the interpretation model fits the local samples that are closer to the sample to be explained. A good approximation of the decision-making behavior of an artificial intelligence model near the sample to be explained.

Step S2114, input the local samples into the artificial intelligence model to obtain the decision results of the local samples.

Step S2115 , perform machine learning training based on the local samples, the sample weights, and the decision results of the local samples, to obtain an explanation model for approximately fitting the artificial intelligence model.

In this embodiment, different interpretation models may be considered as local approximate models of the artificial intelligence model, such as linear regression models, decision tree models, and Bayesian network models.

The interpretation model is trained based on local samples, and the local samples have the same feature fields as the samples to be explained. Therefore, the interpretation model trained based on the local samples has the same feature fields as the artificial intelligence model.

After machine learning training, the coefficients corresponding to each feature field in the explanation model can be obtained.

In step S2116, the coefficient corresponding to the feature field in the interpretation model is used as the reference weight of the feature field when the artificial intelligence model makes a decision.

The interpretation model can approximately simulate the decision-making behavior of the artificial intelligence model near the sample to be explained. Therefore, the coefficients corresponding to each feature field in the interpretation model can be approximated as the reference weight of each feature field when the artificial intelligence model makes a decision on the sample to be explained.

Step S2117, sort the feature fields in descending order based on the reference weights of the feature fields to obtain the second sorting value of the feature fields.

Step S2118: Obtain an importance analysis result from the feature fields with the second sorting value within the second preset sorting range and the corresponding reference weights.

The second preset sorting range in this embodiment may be set in advance according to application scenarios or specific requirements. For example, the second preset sorting range may be [1, 5], that is, the feature fields with the second sorting values of 1, 2, 3, 4, and 5 and the corresponding reference weights are used as the importance analysis result.

There are a lot of feature fields in the artificial intelligence model. Taking the artificial intelligence model used for credit card fraud judgment as an example, 120,000 feature fields can be extracted from the training sample set of the artificial intelligence model for credit card fraud judgment. When explaining, the user only pays attention to the feature fields with larger reference weights. Therefore, the feature fields are sorted in descending order based on the reference weights of the feature fields, and then the feature fields whose second sorting value is in the second preset sorting range are screened out. Sexual analysis results.

In addition to the LIME algorithm, the importance analysis results of the feature fields of the artificial intelligence model can also be obtained through SHAP (SHapley Additive exPlanation), and the interpretability method is not limited here.

In an embodiment of the present disclosure, the first interpretation result further includes a decision rule of the artificial intelligence model.

Obtain the decision rules of the artificial intelligence model, including steps S2121-S2124:

Step S2121, acquiring a training sample set of the artificial intelligence model.

The training sample set in this embodiment may include multiple training samples.

Step S2122, input the training sample set into the artificial intelligence model to obtain the decision result of the training sample set; the decision result includes the probability of the decision result appearing.

The decision result of the training sample set in this embodiment may be the decision result of each training sample in the training sample set.

Taking an artificial intelligence model for credit card fraud determination as an example, the decision result of one of the training samples may be: 80% fraudulent, that is, the probability that the training sample is determined to be fraudulent is 80%.

Step S2123, perform machine learning training according to the training sample set and the decision result of the training sample set, and obtain a single decision tree model.

There are two types of nodes in a single decision tree model: internal nodes and leaf nodes. The internal node represents a feature field, and the leaf node represents a decision result. When classifying, start from the root node, test a feature field of the instance, and assign the instance to its child nodes according to the test result; at this time, each child node corresponds to a feature value of the feature field. This recursively moves downward until a leaf node is reached, and finally the instance is assigned to the decision result of the leaf node.

Step S2124, extracting the splitting condition corresponding to at least one decision result branch in the single decision tree model, to obtain the decision rule of the artificial intelligence model.

A decision tree can be regarded as a set of decision rules: a decision rule is constructed from each path from the root node of the decision tree to the leaf node; the feature fields of the internal nodes on the path correspond to the conditions of the decision rule, and the leaf nodes The dots correspond to the decision results. The path of a decision tree is equivalent to the set of decision rules on its corresponding path, that is, each instance is covered by one path or one decision rule, and only by one decision rule.

Taking the artificial intelligence model for credit card fraud determination as an example, a decision rule obtained according to the splitting condition corresponding to a decision result branch in a single decision tree model can be: Income>10w, and age>20y, and Sex -male, it is judged as non-fraud.

On the basis of this embodiment, the method further includes steps S2131-S2138:

Step S2131, acquiring feature fields in the artificial intelligence model.

For details, reference may be made to the aforementioned step S2111, which will not be repeated here.

Step S2132, select the sample to be explained in the training sample set of the acquired artificial intelligence model; the sample to be explained includes the feature value of the feature field.

For details, reference may be made to the aforementioned step S2112, which will not be repeated here.

In step S2133, the sample to be explained is input into a single decision tree model, and the decision result of the sample to be explained is obtained.

In this embodiment, taking an artificial intelligence model for credit card fraud determination as an example, the decision result of the sample to be explained may be: 80% fraud, that is, the probability that the sample to be explained is determined to be fraud is 80%.

Step S2134, traverse the feature fields.

Step S2135: Transform the feature value of the feature field currently traversed in the sample to be interpreted to obtain the transformed sample corresponding to the feature field currently traversed.

In an embodiment of the present disclosure, for each feature field, a corresponding target value may be set in advance according to an application scenario or specific requirements, and the feature value of the feature field currently traversed in the sample to be interpreted is transformed into The corresponding target value is obtained, and the transformed sample corresponding to the currently traversed feature field is obtained.

In another embodiment of the present disclosure, the target value of the corresponding feature field may be determined according to the feature value of each feature field in the training samples other than the sample to be explained in the training sample set, and the current The eigenvalues of the traversed feature fields are transformed into corresponding target values, and the transformed samples corresponding to the currently traversed feature fields are obtained. Specifically, it can be determined that the eigenvalue of the feature field currently traversed is different from the sample to be explained, and the feature value of the feature field is used as the target value of the feature field; The average value of the feature values of the traversed feature fields is used as the target value of the currently traversed feature fields.

Taking the artificial intelligence model for credit card fraud determination as an example, the current traversal feature field is Income, the feature value corresponding to the Income feature field in the sample to be explained is 5W, and the target value corresponding to the Income feature field can be 10W, then, it can be It is to replace the 5W corresponding to the Income feature field in the sample to be explained with 10W, and the feature values of other feature fields remain unchanged to obtain the transformed sample corresponding to the Income feature field.

Step S2136, input the transformed sample into a single decision tree model to obtain the decision result of the transformed sample.

In this embodiment, taking the artificial intelligence model for credit card fraud determination as an example, the decision result of the transformed sample corresponding to the Income feature field is: fraud 30%, that is, the probability that the transformed sample is determined to be fraud is 30%.

Step S2137: Determine the difference between the probabilities in the decision results of the transformed sample and the sample to be explained, as the reference weight of the currently traversed feature field.

Taking an artificial intelligence model for credit card fraud determination as an example, the difference between the probability in the transformed sample corresponding to the currently traversed feature field Income and the probability in the decision result of the sample to be explained is 50%, Then the reference weight of the currently traversed feature field Income is 50%, that is, 0.5.

Step S2138, when the traversal is completed, the importance analysis result is obtained based on the feature field and the reference weight of the feature field.

In this embodiment, all feature fields and the reference weights of the feature fields may be directly used as the importance analysis result; or the importance analysis result may be obtained by referring to the aforementioned steps S2117 and S2118.

Step S2200, acquiring the association relationship between the feature field and the business meaning.

The association relationship in this embodiment may be set in advance according to an application scenario or specific requirements.

In one example, the relationship may be stored as an excel file or a csv file or a txt file. The association relationship in the file can be in the form of a list, for example, the first column can be the feature field of the artificial intelligence model, the second column can be the business meaning, and the feature field and the corresponding business meaning are in one-to-one correspondence.

Taking an artificial intelligence model for credit card fraud determination as an example, the relationship between some feature fields and business meanings can be as shown in Table 2.

Table 2:

Feature field	Business Implications
amt_10d_sum	Transaction amount in the 10 days before the transaction
amt_20d_sum	Transaction amount in the 20 days before the transaction
amt_30d_sum	Transaction amount 30 days before the transaction
Income	Annual income
Age	age
Sex	gender

Step S2300, based on the association relationship, replace the feature field in the first interpretation result with the associated business meaning to obtain the second interpretation result of the artificial intelligence model.

Taking the artificial intelligence model for credit card fraud determination as an example, when the importance analysis results are shown in Table 1, and the relationship between feature fields and business meanings is shown in Table 2, the second interpretation result of the feature fields is Part of the content of the importance analysis results can be as shown in Table 3.

table 3:

Feature field	Reference weight
Transaction amount in the 10 days before the transaction	0.004
Transaction amount in the 20 days before the transaction	0.003
Transaction amount 30 days before the transaction	0.001
Annual income	0.5
age	0.2
gender	0.2

In the embodiment where the decision rule in the first interpretation result is Income>10w, age>20y, and Sex-male, it is determined as non-fraud, the content of the decision rule in the second interpretation result may be: annual income>10W , and age > 20y, and gender - male, it is determined as non-fraud.

Step S2400, based on the second interpretation result, generate an interpretation report of the artificial intelligence model.

In an embodiment of the present disclosure, step S2400 generates an interpretation report of the artificial intelligence model based on the second interpretation result, which may include steps S2410-S2430:

Step S2410, obtain a reference report.

The reference report may be pre-set according to an application scenario or specific requirements, and pre-stored in the electronic device executing this embodiment.

Step S2420, using natural language processing tools to learn the text structure and writing paradigm in the reference report.

Among them, the text structure refers to the content contained in the reference report and the order in which the content is organized. Paradigm refers to the way in which specific vocabulary and grammatical structures are used in the reference report.

In this embodiment, the text structure and writing paradigm in the learning reference report include: what content should be included in the learning reference report, such as: model name, algorithm name, scene name, important feature name, important feature weight, important rules, etc.; After determining what content needs to be included, you need to learn the order of organization of the content in the reference report. For example, when generating the report content of important features, it will give priority to expressing "what time", "what model", "what data set", and then expressing "there is" What are the important features”, and finally express the “weight of features”; learn how to use the proprietary vocabulary and grammatical structures in the reference report.

Step S2430, based on the learned text structure and writing paradigm, generate an interpretation report of the artificial intelligence model according to the second interpretation result.

In this embodiment, the method of generating the interpretation report of the artificial intelligence model may include: synthesizing at least one sentence according to the text structure of the reference report with the content in the second interpretation result; synthesizing the combined sentence according to the grammatical structure learned in the reference report Add connectives between various information to form a complete sentence; according to the proprietary vocabulary learned in the reference report, further revise the vocabulary in the sentence to generate the final explanation report.

In an embodiment of the present disclosure, the method may further include:

Obtain the descriptive information of the artificial intelligence model; and integrate the descriptive information of the artificial intelligence model into the explanation report of the artificial intelligence model.

The descriptive information in this embodiment may include at least one of the following: the name of the model, the description of the usage of the model, the application scenario of the model, the framework adopted by the model, the algorithm adopted by the model, the accuracy of the model, and the like. Taking the artificial intelligence model for credit card fraud determination as an example, the descriptive information of the model is as follows:

The name of the model: Credit Card Fraud Determination-2020 Model;

Description of the purpose of the model: determine whether a credit card transaction is fraudulent;

The application scenario of the model: two classification;

The framework adopted by the model: GDBT;

The algorithm used in the model: logistic regression;

The size of the training dataset: 500,000 rows and 100 columns;

Accuracy of the model: 98%.

Taking the artificial intelligence model for credit card fraud determination as an example, the generated explanation report can be as follows:

The credit card fraud determination-2020 version model converts the problem of determining whether a credit card transaction has fraudulent behavior into a binary classification problem of artificial intelligence model decision-making scenarios. Applying the logistic regression algorithm in the GDBT framework, by analyzing the historical data of the training sample set with a total of 500,000 rows and 100 columns, an accuracy rate of 98% is achieved.

Analyze the historical data of the credit card fraud determination-2020 version model training sample set, and extract 120,000 feature fields of the artificial intelligence model, of which 3 feature fields have higher reference weights, and 3 feature fields with higher reference weights and the corresponding reference weights can be as shown in Table 4.

Table 4:

Feature field	Reference weight
Annual income	0.5
age	0.2
gender	0.2

And extract the following decision-making rules: annual income > 100,000, and age > 20 years old, and the gender is male, it is judged as non-fraud.

In the analysis of a single case, since the sample to be explained has an annual income > 100,000, an age > 20 years, and a male gender, it is determined that the credit card transaction corresponding to the sample to be explained is not fraudulent.

Based on the above report, business personnel can clearly understand which important features are considered by the Credit Card Fraud Determination-2020 model when making decisions, and what are the rules for the model to make decisions, so that business personnel can clearly explain the decisions of the AI model to customers rule. At the same time, according to the decision rules of the artificial intelligence model in the report, the business personnel can also judge whether there is fraud in a credit card transaction according to the decision rules of the Credit Card Fraud Judgment-2020 Model in the actual business execution process. The interpretation results of the model are applied to subsequent business processes to generate real business value.

In an embodiment of the present disclosure, before step S2400 is performed, the method further includes: optimizing the second interpretation result.

On the basis of this embodiment, based on the second interpretation result, generating an interpretation report of the artificial intelligence model may include:

According to the optimized second interpretation result, an interpretation report of the artificial intelligence model is generated.

In an embodiment of the present disclosure, optimizing the second interpretation result includes steps S2310-S2340:

Step S2310, construct an explanation graph based on the content corresponding to the importance analysis result in the second explanation result; wherein, the nodes in the explanation graph include business meaning and preset business type; the edges of the explanation graph represent the difference between the business meaning and the business type. mapping relationship between.

Taking the artificial intelligence model for credit card fraud determination as an example, one of the business type nodes is "transaction amount", and some business meaning nodes include "transaction amount in the first 10 days of the transaction", "transaction amount in the first 20 days of the transaction" and "transaction amount". Transaction amount in the previous 30 days"; the mapping relationship between the two nodes represented by the edge connecting the "transaction amount" node and the "sum of transaction amount in the previous 10 days" node is "the past 10 days, sum"; The mapping relationship between the two nodes represented by the edge connecting the "transaction amount" node "the sum of the transaction amount in the first 20 days of the transaction" node is "the past 20 days, sum"; in which the "transaction amount" node " The sum of the transaction amount in the first 30 days of the transaction" The mapping relationship between the two nodes represented by the edge connected by the nodes is "the past 30 days, sum", that is, the interpretation graph is obtained.

Step S2320, according to the interpretation map, determine at least one service meaning connected to each service type.

In the above example of the explanation graph, "transaction amount" as a business type node connects three business meaning nodes, namely: "sum of transaction amount in the first 10 days of the transaction", "sum of transaction amount in the first 20 days of the transaction" and "Sum of transaction amount for the first 30 days of the transaction".

Step S2330, for each service type, sum the reference weights of the connected service meanings to obtain the reference weight of the corresponding service type.

In the example of the aforementioned interpretation graph, the reference weight of the business meaning "the sum of the transaction amount in the first 10 days of the transaction" is 0.004, the reference weight of the business meaning "the sum of the transaction amount in the first 20 days of the transaction" is 0.003, and the business meaning "the sum of the transaction amount in the first 30 days of the transaction" is 0.003. The reference weight of "sum of daily transaction amount" is 0.001, then, the reference weight corresponding to the business type of "transaction amount" can be: 0.004+0.003+0.001=0.008.

Step S2340, according to the service type and the reference weight of the service type, obtain the content corresponding to the importance analysis result in the optimized second interpretation result.

In an embodiment of the present disclosure, all service types and the reference weight of each service type may be used as content corresponding to the importance analysis result in the optimized second interpretation result.

In another embodiment of the present disclosure,

According to the service type and the reference weight of the service type, the content corresponding to the importance analysis result in the optimized second interpretation result is obtained, including steps S2341-S2342:

Step S2341 , sort the service types in descending order based on the reference weights of the service types, and obtain the first sorting value of each service type.

Step S2342: Use the service types and corresponding reference weights with the first ranking value within the first preset ranking range as the content corresponding to the importance analysis result in the optimized second interpretation result.

The first preset sorting range in this embodiment may be set in advance according to application scenarios or specific requirements. For example, the first preset sorting range may be [1, 3], that is, the feature fields with the first sorting values of 1, 2, and 3 and the corresponding reference weights are used as the importance analysis result.

There may be a large number of service types in this embodiment. When interpreting the artificial intelligence model, the user may only pay attention to the service types with larger reference weights. Therefore, each service type is sorted in descending order based on the reference weights of the feature fields, and then The business types with the first ranking value in the first preset ranking range are filtered out, and the content corresponding to the importance analysis result in the optimized second interpretation result is generated.

In an example, part of the optimized second interpretation result corresponding to the importance analysis result is shown in Table 5.

table 5:

Feature field	Reference weight
Amount of the transaction	0.008
Annual income	0.5
age	0.2
gender	0.2

<Example>

3 is a schematic flowchart of an example of a method for interpreting an artificial intelligence model according to an embodiment of the present disclosure.

As shown in Figure 3, the method may include:

Step S3001, obtaining a training sample set of the artificial intelligence model;

Step S3002, input the training sample set into the artificial intelligence model to obtain the decision result of the training sample set; the decision result includes the probability of the decision result appearing;

Step S3003, performing machine learning training according to the training sample set and the decision result of the training sample set, to obtain a single decision tree model;

Step S3004, extracting the splitting condition corresponding to at least one decision result branch in the single decision tree model, to obtain the decision rule of the artificial intelligence model.

Step S3005, acquiring feature fields in the artificial intelligence model;

Step S3006, select a sample to be explained in the training sample set of the acquired artificial intelligence model; the sample to be explained includes the feature value of the feature field;

Step S3007, input the sample to be explained into a single decision tree model, and obtain the decision result of the sample to be explained;

Step S3008, traverse feature fields;

Step S3009, transform the feature value of the feature field currently traversed in the sample to be interpreted, and obtain the transformed sample corresponding to the feature field currently traversed;

Step S3010, input the transformed sample into a single decision tree model to obtain a decision result of the transformed sample;

Step S3011, determining the difference between the probabilities in the decision result of the transformed sample and the sample to be explained, as the reference weight of the currently traversed feature field;

Step S3012, when the traversal ends, obtain an importance analysis result based on the feature field and the reference weight of the feature field.

The decision rules and importance analysis results of the artificial intelligence model are the first interpretation results.

Step S3013, acquiring the association relationship between the feature field and the business meaning;

Step S3014, based on the association relationship, replace the feature field in the decision rule and the feature field importance analysis result with the associated business meaning, and obtain the decision rule and feature field importance analysis result based on the business meaning representation.

The decision rule and feature field importance analysis result based on the business meaning representation is the second interpretation result.

Step S3015, constructing an interpretation graph based on the analysis result of the importance of the feature fields represented by the business meaning; wherein, the nodes in the interpretation graph include the business meaning and the preset business type; the edges of the interpretation graph represent the mapping between the business meaning and the business type relation;

Step S3016, according to the interpretation map, determine at least one service meaning connected to each service type;

Step S3017, for each service type, sum the reference weights of the connected service meanings to obtain the reference weight of the corresponding service type;

Step S3018, sort the service types in descending order based on the reference weights of the service types, to obtain the first sorting value of each service type;

In step S3019, the service types with the first sorting value within the first preset sorting range and the corresponding reference weights are used as the optimized feature field importance analysis result.

Step S3020, obtaining a reference report;

Step S3021, using natural language processing tools to learn the text structure and writing paradigm in the reference report;

Step S3022, based on the learned text structure and writing paradigm, the descriptive information of the artificial intelligence model, the decision rules and the optimized feature field importance analysis results are obtained to generate an explanation report of the artificial intelligence model.

<Apparatus Example>

In this embodiment, an apparatus 4000 for interpreting an artificial intelligence model is provided, as shown in FIG. 4 , including: a first interpretation result acquisition module 4100 , an association relationship acquisition module 4200 , a second interpretation result generation module 4300 and Explain report generation module 4400. The first interpretation result acquisition module 4100 is configured to acquire the first interpretation result of the artificial intelligence model; the first interpretation result is the interpretation result represented by the feature field based on the artificial intelligence model; the association relationship acquisition module 4200 is configured In order to obtain the association relationship between the feature field and the business meaning; the second interpretation result generation module 4300 is configured to replace the feature field in the first interpretation result with the associated business meaning based on the association relationship, and obtain the second result of the artificial intelligence model. Interpretation result; the interpretation report generation module 4300 is configured to generate an interpretation report of the artificial intelligence model based on the second interpretation result.

In an embodiment of the present disclosure, the first interpretation result includes an importance analysis result of the feature field;

The importance analysis results include feature fields and their reference weights when the AI model makes decisions.

In an embodiment of the present disclosure, it also includes:

an optimization module, configured to optimize the second interpretation result;

The explanation report generating module is further configured to generate an explanation report of the artificial intelligence model according to the optimized second explanation result.

In one embodiment of the present disclosure, the optimization module is configured to:

An explanation graph is constructed based on the content corresponding to the importance analysis result in the second explanation result; the nodes in the explanation graph include business meanings and preset business types; the edges of the explanation graph represent the mapping between business meanings and business types relation;

According to the interpretation map, determine at least one business meaning connected to each business type;

For each service type, sum the reference weights of the connected service meanings to obtain the reference weight of the corresponding service type;

According to the business type and the reference weight of the business type, the content corresponding to the importance analysis result in the optimized second interpretation result is obtained.

In one embodiment of the present disclosure, the optimization module is configured to:

Sort the business types in descending order based on the reference weights of the business types, and obtain the first sorting value of each business type;

The service types and corresponding reference weights with the first ranking value within the first preset ranking range are taken as the content corresponding to the importance analysis result in the optimized second interpretation result.

In an embodiment of the present disclosure, the first interpretation result obtaining module is configured to:

Get the feature fields in the artificial intelligence model;

Select the samples to be explained in the training sample set of the acquired artificial intelligence model; the samples to be explained include the feature values of the feature fields;

Based on the samples to be explained, generate local samples and sample weights of the local samples; the local samples include the feature values of the feature fields;

Input the local samples into the artificial intelligence model to obtain the decision results of the local samples;

Carry out machine learning training based on local samples, sample weights, and decision results of local samples, and obtain an explanation model configured to approximate the artificial intelligence model;

The coefficient corresponding to the feature field in the interpretation model will be used as the reference weight of the feature field when the artificial intelligence model makes a decision;

Sort the feature fields in descending order based on the reference weights of the feature fields to obtain the second sorting value of the feature fields;

The feature fields with the second ranking value within the second preset ranking range and the corresponding reference weights are used as the results of the importance analysis.

In an embodiment of the present disclosure, the first interpretation result obtaining module is configured to:

Transform the samples to be interpreted according to the preset transformation rules to obtain local samples;

Determine the similarity between the local sample and the sample to be explained as the sample weight of the local sample.

In an embodiment of the present disclosure, the first interpretation result further includes a decision rule of the artificial intelligence model;

The first interpretation result acquisition module is also configured as:

Obtain the training sample set of the artificial intelligence model;

Input the training sample set into the artificial intelligence model to get the decision result of the training sample set; the decision result includes the probability of the decision result appearing;

Perform machine learning training according to the training sample set and the decision results of the training sample set to obtain a single decision tree model;

The splitting conditions corresponding to at least one decision result branch in the single decision tree model are extracted, and the decision rules of the artificial intelligence model are obtained.

In an embodiment of the present disclosure, the first interpretation result obtaining module is further configured to:

Get the feature fields in the artificial intelligence model;

Select the samples to be explained in the training sample set of the acquired artificial intelligence model; the samples to be explained include the feature values of the feature fields;

Input the sample to be explained into a single decision tree model to obtain the decision result of the sample to be explained;

Traverse feature fields;

Transform the feature value of the feature field currently traversed in the sample to be interpreted, and obtain the transformed sample corresponding to the feature field currently traversed;

Input the transformed sample into a single decision tree model to obtain the decision result of the transformed sample;

Determine the difference between the probability in the decision result of the transformed sample and the sample to be explained, as the reference weight of the feature field currently traversed;

When the traversal ends, the importance analysis result is obtained based on the feature field and the reference weight of the feature field.

In one embodiment of the present disclosure, the interpretation report generation module is configured to:

obtain reference reports;

Use natural language processing tools to learn text structure and writing paradigms in reference reports;

Based on the learned text structure and writing paradigm, the interpretation report of the artificial intelligence model is generated according to the second interpretation result.

In an embodiment of the present disclosure, it also includes:

a descriptive information acquisition module, configured to acquire descriptive information of the artificial intelligence model;

The explanation report generation module is configured to integrate the descriptive information of the artificial intelligence model into the explanation report of the artificial intelligence model.

Those skilled in the art should understand that the apparatus 4000 for interpreting an artificial intelligence model can be implemented in various ways. For example, the apparatus 4000 for interpreting an artificial intelligence model may be implemented by configuring a processor with instructions. For example, the instructions may be stored in ROM, and when the device is started, the instructions may be read from the ROM into a programmable device to implement the apparatus 4000 for interpreting an artificial intelligence model. For example, the apparatus 4000 for interpreting an artificial intelligence model can be built into a dedicated device (eg, an ASIC). The apparatus 4000 for interpreting an artificial intelligence model may be divided into mutually independent units, or they may be implemented by combining them together. The apparatus 4000 for interpreting an artificial intelligence model may be implemented by one of the above-mentioned various implementation manners, or may be implemented by a combination of two or more of the above-mentioned various implementation manners.

In this embodiment, the apparatus 4000 for interpreting an artificial intelligence model may have various implementation forms. For example, the apparatus 4000 for interpreting an artificial intelligence model may be run in any software product or application that provides interpretable model services function modules, or peripheral embedded parts, plug-ins, patches, etc. of these software products or applications, or these software products or applications themselves.

<System Example>

In this embodiment, as shown in FIG. 5 , a system 5000 including at least one computing device 5100 and at least one storage device 5200 is also provided. The at least one storage device 5200 is configured to store executable instructions; the instructions, when executed by the at least one computing device, cause the at least one computing device 5100 to perform a method according to any embodiment of the present disclosure.

In this embodiment, the system 5000 may be a mobile phone, a tablet computer, a palmtop computer, a desktop computer, a notebook computer, a workstation, a game console, etc., or a distributed system composed of multiple devices.

<Computer-readable storage medium>

In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, and when the computer program is executed by a processor, implements the method according to any embodiment of the present disclosure.

The present disclosure may be an apparatus, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present disclosure.

A computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically coded devices, such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above. Computer-readable storage media, as used herein, are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.

The computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for carrying out operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages. Source or object code, written in any combination, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the "C" language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect). In some embodiments, custom electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can be personalized by utilizing state information of computer readable program instructions. Computer readable program instructions are executed to implement various aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are all equivalent.

Various embodiments of the present disclosure have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.

Industrial Applicability

Through the embodiment of the present disclosure, the association relationship between the feature field and the business meaning in the first interpretation result of the artificial intelligence model is obtained; based on the obtained association relationship, the feature field that the user cannot understand in the first interpretation result is replaced with an association The second interpretation result based on the business meaning is obtained; the interpretation report of the artificial intelligence model is generated based on the second interpretation result based on the business meaning and presented to the user, and the user can clearly understand the artificial intelligence model through the explanation report. The basis of various prediction results improves the user's ability to understand the artificial intelligence model products, and at the same time, it can also optimize and adjust the business process based on the decision rules of the artificial intelligence model in the interpretation report. Therefore, the present disclosure has strong industrial applicability.

Claims (34)

1. A method for interpreting artificial intelligence models, including:

   Obtain a first interpretation result of the artificial intelligence model; the first interpretation result is an interpretation result represented by a feature field based on the artificial intelligence model;

   Obtain the association relationship between the feature field and the business meaning;

   Based on the association relationship, replace the feature field in the first interpretation result with the associated business meaning to obtain the second interpretation result of the artificial intelligence model;

   Based on the second interpretation result, an interpretation report of the artificial intelligence model is generated.
2. The method according to claim 1, wherein the first interpretation result comprises an importance analysis result of the feature field;

   The importance analysis result includes the feature field and the reference weight of the feature field when the artificial intelligence model makes a decision.
3. The method of claim 2, wherein,

   Before generating the interpretation report of the artificial intelligence model based on the second interpretation result, the method further includes: optimizing the second interpretation result;

   The generating an explanation report of the artificial intelligence model based on the second explanation result includes:

   According to the optimized second interpretation result, an interpretation report of the artificial intelligence model is generated.
4. The method of claim 3, wherein the optimizing the second interpretation result comprises:

   An explanation graph is constructed based on the content corresponding to the importance analysis result in the second explanation result; wherein, the nodes in the explanation graph include the business meaning and the preset business type; the edges of the explanation graph Indicates the mapping relationship between the business meaning and the business type;

   According to the interpretation map, determine at least one service meaning connected to each service type;

   For each of the service types, the reference weights of the connected service meanings are summed to obtain the reference weights of the corresponding service types;

   According to the service type and the reference weight of the service type, the content corresponding to the importance analysis result in the optimized second interpretation result is obtained.
5. The method according to claim 4, wherein obtaining the content corresponding to the importance analysis result in the optimized second interpretation result according to the service type and the reference weight of the service type, comprising: :

   Sorting the service types in descending order based on the reference weights of the service types to obtain a first ranking value of each of the service types;

   The service types and corresponding reference weights with the first ranking value within the first preset ranking range are taken as the content corresponding to the importance analysis result in the optimized second interpretation result.
6. The method according to any one of claims 2 to 5, wherein acquiring the importance analysis result of the feature field of the artificial intelligence model comprises:

   obtaining the feature field in the artificial intelligence model;

   Select the sample to be explained in the acquired training sample set of the artificial intelligence model; the sample to be explained includes the feature value of the feature field;

   generating a local sample and a sample weight of the local sample based on the to-be-interpreted sample; the local sample includes the feature value of the feature field;

   Inputting the local samples into the artificial intelligence model to obtain the decision results of the local samples;

   Perform machine learning training based on the local samples, the sample weights and the decision results of the local samples, to obtain an explanation model for approximately fitting the artificial intelligence model;

   Taking the coefficient corresponding to the feature field in the interpretation model as the reference weight of the feature field when the artificial intelligence model makes a decision;

   Sort the feature fields in descending order based on the reference weights of the feature fields to obtain a second sorting value of the feature fields;

   The feature fields with the second sorting value within the second preset sorting range and the corresponding reference weights are used as the importance analysis result.
7. The method according to claim 6, wherein the generating a partial sample and a sample weight of the partial sample based on the to-be-interpreted sample comprises:

   Transform the to-be-interpreted sample according to a preset transformation rule to obtain the local sample;

   The similarity between the local samples and the to-be-interpreted samples is determined as a sample weight of the local samples.
8. The method according to any one of claims 2 to 7, wherein the first interpretation result further comprises a decision rule of the artificial intelligence model;

   Obtain the decision rules of the artificial intelligence model, including:

   obtaining a training sample set of the artificial intelligence model;

   Inputting the training sample set into the artificial intelligence model to obtain a decision result of the training sample set; the decision result includes the probability of the decision result occurring;

   Carry out machine learning training according to the training sample set and the decision result of the training sample set to obtain a single decision tree model;

   Extracting the splitting condition corresponding to at least one decision result branch in the single decision tree model, to obtain the decision rule of the artificial intelligence model.
9. The method of claim 8, further comprising:

   obtaining the feature field in the artificial intelligence model;

   Select the sample to be explained in the acquired training sample set of the artificial intelligence model; the sample to be explained includes the feature value of the feature field;

   Inputting the sample to be explained into the single decision tree model to obtain the decision result of the sample to be explained;

   traverse the feature field;

   Transform the feature value of the feature field currently traversed in the to-be-interpreted sample to obtain a transformed sample corresponding to the feature field currently traversed;

   Inputting the transformed sample into the single decision tree model to obtain the decision result of the transformed sample;

   determining the difference between the probabilities in the decision result of the transformed sample and the sample to be explained, as the reference weight of the currently traversed feature field;

   When the traversal ends, the importance analysis result is obtained based on the feature field and the reference weight of the feature field.
10. The method according to any one of claims 1 to 9, wherein the generating an explanation report of the artificial intelligence model based on the second explanation result comprises:

    obtain reference reports;

    Use natural language processing tools to learn the text structure and writing paradigm in the reference report;

    Based on the learned text structure and writing paradigm, an interpretation report of the artificial intelligence model is generated according to the second interpretation result.
11. The method according to any one of claims 1 to 10, wherein, further comprising:

    obtain descriptive information of the artificial intelligence model;

    Integrate the descriptive information of the artificial intelligence model into the interpretation report of the artificial intelligence model.
12. An apparatus for interpreting artificial intelligence models, including:

    a first interpretation result acquisition module, configured to acquire a first interpretation result of the artificial intelligence model; the first interpretation result is an interpretation result represented by a feature field based on the artificial intelligence model;

    an association relationship obtaining module, configured to obtain the association relationship between the feature field and the business meaning;

    A second interpretation result generating module, configured to replace the feature field in the first interpretation result with the associated business meaning based on the association relationship, to obtain a second interpretation result of the artificial intelligence model;

    An interpretation report generating module is configured to generate an interpretation report of the artificial intelligence model based on the second interpretation result.
13. The apparatus of claim 12, wherein the first interpretation result comprises an importance analysis result of the feature field;

    The importance analysis result includes the feature field and the reference weight of the feature field when the artificial intelligence model makes a decision.
14. The apparatus of claim 13, further comprising:

    an optimization module configured to optimize the second interpretation result;

    The interpretation report generating module is further configured to generate an interpretation report of the artificial intelligence model according to the optimized second interpretation result.
15. The apparatus of claim 14, wherein the optimization module is configured to:

    An explanation graph is constructed based on the content corresponding to the importance analysis result in the second explanation result; wherein, the nodes in the explanation graph include the business meaning and the preset business type; the edges of the explanation graph Indicates the mapping relationship between the business meaning and the business type;

    According to the interpretation map, determine at least one service meaning connected to each service type;

    For each of the service types, the reference weights of the connected service meanings are summed to obtain the reference weights of the corresponding service types;

    According to the service type and the reference weight of the service type, the content corresponding to the importance analysis result in the optimized second interpretation result is obtained.
16. The apparatus of claim 15, wherein the optimization module is configured to:

    Sorting the service types in descending order based on the reference weights of the service types to obtain a first ranking value of each of the service types;

    The service types and corresponding reference weights with the first ranking value within the first preset ranking range are taken as the content corresponding to the importance analysis result in the optimized second interpretation result.
17. The apparatus according to any one of claims 13 to 16, wherein the first interpretation result acquisition module is configured to:

    obtaining the feature field in the artificial intelligence model;

    Select the sample to be explained in the acquired training sample set of the artificial intelligence model; the sample to be explained includes the feature value of the feature field;

    Based on the to-be-interpreted sample, a partial sample and a sample weight of the partial sample are generated; the partial sample includes the feature value of the feature field;

    Inputting the local samples into the artificial intelligence model to obtain the decision results of the local samples;

    Perform machine learning training based on the local samples, the sample weights and the decision results of the local samples, to obtain an explanation model for approximately fitting the artificial intelligence model;

    Taking the coefficient corresponding to the feature field in the interpretation model as the reference weight of the feature field when the artificial intelligence model makes a decision;

    Sort the feature fields in descending order based on the reference weights of the feature fields to obtain a second sorting value of the feature fields;

    The feature fields with the second sorting value within the second preset sorting range and the corresponding reference weights are used as the result of the importance analysis.
18. The apparatus according to claim 17, wherein the first interpretation result obtaining module is configured to:

    Transform the to-be-interpreted sample according to a preset transformation rule to obtain the local sample;

    The similarity between the local samples and the to-be-interpreted samples is determined as a sample weight of the local samples.
19. An apparatus according to any one of claims 12 to 18, wherein,

    The first interpretation result further includes the decision rule of the artificial intelligence model;

    The first interpretation result acquisition module is configured to:

    obtaining a training sample set of the artificial intelligence model;

    Inputting the training sample set into the artificial intelligence model to obtain a decision result of the training sample set; the decision result includes the probability of the decision result occurring;

    Carry out machine learning training according to the training sample set and the decision result of the training sample set to obtain a single decision tree model;

    Extracting the splitting condition corresponding to at least one decision result branch in the single decision tree model, to obtain the decision rule of the artificial intelligence model.
20. The apparatus of claim 19, wherein the first interpretation result acquisition module is configured to:

    obtaining the feature field in the artificial intelligence model;

    Select the sample to be explained in the acquired training sample set of the artificial intelligence model; the sample to be explained includes the feature value of the feature field;

    Inputting the sample to be explained into the single decision tree model to obtain the decision result of the sample to be explained;

    traverse the feature field;

    Transform the feature value of the feature field currently traversed in the to-be-interpreted sample to obtain a transformed sample corresponding to the feature field currently traversed;

    Inputting the transformed sample into the single decision tree model to obtain the decision result of the transformed sample;

    determining the difference between the probabilities in the decision result of the transformed sample and the sample to be explained, as the reference weight of the currently traversed feature field;

    When the traversal ends, the importance analysis result is obtained based on the feature field and the reference weight of the feature field.
21. The apparatus of any one of claims 12 to 20, wherein the interpretation report generation module is configured to:

    obtain reference reports;

    Use natural language processing tools to learn the text structure and writing paradigm in the reference report;

    Based on the learned text structure and writing paradigm, an interpretation report of the artificial intelligence model is generated according to the second interpretation result.
22. The apparatus of any one of claims 12 to 21, further comprising:

    a descriptive information acquisition module, configured to acquire descriptive information of the artificial intelligence model;

    The explanation report generation module is configured to integrate the descriptive information of the artificial intelligence model into an explanation report of the artificial intelligence model.
23. A system comprising at least one computing device and at least one storage device, wherein the at least one storage device is configured to store instructions that, when executed by the at least one computing device, cause the at least one computing device Perform the following steps for explaining the artificial intelligence model:

    Obtain a first interpretation result of the artificial intelligence model; the first interpretation result is an interpretation result represented by a feature field based on the artificial intelligence model;

    Obtain the association relationship between the feature field and the business meaning;

    Based on the association relationship, replace the feature field in the first interpretation result with the associated business meaning to obtain the second interpretation result of the artificial intelligence model;

    Based on the second interpretation result, an interpretation report of the artificial intelligence model is generated.
24. The system of claim 23, wherein the first interpretation result comprises an importance analysis result of the feature field;

    The importance analysis result includes the feature field and the reference weight of the feature field when the artificial intelligence model makes a decision.
25. 25. The system of claim 24, wherein the instructions, when executed by the at least one computing device, further cause the at least one computing device to perform the steps of:

    optimizing the second interpretation result;

    The generating an explanation report of the artificial intelligence model based on the second explanation result includes:

    According to the optimized second interpretation result, an interpretation report of the artificial intelligence model is generated.
26. 26. The system of claim 25, wherein the instructions, when executed by the at least one computing device, further cause the at least one computing device to perform the steps of:

    An explanation graph is constructed based on the content corresponding to the importance analysis result in the second explanation result; wherein, the nodes in the explanation graph include the business meaning and the preset business type; the edges of the explanation graph representing the mapping relationship between the business meaning and the business type; according to the interpretation map, determine at least one business meaning connected to each business type;

    For each of the service types, the reference weights of the connected service meanings are summed to obtain the reference weights of the corresponding service types;

    According to the service type and the reference weight of the service type, the content corresponding to the importance analysis result in the optimized second interpretation result is obtained.
27. 27. The system of claim 26, wherein the instructions, when executed by the at least one computing device, further cause the at least one computing device to perform the steps of:

    Sorting the service types in descending order based on the reference weights of the service types to obtain a first ranking value of each of the service types;

    The service types and corresponding reference weights with the first ranking value within the first preset ranking range are taken as the content corresponding to the importance analysis result in the optimized second interpretation result.
28. 27. The system of any one of claims 24 to 27, wherein the instructions, when executed by the at least one computing device, further cause the at least one computing device to perform the steps of:

    obtaining the feature field in the artificial intelligence model;

    Select the sample to be explained in the acquired training sample set of the artificial intelligence model; the sample to be explained includes the feature value of the feature field;

    generating a local sample and a sample weight of the local sample based on the to-be-interpreted sample; the local sample includes the feature value of the feature field;

    Inputting the local samples into the artificial intelligence model to obtain the decision results of the local samples;

    Perform machine learning training based on the local samples, the sample weights and the decision results of the local samples, to obtain an explanation model for approximately fitting the artificial intelligence model;

    Taking the coefficient corresponding to the feature field in the interpretation model as the reference weight of the feature field when the artificial intelligence model makes a decision;

    Sort the feature fields in descending order based on the reference weights of the feature fields to obtain a second sorting value of the feature fields;

    The feature fields with the second sorting value within the second preset sorting range and the corresponding reference weights are used as the importance analysis result.
29. The system of claim 28, wherein the instructions, when executed by the at least one computing device, further cause the at least one computing device to perform the steps of:

    Transform the to-be-interpreted sample according to a preset transformation rule to obtain the local sample;

    The similarity between the local samples and the to-be-interpreted samples is determined as a sample weight of the local samples.
30. The system of any one of claims 24 to 29, wherein the first interpretation result further comprises decision rules for the artificial intelligence model;

    The instructions, when executed by the at least one computing device, further cause the at least one computing device to perform the following steps:

    obtaining a training sample set of the artificial intelligence model;

    Inputting the training sample set into the artificial intelligence model to obtain a decision result of the training sample set; the decision result includes the probability of the decision result occurring;

    Carry out machine learning training according to the training sample set and the decision result of the training sample set to obtain a single decision tree model;

    Extracting the splitting condition corresponding to at least one decision result branch in the single decision tree model, to obtain the decision rule of the artificial intelligence model.
31. The system of claim 30, wherein the instructions, when executed by the at least one computing device, further cause the at least one computing device to perform the steps of:

    obtaining the feature field in the artificial intelligence model;

    Select the sample to be explained in the acquired training sample set of the artificial intelligence model; the sample to be explained includes the feature value of the feature field;

    Inputting the sample to be explained into the single decision tree model to obtain the decision result of the sample to be explained;

    traverse the feature field;

    Transform the feature value of the feature field currently traversed in the to-be-interpreted sample to obtain a transformed sample corresponding to the feature field currently traversed;

    Inputting the transformed sample into the single decision tree model to obtain the decision result of the transformed sample;

    determining the difference between the probabilities in the decision result of the transformed sample and the sample to be explained, as the reference weight of the currently traversed feature field;

    When the traversal ends, the importance analysis result is obtained based on the feature field and the reference weight of the feature field.
32. 31. The system of any one of claims 23 to 31, wherein the instructions, when executed by the at least one computing device, further cause the at least one computing device to perform the steps of:

    obtain reference reports;

    Use natural language processing tools to learn the text structure and writing paradigm in the reference report;

    Based on the learned text structure and writing paradigm, an interpretation report of the artificial intelligence model is generated according to the second interpretation result.
33. 32. The system of any one of claims 23 to 32, wherein the instructions, when executed by the at least one computing device, further cause the at least one computing device to perform the steps of:

    obtain descriptive information of the artificial intelligence model;

    Integrate the descriptive information of the artificial intelligence model into the interpretation report of the artificial intelligence model.
34. A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any one of claims 1 to 11.

Images