CN111768040A - Model interpretation method, device, equipment and readable storage medium - Google Patents
Model interpretation method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN111768040A CN111768040A CN202010621838.8A CN202010621838A CN111768040A CN 111768040 A CN111768040 A CN 111768040A CN 202010621838 A CN202010621838 A CN 202010621838A CN 111768040 A CN111768040 A CN 111768040A
- Authority
- CN
- China
- Prior art keywords
- interpretation
- model
- result
- target
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/03—Credit; Loans; Processing thereof
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Economics (AREA)
- Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- Accounting & Taxation (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Development Economics (AREA)
- Finance (AREA)
- Software Systems (AREA)
- Tourism & Hospitality (AREA)
- Data Mining & Analysis (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Technology Law (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Entrepreneurship & Innovation (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Game Theory and Decision Science (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a model interpretation method, a device, equipment and a readable storage medium, wherein the method comprises the following steps: interpreting the model prediction result of the target user by adopting a preset interpretation algorithm to obtain an interpretation result; generating a plurality of interpretation text contents according to the interpretation result, and analyzing each interpretation text content by adopting an acceptance judging model to obtain the corresponding acceptability of each interpretation text content, wherein the acceptance judging model is obtained by training on the basis of the feedback data of the historical user on the historical interpretation text content; and selecting target explanation text contents from the explanation text contents based on the acceptability and outputting the target explanation text contents. The invention improves the practical application effect of model interpretation.
Description
Technical Field
The invention relates to the technical field of machine learning, in particular to a model interpretation method, a device, equipment and a readable storage medium.
Background
With the rapid development of machine learning, the academic world provides a series of high-performance machine learning models, including support vector machines, random forests, gradient lifting trees, deep learning and the like. The mathematical indexes of the complex models are generally better than simple models such as decision trees, logistic regression and the like, but the internal principle is difficult to understand. However, in many business applications, it is necessary to explain the model, i.e. why the model outputs a certain prediction, for example in banking credit business, it is sometimes necessary to explain to the person being denied a loan why the loan application is denied. The model interpretation technology is in the initial stage of development, the current model interpretation method is generally derived through theoretical assumption or mathematical methods, the result of model interpretation belongs to the estimation result and has certain credibility, but in the actual application scene, the result cannot be accepted by an interpreter necessarily, namely the actual application effect of model interpretation is not good.
Disclosure of Invention
The invention mainly aims to provide a model interpretation method, a model interpretation device, model interpretation equipment and a readable storage medium, and aims to solve the problems that the current model interpretation result belongs to an estimation result and has certain credibility, but can not be accepted by an interpreter necessarily in an actual application scene, namely the actual application effect of model interpretation is poor.
To achieve the above object, the present invention provides a model interpretation method, comprising the steps of:
interpreting the model prediction result of the target user by adopting a preset interpretation algorithm to obtain an interpretation result;
generating a plurality of interpretation text contents according to the interpretation result, and analyzing each interpretation text content by adopting an acceptance judging model to obtain the corresponding acceptability of each interpretation text content, wherein the acceptance judging model is obtained by training on the basis of the feedback data of the historical user on the historical interpretation text content;
and selecting target explanation text contents from the explanation text contents based on the acceptability and outputting the target explanation text contents.
Optionally, the preset interpretation algorithm at least includes one interpretation algorithm, and the step of interpreting the model prediction result of the target user by using the preset interpretation algorithm to obtain the interpretation result includes:
respectively adopting each interpretation algorithm to interpret the model prediction result of the target user to obtain the interpretation result corresponding to each interpretation algorithm;
the step of generating a plurality of interpretation text contents according to the interpretation result comprises:
and respectively combining each interpretation result with a preset interpretation text template to generate the interpretation text content corresponding to each interpretation result, wherein the preset interpretation text template comprises a plurality of interpretation text templates.
Optionally, the interpretation result includes a contribution degree corresponding to each feature to be interpreted of the target user, the feature to be interpreted is a preset non-sensitive feature or a preset important feature, and the step of generating a plurality of interpretation text contents according to the interpretation result includes:
selecting the feature to be explained with the contribution degree larger than a preset contribution degree from the features to be explained as a target feature;
and combining the target characteristics and a plurality of preset interpretation text templates corresponding to the target characteristics respectively to generate each interpretation text content.
Optionally, after the step of selecting and outputting the target explanatory text content from the explanatory text contents based on the acceptability, the method further includes:
receiving a feedback instruction triggered based on the target interpretation text content, and extracting user feedback data in the feedback instruction;
taking the user characteristic data of the target user and the target interpretation text content as input characteristic data, taking the user feedback data as label data, and taking the input characteristic data and the label data as a piece of training data to be added to a model training data set;
and when a model optimization instruction is received, performing optimization training on the acceptance judging model based on the existing training data in the model training data set so as to update the acceptance judging model.
Optionally, after the step of selecting and outputting the target explanatory text content from the explanatory text contents based on the acceptability, the method further includes:
receiving a feedback instruction triggered based on the target explanation text content, extracting user feedback data in the feedback instruction, and adding the user feedback data to a feedback data set, wherein the user feedback data at least comprises an acceptance result of the target explanation text content;
and calculating the proportion of the feedback data with a positive result in the existing feedback data based on the existing feedback data in the feedback data set, and outputting the proportion as the accuracy of model interpretation.
Optionally, after the step of calculating, based on the existing feedback data in the feedback data set, a ratio of feedback data in which an acceptance result is a positive result in the existing feedback data, and outputting the ratio as an accuracy of model interpretation, the method further includes:
detecting whether the accuracy is smaller than a preset threshold value;
and if the accuracy is smaller than the preset threshold value, triggering a model optimization instruction of the acceptance judging model.
Optionally, before the step of interpreting the model prediction result of the target user by using a preset interpretation algorithm to obtain an interpretation result, the method further includes:
acquiring data to be predicted of the target user;
inputting the data to be predicted into a preset risk credit model to obtain a model prediction result;
outputting the model prediction result, and detecting a reason checking instruction aiming at the model prediction result;
and when the reason checking instruction is detected, executing the step of adopting a preset interpretation algorithm to interpret the model prediction result of the target user to obtain an interpretation result.
To achieve the above object, the present invention provides a model interpretation apparatus comprising:
the interpretation module is used for interpreting the model prediction result of the target user by adopting a preset interpretation algorithm to obtain an interpretation result;
the generating module is used for generating a plurality of interpretation text contents according to the interpretation result and analyzing each interpretation text content by adopting an acceptance judging model to obtain the corresponding acceptability of each interpretation text content, wherein the acceptance judging model is obtained by training the feedback data of the historical interpretation text content based on the historical user;
and the output module is used for selecting target explanation text contents from the explanation text contents based on the acceptability and outputting the target explanation text contents.
To achieve the above object, the present invention also provides a model interpretation apparatus, comprising: a memory, a processor and a model interpreter stored on the memory and executable on the processor, the model interpreter when executed by the processor implementing the steps of the model interpretation method as described above.
Furthermore, to achieve the above object, the present invention also proposes a computer readable storage medium having stored thereon a model interpreter, which when executed by a processor implements the steps of the model interpretation method as described above.
According to the method, an interpretation algorithm is adopted to interpret a model prediction result of a target user to obtain an interpretation result, a plurality of interpretation text contents are generated according to the interpretation result, an acceptance degree discrimination model obtained based on historical feedback data training is adopted to analyze and obtain the corresponding acceptance degree of each interpretation text content, and then the target interpretation text content is selected from each interpretation text content according to the acceptance degree and is output. The explanation result is output in the form of explaining text content, so that the reason for applying a theoretical explanation algorithm to an actual scene to visualize a user is explained; and the acceptability of the user to various explanation text contents is learned by the acceptability judging model trained on the basis of the historical feedback data, so that the acceptability of the various explanation text contents is analyzed by adopting the acceptability judging model, and the proper target explanation text contents are selected and output on the basis of the acceptability, thereby improving the acceptability of the user to the visualized reason explanation, namely improving the reliability and the explanation quality of the model explanation, and further improving the practical application effect of the model explanation. Moreover, by automatically explaining the model prediction result and generating the explanation result into intuitive explanation text content, the labor cost can be saved as much as possible in a specific business scene, and the business processing efficiency is improved.
Drawings
FIG. 1 is a schematic diagram of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a first embodiment of the model interpretation method of the present invention;
FIG. 3 is a schematic diagram of a model interpretation optimization process according to various embodiments of the present invention;
FIG. 4 is a block diagram of a preferred embodiment of a model interpretation apparatus according to the invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present invention.
It should be noted that, the model interpretation device in the embodiment of the present invention may be a smart phone, a personal computer, a server, and the like, and is not limited herein.
As shown in fig. 1, the model interpretation apparatus may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the device configuration shown in fig. 1 does not constitute a limitation of the model interpretation device and may include more or fewer components than those shown, or some components in combination, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a model interpreter. Among them, the operating system is a program that manages and controls the hardware and software resources of the device, supporting the operation of the model interpreter and other software or programs. In the device shown in fig. 1, the user interface 1003 is mainly used for data communication with a client; the network interface 1004 is mainly used for establishing communication connection with a server; processor 1001 may be configured to invoke a model interpreter stored in memory 1005 and perform the following operations:
interpreting the model prediction result of the target user by adopting a preset interpretation algorithm to obtain an interpretation result;
generating a plurality of interpretation text contents according to the interpretation result, and analyzing each interpretation text content by adopting an acceptance judging model to obtain the corresponding acceptability of each interpretation text content, wherein the acceptance judging model is obtained by training on the basis of the feedback data of the historical user on the historical interpretation text content;
and selecting target explanation text contents from the explanation text contents based on the acceptability and outputting the target explanation text contents.
Further, the preset interpretation algorithm at least comprises one interpretation algorithm, and the step of interpreting the model prediction result of the target user by using the preset interpretation algorithm to obtain the interpretation result comprises:
respectively adopting each interpretation algorithm to interpret the model prediction result of the target user to obtain the interpretation result corresponding to each interpretation algorithm;
the step of generating a plurality of interpretation text contents according to the interpretation result comprises:
and respectively combining each interpretation result with a preset interpretation text template to generate the interpretation text content corresponding to each interpretation result, wherein the preset interpretation text template comprises a plurality of interpretation text templates.
Further, the interpretation result includes a contribution degree corresponding to each feature to be interpreted of the target user, the feature to be interpreted is a preset non-sensitive feature or a preset important feature, and the step of generating a plurality of interpretation text contents according to the interpretation result includes:
selecting the feature to be explained with the contribution degree larger than a preset contribution degree from the features to be explained as a target feature;
and combining the target characteristics and a plurality of preset interpretation text templates corresponding to the target characteristics respectively to generate each interpretation text content.
Further, after the step of selecting and outputting the target interpreted text content from the interpreted text contents based on the acceptability, the processor 1001 may be further configured to invoke a model interpreter stored in the memory 1005 to perform the following operations:
receiving a feedback instruction triggered based on the target interpretation text content, and extracting user feedback data in the feedback instruction;
taking the user characteristic data of the target user and the target interpretation text content as input characteristic data, taking the user feedback data as label data, and taking the input characteristic data and the label data as a piece of training data to be added to a model training data set;
and when a model optimization instruction is received, performing optimization training on the acceptance judging model based on the existing training data in the model training data set so as to update the acceptance judging model.
Further, after the step of selecting and outputting the target interpreted text content from the interpreted text contents based on the acceptability, the processor 1001 may be further configured to invoke a model interpreter stored in the memory 1005 to perform the following operations:
receiving a feedback instruction triggered based on the target explanation text content, extracting user feedback data in the feedback instruction, and adding the user feedback data to a feedback data set, wherein the user feedback data at least comprises an acceptance result of the target explanation text content;
and calculating the proportion of the feedback data with a positive result in the existing feedback data based on the existing feedback data in the feedback data set, and outputting the proportion as the accuracy of model interpretation.
Further, after the step of calculating, based on the existing feedback data in the feedback data set, a ratio of feedback data in the existing feedback data that receives a positive result, and outputting the ratio as the accuracy of model interpretation, the processor 1001 may be further configured to invoke a model interpreter stored in the memory 1005, and perform the following operations:
detecting whether the accuracy is smaller than a preset threshold value;
and if the accuracy is smaller than the preset threshold value, triggering a model optimization instruction of the acceptance judging model.
Further, before the step of interpreting the model prediction result of the target user by using the preset interpretation algorithm to obtain the interpretation result, the processor 1001 may be further configured to invoke a model interpreter stored in the memory 1005, and perform the following operations:
acquiring data to be predicted of the target user;
inputting the data to be predicted into a preset risk credit model to obtain a model prediction result;
outputting the model prediction result, and detecting a reason checking instruction aiming at the model prediction result;
and when the reason checking instruction is detected, executing the step of adopting a preset interpretation algorithm to interpret the model prediction result of the target user to obtain an interpretation result.
Based on the above structure, embodiments of a model interpretation method are proposed.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the model interpretation method of the present invention. It should be noted that, although a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in an order different than that shown or described herein. The execution subject of each embodiment of the model interpretation method of the present invention may be a device such as a smart phone, a personal computer, and a server, and for convenience of description, the execution subject is omitted in the following embodiments for explanation. In this embodiment, the model interpretation method includes:
step S10, interpreting the model prediction result of the target user by adopting a preset interpretation algorithm to obtain an interpretation result;
in this embodiment, in the field of practical application, a machine learning model may be trained to complete a corresponding prediction or classification task according to a business scenario, and the trained machine learning model may be used to predict a target user to obtain a model prediction result. The type of the machine learning model is not limited, and for example, the machine learning model may be a random forest, a gradient lifting tree, a deep learning model, or the like. The specific prediction or classification task is not limiting, for example, in the field of financial credit, a risk credit model may be trained that predicts whether to loan a user based on user data.
For the model prediction result of the target user, the reason why the machine learning model predicts the result needs to be explained, that is, model explanation is needed. Specifically, the reason may be expressed as the contribution of each input feature of the target user to the model prediction result, that is, the contribution degree to the model prediction result. Specifically, for the model prediction result of the target user, a preset interpretation algorithm may be adopted to interpret the result, so as to obtain an interpretation result. The preset interpretation algorithm may adopt a common Model interpretation algorithm, for example, LIME (Local interactive Model-agnostic extensions, Model-independent Local interpretation) algorithm, and SHAP (SHapley Additive extensions, SHapley Additive Model interpretation) algorithm. The explanation process according to the preset explanation algorithm is not described in detail herein. The interpretation result may include the contribution degree corresponding to each input feature of the target user.
Step S20, generating a plurality of interpretation text contents according to the interpretation results, and analyzing each interpretation text content by adopting an acceptance degree discrimination model to obtain the corresponding acceptance degree of each interpretation text content, wherein the acceptance degree discrimination model is obtained by training the feedback data of the historical interpretation text contents based on the historical users;
and generating a plurality of interpretation text contents according to the interpretation result. Specifically, there are various generation methods, for example, a template method or a model method is adopted. The template mode may specifically be that a plurality of interpretation text templates can be preset, and the various interpretation text templates have differences in language description styles so as to adapt to different users. The content of the interpretation text template may be used to interpret which one or more of the various input features caused the prediction. In the process of generating the explanation text content according to the explanation result, the feature with a high contribution degree in the explanation result can be substituted into the explanation text template to obtain the explanation text content. For example, if the content of the explanation text template is "you cannot loan because your XX does not meet the requirements", and the contribution degree corresponding to the feature of "age" in the explanation result is large, the explanation text template is substituted with the feature of "age" to obtain the explanation text content "you cannot loan because the age does not meet the requirements". And combining the interpretation results with the multiple interpretation text templates respectively to obtain multiple interpretation text contents.
The model approach may be to employ a machine learning model to interpret the textual content. Specifically, the machine learning model may adopt a common natural language processing model, and input data of the model may include keywords related to an interpretation result, and output as a plurality of interpretation text contents. For example, if the feature "age" in the interpretation result has a large contribution degree, keywords such as "age", "older" and "slightly older" may be input to the machine learning model, and a plurality of styles of interpreted text contents may be generated in response to the natural language processing of the machine learning model.
The acceptance judging model can be preset, feedback data of the historical user on the output historical interpretation text content is collected, and the feedback data reflects the acceptance degree of the user on the interpretation text content, so that the acceptance judging model can be trained by taking the feedback data as a label for supervised learning. The receptivity judging model may adopt a common machine learning model, and is not limited herein. The input data of the acceptance judging model can comprise user characteristic data of the user and the interpreted text content, and the output is the acceptance for representing the acceptance of the interpreted text content by the user. The user characteristic data may include the user's age, academic history, hobbies, and the like. By adopting the training of the feedback data of the historical user on the acceptance judging model, the acceptance judging model learns the possibility of judging the explanation text content of various description styles favored by the user based on the characteristics of the user.
After the acceptance judging model is obtained through training, the acceptance judging model can be adopted to analyze each currently generated explanation text content to obtain the corresponding acceptance of each explanation text content. Specifically, each of the interpreted text contents and the user characteristic data of the target user may be input into the acceptability judging model, and the acceptability corresponding to each of the interpreted text contents, that is, the acceptability of the target user to each of the interpreted text contents, may be output through the processing of the acceptability judging model.
In step S30, a target explanatory text content is selected from each of the explanatory text contents based on the acceptability and output.
And selecting target explanation text contents from the explanation text contents based on the acceptability and outputting the target explanation text contents. Specifically, the interpreted text content with the highest acceptability among the respective interpreted text contents may be taken as the target interpreted text content. Further, when a plurality of interpretation algorithms are adopted to interpret the model prediction result to obtain a plurality of interpretation results, the input features with larger contribution degrees in each interpretation result may be different, and because the sensitivity degrees of each input feature are different, the sensitivity degrees corresponding to each interpretation result are also different, and the sensitivity degrees of the interpreted text contents correspondingly generated based on each interpretation result are also different; after the acceptability corresponding to each explanation text content is obtained, the explanation text content with the acceptability more than a certain threshold and the sensitivity less than the certain threshold can be selected as the target explanation text content; the sensitivity of the interpreted text content can be determined according to the sensitivity of the input features contained in the interpreted text content, for example, when the interpreted text content comprises a plurality of input features, the sensitivity of each input feature can be added to obtain the sensitivity of the highly interpreted text content; the sensitivity of the input feature can be set in advance according to the service scene, the sensitivity is used for indicating the degree of the feature that can be shown to the user, and the higher the sensitivity is, the higher the secrecy degree of the input feature in the service scene is, the less suitable the input feature is for disclosing to the user.
After determining the target interpreted textual content, the target interpreted textual content may be output. Specifically, the target explanation text content may be visually output, for example, the target explanation text content is output on a display of a self-service banking loan processing device, so that a user can know the reason for refusing or agreeing to a loan, and further, the loan service may be automated, and the labor cost may be reduced.
In the embodiment, the model prediction result of the target user is interpreted by an interpretation algorithm to obtain an interpretation result, a plurality of interpretation text contents are generated according to the interpretation result, the acceptability corresponding to each interpretation text content is obtained by adopting an acceptability judging model obtained by training based on historical feedback data, and then the target interpretation text content is selected from each interpretation text content according to the acceptability and is output. The explanation result is output in the form of explaining text content, so that the reason for applying a theoretical explanation algorithm to an actual scene to visualize a user is explained; and the acceptability of the user to various explanation text contents is learned by the acceptability judging model trained on the basis of the historical feedback data, so that the acceptability of the various explanation text contents is analyzed by adopting the acceptability judging model, and the proper target explanation text contents are selected and output on the basis of the acceptability, thereby improving the acceptability of the user to the visualized reason explanation, namely improving the reliability and the explanation quality of the model explanation, and further improving the practical application effect of the model explanation. Moreover, by automatically explaining the model prediction result and generating the explanation result into intuitive explanation text content, the labor cost can be saved as much as possible in a specific business scene, and the business processing efficiency is improved.
Further, based on the first embodiment, a second embodiment of the model interpretation method of the present invention is provided, in this embodiment, the preset interpretation algorithm at least includes one interpretation algorithm, and the step S10 includes:
s101, respectively adopting each interpretation algorithm to interpret the model prediction result of the target user to obtain the interpretation result corresponding to each interpretation algorithm;
further, in this embodiment, the preset interpretation algorithm may include at least one, that is, at least one interpretation algorithm may be used to interpret the model prediction result, and the plurality of interpretation algorithms may use a common model interpretation algorithm, which is not limited herein.
And for the model prediction result of the target user, respectively adopting each interpretation algorithm to interpret the model prediction result to obtain the interpretation result corresponding to each interpretation algorithm. For example, when there are two interpretation algorithms, two model prediction results are used for interpretation respectively, and two interpretation results can be obtained.
The step of generating a plurality of interpreted text contents according to the interpretation result in the step S20 includes:
step S201, respectively combining each interpretation result with a preset interpretation text template to generate an interpretation text content corresponding to each interpretation result, where the preset interpretation text template includes multiple interpretation text templates.
And respectively combining each interpretation result with a preset interpretation text template to generate the interpretation text content corresponding to each interpretation result. The preset interpretation text template comprises a plurality of interpretation text templates, namely interpretation text templates with a plurality of description styles. And for each interpretation result, combining the interpretation result with each interpretation text template respectively to obtain a plurality of interpretation text contents corresponding to the interpretation result. And finally, obtaining a plurality of interpretation text contents corresponding to each interpretation result, and analyzing the acceptability of the interpretation text contents by adopting an acceptability judging model.
In the embodiment, the model prediction result is interpreted by adopting a plurality of interpretation algorithms, and finally appropriate target interpretation text content is selected from a plurality of interpretation text contents generated by a plurality of interpretation results and output, so that selectable interpretation text content is richer, and further the model interpretation quality of the finally selected target interpretation text content can be improved.
Further, the interpretation result includes a contribution degree corresponding to each feature to be interpreted of the target user, the feature to be interpreted is a preset non-sensitive feature or a preset important feature, and the step of generating a plurality of interpretation text contents according to the interpretation result in step S20 includes:
step S202, selecting the feature to be explained with the contribution degree larger than the preset contribution degree from all the features to be explained as a target feature;
in an embodiment, in order to adapt to the information sensitivity level in a specific business field, a feature (non-sensitive feature) with a lower sensitivity level in the input features may be used as a feature to be interpreted, and a feature with a higher sensitivity level is avoided from being adopted to explain reasons to a user, so as to ensure that a business process is normally performed. Alternatively, in an embodiment, when the input features of the model are considered to be excessive, if the explanation reasons corresponding to all the features are displayed to the user, the user cannot grasp the emphasis and some features are unimportant, so that some important features in the input features can be preset as features to be explained.
Then, the interpretation result obtained by interpreting the model prediction result of the target user by using the interpretation algorithm may only include the contribution degree corresponding to the feature to be interpreted.
After the contribution degree corresponding to each feature to be explained is obtained, the feature to be explained with the contribution degree larger than the preset contribution degree can be selected from each feature to be explained as the target feature. The preset contribution degree may be set in advance according to needs, and if the contribution degree of the feature to be explained is greater than the preset contribution degree, it indicates that the effect of the feature on the model to make the prediction result is greater.
Step S203, combining the target feature and a plurality of preset interpretation text templates corresponding to the target feature, respectively, to generate each interpretation text content.
A plurality of interpretation text targets corresponding to each feature to be interpreted are preset, and after the target feature is determined, the target feature and the plurality of interpretation text targets corresponding to the target feature can be respectively combined to obtain a plurality of interpretation text contents corresponding to the target feature. It should be noted that, when there are a plurality of target features, the interpretation text contents corresponding to each target feature may be randomly combined to obtain each group of interpretation text contents, and the acceptability of each group of interpretation text contents is analyzed. For example, the target feature 1 corresponds to the interpreted text content 11 and the interpreted text content 12, and the target feature 2 corresponds to the interpreted text content 21 and the interpreted text content 22, so that four sets of interpreted text contents are obtained by combination: the explanation text content 11+ the explanation text content 21, the explanation text content 11+ the explanation text content 22, the explanation text content 12+ the explanation text content 21, and the explanation text content 12+ the explanation text content 22.
Further, based on the first and/or second embodiments, a third embodiment of the model interpretation method of the present invention is proposed. In this embodiment, the model interpretation method further includes:
step S40, receiving a feedback instruction triggered based on the target interpretation text content, and extracting user feedback data in the feedback instruction;
after outputting the target interpreted text content, it may be detected whether a feedback instruction triggered based on the target interpreted text content is received. Specifically, the target interpretation text content can be output to the visualization interface in a visualization form, a control for the user to input the feedback content is set in a control panel of the visualization interface, the user can input the feedback content based on the control to trigger the feedback instruction, and the feedback instruction carries the feedback content. The form of the feedback content differs according to the setting of the visual interface, for example, an option for the user to select whether to continue to ask is set in the visual interface, the "ask" or "no ask" selected by the user is the feedback content, for example, an option for the user to score the target interpretation text content is set in the visual interface, and the score selected by the user is the feedback content.
After receiving the feedback instruction, the feedback content in the feedback instruction may be extracted as the user feedback data, or the feedback content may be converted into the acceptance of the target interpretation text content by the user, and the acceptance is used as the user feedback data. The receptivity can be a numerical value between 0 and 1, for example, when the feedback content is the user selection of "question hunting", the receptivity of the user to the target interpretation text content is determined to be 0, and when the feedback content is the user selection of "no question", the receptivity of the user to the target interpretation text content is determined to be 1.
Step S50, taking the user characteristic data of the target user and the target interpretation text content as input characteristic data, taking the user feedback data as label data, and taking the input characteristic data and the label data as a piece of training data to be added to a model training data set;
and step S60, when a model optimization instruction is received, performing optimization training on the acceptance degree discrimination model based on the existing training data in the model training data set to update the acceptance degree discrimination model.
A model training data set for training an acceptance degree discrimination model is preset. And taking the user characteristic data and the target explanation text content of the target user as input characteristic data, taking the user feedback data as label data, and taking the input characteristic data and the label data as a piece of training data to be added to the model training data set. Further, when there are a plurality of interpretation algorithms, the type of the interpretation algorithm, the user feature data, and the target interpretation text content may be used as input feature data, the user feedback data may be used as tag data, and the input feature data and the tag data may be added to the model training data set as a piece of training data, so that the receptivity judging model learns to judge the receptivity of the user to the various interpretation algorithms and the various types of interpretation text content based on the feature data of the user.
That is, when the model interpretation function is operated on line, the feedback data of the user to the output interpretation text content can be obtained in real time, and the user feature data of the user, the interpretation text content and the feedback data are added into the model training data set as training data, that is, the feedback data of each user to various interpretation text contents are collected to be used as the basis for optimizing the acceptance judging model.
When a model optimization instruction is received, optimization training is carried out on the acceptance judging model based on the existing training data in the model training data set so as to update the acceptance judging model, and subsequently, when a target user is explained, the acceptance of each explained text content is analyzed by adopting the updated acceptance judging model. The model optimization instructions may be triggered periodically, i.e., at intervals. The optimized training mode can adopt a supervised training mode of a common machine learning model, and detailed description is omitted here.
In the embodiment, feedback data of the user on the interpretation text content is acquired on line, the feedback data is added into the model training data set of the acceptance judging model, and the acceptance judging model is optimized through the model training data set, so that the accuracy of the acceptance judging model analysis acceptability is continuously improved based on the real-time user feedback data, the quality of the target interpretation text content selected based on the acceptance is higher, and the model interpretation quality is improved.
Further, in a real-time manner, the model interpretation method further includes:
step S70, receiving a feedback instruction triggered based on the target interpreted text content, extracting user feedback data in the feedback instruction, and adding the user feedback data to a feedback data set, where the user feedback data at least includes an acceptance result of the target interpreted text content;
after the target explanation text content is output, if a feedback instruction triggered based on the target explanation text content is received, the feedback content in the feedback instruction can be extracted as user feedback data. User feedback data is added to the feedback data set. The user feedback data at least comprises an acceptance result of the user to the target receiving text content, and the acceptance result can be a positive result and a negative result. For example, if the feedback content is "question asking", the reception result is negative, and if the feedback content is "question not asking", the reception result is positive.
And step S80, calculating the proportion of the feedback data with the positive result in the existing feedback data based on the existing feedback data in the feedback data set, and outputting the proportion as the accuracy of model interpretation.
After the value feedback data set is added to the user feedback data, the proportion of the feedback data with the positive result in the existing feedback data can be calculated for the existing feedback data in the current feedback data set, the proportion is used as the accuracy of model interpretation, and the accuracy is output. The ratio may be obtained by dividing the number of feedback data with a positive result in the existing feedback data by the total number of the existing feedback data. The accuracy can be visually output, and the output object can be an administrator, so that the administrator can know the accuracy of the current model interpretation function in time.
The accuracy reflects the general acceptance of each user to the output explanation text content, and the administrator can timely know the quality condition of the model explanation function by outputting the accuracy, so that when the accuracy is low, the model explanation function can be adjusted, for example, a model optimization instruction of an acceptance judging model is triggered, or an explanation algorithm, an explanation text template and the like are changed, and the explanation quality of the model explanation function is improved. That is, the embodiment of the present invention realizes that the actual effect of the model interpretation can be tracked, rather than staying in the theoretical stage.
Further, after the step S80, the method further includes:
step S90, detecting whether the accuracy is less than a preset threshold value;
step A10, if the accuracy is detected to be less than the preset threshold, triggering a model optimization instruction of the acceptance judging model.
After the accuracy of the model interpretation is calculated, it can be detected whether the accuracy is less than a preset threshold. The preset threshold value can be set according to specific needs, and when the requirement on the quality of model interpretation is high, the preset threshold value can be set to be high. If the accuracy is detected to be smaller than the preset threshold value, triggering a model optimization instruction of the acceptance judging model, and carrying out optimization training on the acceptance judging model according to the model optimization instruction. If the detected accuracy is not less than the preset threshold, the current model interpretation function meets the quality requirement, and no processing is required.
In this embodiment, the accuracy of model interpretation is monitored in real time according to the feedback data of the user, and when the accuracy of model interpretation is low, an optimization instruction for the acceptance judging model is triggered in time to optimize and update the acceptance judging model, so that the quality of model interpretation is ensured.
Further, before the step S10, the method further includes:
step A20, acquiring data to be predicted of the target user;
step A30, inputting the data to be predicted into a preset risk credit model to obtain a model prediction result;
a step a40 of outputting the model prediction result and detecting a cause viewing instruction for the model prediction result;
and step A50, when the reason checking instruction is detected, executing the step of adopting a preset interpretation algorithm to interpret the model prediction result of the target user to obtain an interpretation result.
In an embodiment, the model interpretation method according to the embodiment of the present invention may be applied to the field of financial services, and specifically may perform model interpretation on a risk credit model to explain to a user a reason why the model makes a risk prediction result.
A pre-trained risk credit granting model can be deployed on line to operate, and the risk credit granting model can be a model for predicting whether to loan the user, that is, the output result of the model can be the loan risk. A visual interactive interface can be set for the user to apply for loan operation in the interface. When the target user needs to apply for a loan, a loan request is triggered in the interface. And after the loan request triggered by the user is detected, carrying out loan risk assessment in the background. Specifically, the data to be predicted of the target user is obtained, where the data to be predicted may be personal data input by the user when the user triggers a loan request, or may also be related information of the user, which is obtained from a background database and stored in the past, and a specific type of the data to be predicted may be set according to a specific service, which is not limited herein. After the data to be predicted of the target user is obtained, the data to be predicted is input into the risk credit granting model for prediction, and a model prediction result is obtained, wherein the model prediction result can be the loan risk degree. The model prediction result can be output in the visual interactive interface, and the model prediction result can be specifically converted into visual text description, for example, when the loan risk degree is greater than a preset degree, a description related to loan refusal is output, otherwise, a description related to loan approval is output.
In the visual interface, a control for the user to query the reason can be set, so that the user can trigger the reason viewing instruction based on the control after viewing the result. When a reason checking instruction is detected, the relevant steps of model interpretation in the embodiment are executed, that is, a preset interpretation algorithm is adopted to interpret the model prediction result of the target user to obtain an interpretation result, interpretation text content is generated according to the interpretation result, the acceptability corresponding to each interpretation text content is obtained by adopting the acceptability judging model obtained by training based on historical feedback data, and the target interpretation text content is selected from each interpretation text content according to the acceptability and is output.
In this embodiment, a model interpretation method based on user feedback is provided, and according to the user feedback, the validity of model interpretation is verified, and the quality of model interpretation is improved. The actual effect of model interpretation can be tracked, rather than staying in the theoretical stage.
Fig. 3 is a schematic diagram illustrating a model interpretation optimization process according to an embodiment of the present invention.
1. And training a corresponding prediction model according to the service scene.
2. And deploying the prediction model to an online operation, enabling user data to enter the model, predicting the application of the user, and displaying the model result to the user.
3. If the result of the model is in doubt, the reason can be selected and checked in the interactive interface, so that the model explanation function is triggered, and the contribution degree of each characteristic of the user in the service judgment is analyzed.
4. And generating a corresponding explanation case (explanation text content) according to the model explanation.
5. The user can choose whether to continue to ask on the interactive interface according to the seen explanation case.
6. According to the user feedback, if the user does not ask any more, the explanation is accepted; if the question continues, it means that no interpretation is received. The degree of accuracy of the interpretation can be obtained from the ratio of "accept" to "not accept". Based on the feedback of the user to the model interpretation, in addition to other characteristics of the user, such as, but not limited to, age, academic history, hobbies, etc., an acceptance decision model can be trained to determine whether the interpretation scheme is acceptable. By means of the model, model interpretations with higher quality can be screened out and displayed to users, and user experience is improved.
In addition, an embodiment of the present invention further provides a model interpretation apparatus, and referring to fig. 4, the apparatus includes:
the interpretation module 10 is configured to interpret the model prediction result of the target user by using a preset interpretation algorithm to obtain an interpretation result;
the generating module 20 is configured to generate a plurality of interpretation text contents according to the interpretation result, and analyze each interpretation text content by using an acceptance degree discrimination model to obtain an acceptance degree corresponding to each interpretation text content, where the acceptance degree discrimination model is obtained by training based on feedback data of a historical user on the historical interpretation text content;
and the output module 30 is used for selecting target explanation text contents from the explanation text contents based on the acceptability and outputting the target explanation text contents.
Further, the preset interpretation algorithm at least comprises one interpretation algorithm, and the interpretation module 10 is further configured to respectively adopt each interpretation algorithm to interpret the model prediction result of the target user, so as to obtain an interpretation result corresponding to each interpretation algorithm;
the generating module 20 is further configured to: and respectively combining each interpretation result with a preset interpretation text template to generate the interpretation text content corresponding to each interpretation result, wherein the preset interpretation text template comprises a plurality of interpretation text templates.
Further, the interpretation result includes a contribution degree corresponding to each feature to be interpreted of the target user, where the feature to be interpreted is a preset non-sensitive feature or a preset important feature, and the generating module 20 includes:
the selection unit is used for selecting the to-be-interpreted features with the contribution degree larger than the preset contribution degree from the to-be-interpreted features as target features;
and the combination unit is used for respectively combining the target characteristics and the plurality of preset interpretation text templates corresponding to the target characteristics to generate each interpretation text content.
Further, the apparatus further comprises:
the receiving module is used for receiving a feedback instruction triggered based on the target interpretation text content and extracting user feedback data in the feedback instruction;
a first adding module, configured to use the user feature data of the target user and the target interpretation text content as input feature data, use the user feedback data as tag data, and add the input feature data and the tag data as a piece of training data to a model training data set;
and the optimization module is used for performing optimization training on the acceptance judging model based on the existing training data in the model training data set when a model optimization instruction is received so as to update the acceptance judging model.
Further, the apparatus further comprises:
a second adding module, configured to receive a feedback instruction triggered based on the target interpreted text content, extract user feedback data in the feedback instruction, and add the user feedback data to a feedback data set, where the user feedback data at least includes an acceptance result of the target interpreted text content;
and the calculation module is used for calculating the proportion of the feedback data with the positive result in the existing feedback data based on the existing feedback data in the feedback data set, and outputting the proportion as the accuracy of model interpretation.
Further, the apparatus further comprises:
the detection module is used for detecting whether the accuracy is smaller than a preset threshold value;
and the triggering module is used for triggering a model optimization instruction of the acceptance judging model if the accuracy is detected to be smaller than the preset threshold value.
Further, the apparatus further comprises:
the acquisition module is used for acquiring data to be predicted of the target user;
the input module is used for inputting the data to be predicted into a preset risk credit model to obtain a model prediction result;
the output module 30 is further configured to output the model prediction result and detect a reason checking instruction for the model prediction result;
the interpretation module 10 is further configured to, when the reason checking instruction is detected, execute the step of interpreting the model prediction result of the target user by using a preset interpretation algorithm to obtain an interpretation result.
The specific implementation of the model interpretation apparatus of the present invention is basically the same as the embodiments of the model interpretation method, and is not described herein again.
Furthermore, an embodiment of the present invention further provides a computer-readable storage medium, where a model interpreter is stored on the storage medium, and the model interpreter, when executed by a processor, implements the steps of the model interpretation method as described below.
The embodiments of the model interpretation apparatus and the computer-readable storage medium of the present invention can refer to the embodiments of the model interpretation method of the present invention, and are not described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method of model interpretation, said method comprising the steps of:
interpreting the model prediction result of the target user by adopting a preset interpretation algorithm to obtain an interpretation result;
generating a plurality of interpretation text contents according to the interpretation result, and analyzing each interpretation text content by adopting an acceptance judging model to obtain the corresponding acceptability of each interpretation text content, wherein the acceptance judging model is obtained by training on the basis of the feedback data of the historical user on the historical interpretation text content;
and selecting target explanation text contents from the explanation text contents based on the acceptability and outputting the target explanation text contents.
2. The model interpretation method of claim 1, wherein the predetermined interpretation algorithm comprises at least one interpretation algorithm, and the step of interpreting the model prediction result of the target user by using the predetermined interpretation algorithm to obtain the interpretation result comprises:
respectively adopting each interpretation algorithm to interpret the model prediction result of the target user to obtain the interpretation result corresponding to each interpretation algorithm;
the step of generating a plurality of interpretation text contents according to the interpretation result comprises:
and respectively combining each interpretation result with a preset interpretation text template to generate the interpretation text content corresponding to each interpretation result, wherein the preset interpretation text template comprises a plurality of interpretation text templates.
3. The model interpretation method according to claim 1, wherein the interpretation result includes a contribution degree corresponding to each feature to be interpreted of the target user, the feature to be interpreted is a preset non-sensitive feature or a preset important feature, and the step of generating a plurality of interpretation text contents according to the interpretation result includes:
selecting the feature to be explained with the contribution degree larger than a preset contribution degree from the features to be explained as a target feature;
and combining the target characteristics and a plurality of preset interpretation text templates corresponding to the target characteristics respectively to generate each interpretation text content.
4. The model interpretation method of claim 1, wherein after the step of selecting and outputting the target interpreted text contents from among the interpreted text contents based on the acceptability, further comprising:
receiving a feedback instruction triggered based on the target interpretation text content, and extracting user feedback data in the feedback instruction;
taking the user characteristic data of the target user and the target interpretation text content as input characteristic data, taking the user feedback data as label data, and taking the input characteristic data and the label data as a piece of training data to be added to a model training data set;
and when a model optimization instruction is received, performing optimization training on the acceptance judging model based on the existing training data in the model training data set so as to update the acceptance judging model.
5. The model interpretation method of claim 1, wherein after the step of selecting and outputting the target interpreted text contents from among the interpreted text contents based on the acceptability, further comprising:
receiving a feedback instruction triggered based on the target explanation text content, extracting user feedback data in the feedback instruction, and adding the user feedback data to a feedback data set, wherein the user feedback data at least comprises an acceptance result of the target explanation text content;
and calculating the proportion of the feedback data with a positive result in the existing feedback data based on the existing feedback data in the feedback data set, and outputting the proportion as the accuracy of model interpretation.
6. The model interpretation method of claim 5, wherein after the step of calculating a ratio of the feedback data of which the acceptance result is a positive result among the existing feedback data based on the existing feedback data in the feedback data set, and outputting the ratio as the accuracy of model interpretation, further comprises:
detecting whether the accuracy is smaller than a preset threshold value;
and if the accuracy is smaller than the preset threshold value, triggering a model optimization instruction of the acceptance judging model.
7. The model interpretation method according to any one of claims 1 to 6, wherein before the step of interpreting the model prediction result of the target user by using a preset interpretation algorithm to obtain the interpretation result, the method further comprises:
acquiring data to be predicted of the target user;
inputting the data to be predicted into a preset risk credit model to obtain a model prediction result;
outputting the model prediction result, and detecting a reason checking instruction aiming at the model prediction result;
and when the reason checking instruction is detected, executing the step of adopting a preset interpretation algorithm to interpret the model prediction result of the target user to obtain an interpretation result.
8. A model interpretation apparatus, characterized in that the apparatus comprises:
the interpretation module is used for interpreting the model prediction result of the target user by adopting a preset interpretation algorithm to obtain an interpretation result;
the generating module is used for generating a plurality of interpretation text contents according to the interpretation result and analyzing each interpretation text content by adopting an acceptance judging model to obtain the corresponding acceptability of each interpretation text content, wherein the acceptance judging model is obtained by training the feedback data of the historical interpretation text content based on the historical user;
and the output module is used for selecting target explanation text contents from the explanation text contents based on the acceptability and outputting the target explanation text contents.
9. A model interpretation apparatus characterized by comprising: memory, a processor and a model interpreter stored on the memory and executable on the processor, which when executed by the processor implements the steps of the model interpretation method of any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a model interpreter which, when executed by a processor, implements the steps of the model interpretation method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010621838.8A CN111768040A (en) | 2020-07-01 | 2020-07-01 | Model interpretation method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010621838.8A CN111768040A (en) | 2020-07-01 | 2020-07-01 | Model interpretation method, device, equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111768040A true CN111768040A (en) | 2020-10-13 |
Family
ID=72723242
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010621838.8A Pending CN111768040A (en) | 2020-07-01 | 2020-07-01 | Model interpretation method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111768040A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112612872A (en) * | 2020-12-17 | 2021-04-06 | 第四范式（北京）技术有限公司 | Method, device, equipment and storage medium for realizing artificial intelligence interpretability |
| CN112699947A (en) * | 2020-12-30 | 2021-04-23 | 深圳前海微众银行股份有限公司 | Decision tree based prediction method, apparatus, device, medium, and program product |
| CN112766415A (en) * | 2021-02-09 | 2021-05-07 | 第四范式（北京）技术有限公司 | Method, device and system for explaining artificial intelligence model |
| CN113377640A (en) * | 2021-06-23 | 2021-09-10 | 杭州网易云音乐科技有限公司 | Method, medium, device and computing equipment for explaining model under business scene |
| CN114997549A (en) * | 2022-08-08 | 2022-09-02 | 阿里巴巴（中国）有限公司 | Interpretation method, device and equipment of black box model |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8515862B2 (en) * | 2008-05-29 | 2013-08-20 | Sas Institute Inc. | Computer-implemented systems and methods for integrated model validation for compliance and credit risk |
| CN107730286A (en) * | 2016-08-10 | 2018-02-23 | 中国移动通信集团黑龙江有限公司 | A kind of target customer's screening technique and device |
| CN108665277A (en) * | 2017-03-27 | 2018-10-16 | 阿里巴巴集团控股有限公司 | A kind of information processing method and device |
| CN111340231A (en) * | 2020-02-11 | 2020-06-26 | 深圳前海微众银行股份有限公司 | SHAP feature attribution method, device, equipment and readable storage medium |
-
2020
- 2020-07-01 CN CN202010621838.8A patent/CN111768040A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8515862B2 (en) * | 2008-05-29 | 2013-08-20 | Sas Institute Inc. | Computer-implemented systems and methods for integrated model validation for compliance and credit risk |
| CN107730286A (en) * | 2016-08-10 | 2018-02-23 | 中国移动通信集团黑龙江有限公司 | A kind of target customer's screening technique and device |
| CN108665277A (en) * | 2017-03-27 | 2018-10-16 | 阿里巴巴集团控股有限公司 | A kind of information processing method and device |
| CN111340231A (en) * | 2020-02-11 | 2020-06-26 | 深圳前海微众银行股份有限公司 | SHAP feature attribution method, device, equipment and readable storage medium |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112612872A (en) * | 2020-12-17 | 2021-04-06 | 第四范式（北京）技术有限公司 | Method, device, equipment and storage medium for realizing artificial intelligence interpretability |
| CN112699947A (en) * | 2020-12-30 | 2021-04-23 | 深圳前海微众银行股份有限公司 | Decision tree based prediction method, apparatus, device, medium, and program product |
| CN112766415A (en) * | 2021-02-09 | 2021-05-07 | 第四范式（北京）技术有限公司 | Method, device and system for explaining artificial intelligence model |
| WO2022171037A1 (en) * | 2021-02-09 | 2022-08-18 | 第四范式（北京）技术有限公司 | Method and apparatus for interpreting artificial intelligence model, and system |
| CN112766415B (en) * | 2021-02-09 | 2023-01-24 | 第四范式（北京）技术有限公司 | Method, device and system for explaining artificial intelligence model |
| CN113377640A (en) * | 2021-06-23 | 2021-09-10 | 杭州网易云音乐科技有限公司 | Method, medium, device and computing equipment for explaining model under business scene |
| CN113377640B (en) * | 2021-06-23 | 2022-07-08 | 杭州网易云音乐科技有限公司 | Method, medium, device and computing equipment for explaining model under business scene |
| CN114997549A (en) * | 2022-08-08 | 2022-09-02 | 阿里巴巴（中国）有限公司 | Interpretation method, device and equipment of black box model |
| CN114997549B (en) * | 2022-08-08 | 2022-10-28 | 阿里巴巴（中国）有限公司 | Interpretation method, device and equipment of black box model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111768040A (en) | Model interpretation method, device, equipment and readable storage medium | |
| WO2018103595A1 (en) | Authorization policy recommendation method and device, server, and storage medium | |
| US20190311114A1 (en) | Man-machine identification method and device for captcha | |
| CN108596616B (en) | User data authenticity analysis method and device, storage medium and electronic equipment | |
| US11520556B2 (en) | Application replication platform | |
| CN109816483B (en) | Information recommendation method and device and readable storage medium | |
| CN112017059A (en) | Hierarchical optimization risk control method and device and electronic equipment | |
| CN112307464A (en) | Fraud identification method and device and electronic equipment | |
| CN114662006A (en) | End cloud collaborative recommendation system and method and electronic equipment | |
| KR102226536B1 (en) | Method, device and program for recommending charts to apply security data using AI model | |
| CN110704614B (en) | Information processing method and device for predicting user group type in application | |
| CN117435999A (en) | Risk assessment method, apparatus, device and medium | |
| CN116664306A (en) | Intelligent recommendation method and device for wind control rules, electronic equipment and medium | |
| CN115757075A (en) | Task abnormity detection method and device, computer equipment and storage medium | |
| CN111382383A (en) | Method, device, medium and computer equipment for determining sensitive type of webpage content | |
| CN115600818A (en) | Multi-dimensional scoring method and device, electronic equipment and storage medium | |
| CN112950218A (en) | Business risk assessment method and device, computer equipment and storage medium | |
| CN114972500A (en) | Checking method, marking method, system, device, terminal, equipment and medium | |
| CN113590484A (en) | Algorithm model service testing method, system, equipment and storage medium | |
| CN113362069A (en) | Dynamic adjustment method, device and equipment of wind control model and readable storage medium | |
| CN112328769A (en) | Automatic customer service response method, device and computer readable storage medium | |
| CN112785418A (en) | Credit risk modeling method, credit risk modeling device, credit risk modeling equipment and computer readable storage medium | |
| KR20220001065A (en) | Method and apparatus for assessing relevancy between contents and asset | |
| CN111352840B (en) | Online behavior risk assessment method, device, equipment and readable storage medium | |
| CN115826899B (en) | Graffiti image processing method and system based on remote desktop sharing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201013 |