CN113377640A

CN113377640A - Method, medium, device and computing equipment for explaining model under business scene

Info

Publication number: CN113377640A
Application number: CN202110698960.XA
Authority: CN
Inventors: 段石石; 王军正; 谭钧心; 刘长伟; 程纯; 汪磊; 朱一飞; 岳猛; 苏杭; 郭元; 李宽
Original assignee: Hangzhou Netease Cloud Music Technology Co Ltd
Current assignee: Hangzhou Netease Cloud Music Technology Co Ltd
Priority date: 2021-06-23
Filing date: 2021-06-23
Publication date: 2021-09-10
Anticipated expiration: 2041-06-23
Also published as: CN113377640B

Abstract

The embodiment of the disclosure provides a method, medium, device and computing equipment for explaining a model in a business scene. The method comprises the following steps: collecting core data related to target model training in a business scene; interpreting the core data to obtain a first interpretation result; collecting object model related data, the object model related data comprising at least one of: environmental data relating to an application environment of the target model and monitoring data relating to the target model; and interpreting the target model by using at least one of the target model related data and the first interpretation result to obtain a second interpretation result. According to the method of the embodiment, the environment data and the monitoring data under the business scene are collected for participating in interpretation besides the core data of the target model, so that the interpretability of the model is improved, the interpretability of the model is independent of the model, and the same interpretation method can be used for the business scene of any model training.

Description

Method, medium, device and computing equipment for explaining model under business scene

Technical Field

The embodiments of the present disclosure relate to the field of communications and computer technologies, and in particular, to a method, a medium, an apparatus, and a computing device for interpreting a model in a service scenario.

Background

This section is intended to provide a background or context to the embodiments of the disclosure recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.

The related model interpretability technology is based on an interpretable model, and after training of the model is completed, interpretation logic of the model is irrelevant to samples and scenes and only relevant to sample characteristic information introduced by the model, so that interpretation capability is weak. In addition, the interpretable model is generally single in logic and cannot cover the complex modeling requirement under the current industry business scene, the interpretable research on the complex model is in an early black box stage, and the interpretable capacity similar to that of the interpretable model cannot cover the complex model. With the increasing complexity of the service, the dramatic increase of the data volume and the increasing popularity of the complex model, for the service, the interpretable performance deficiency is difficult to explain the reason for the change of the service index.

Disclosure of Invention

Model interpretability is not particularly accurate in a business scene, so that besides the model itself is interpreted, the model is put into a specific operation scene, the influence of environmental parameters and some parameters on the model is considered, and meanwhile, some characteristic information and business data information of the interpretable model are utilized. In the related technology, the weight of the trained model can be visually displayed, and the model training sample is obtained in the method, and the distribution of the sample on the line is analyzed to assist business personnel. In addition, the environmental data is acquired, and the performance condition after the model is online is considered, for example, whether more traffic generates an operation bottleneck after the model is online or not, and the traffic QPS suddenly increases in the traffic scene, which may cause a problem in the traffic.

Therefore, the training samples of the machine learning model are collected and stored, and the data of the real business are analyzed and interpreted. The interpretation does not depend on the model completely, and also depends on sample data, environmental data and monitoring data; and providing the service personnel for analysis through the combination of the data.

Therefore, model training sample data, core data reported by related systems, environmental data and monitoring data are needed to improve the interpretability of the model on a business scene.

In this context, embodiments of the present disclosure are intended to provide a method, medium, apparatus, and computing device to interpret models under a business scenario.

In a first aspect of the disclosed embodiments, there is provided a method for interpreting a model in a service scenario, including:

collecting core data related to target model training in a business scene;

interpreting the core data to obtain a first interpretation result;

collecting object model related data, the object model related data comprising at least one of: environmental data relating to an application environment of the target model and monitoring data relating to the target model;

and interpreting the target model by using at least one of the target model related data and the first interpretation result to obtain a second interpretation result.

In some embodiments of the present disclosure, based on the foregoing solution, the core data includes real-time samples of the target model and an intermediate file related to training of the target model, and collecting the core data related to training of the target model in a business scenario includes:

collecting real-time samples of the target model in response to real-time behaviors of the user, wherein the real-time samples comprise real-time features and real-time behavior data or marking behavior data;

collecting intermediate files related to the target model training and storing the intermediate files to a data storage module;

correspondingly, interpreting the core data to obtain a first interpretation result comprises:

and interpreting the correlation between the real-time behavior data and the real-time characteristics or the correlation between the marking behavior data and the real-time characteristics to obtain a first interpretation result.

In some embodiments of the present disclosure, based on the foregoing solution, the model is a recommendation model, and collecting real-time samples of the target model in response to real-time behavior of the user includes:

acquiring real-time characteristics of a user;

collecting real-time behavior data, wherein the real-time behavior data represents real-time behaviors performed by a user on a target object recommended according to real-time characteristics of the user;

and splicing the real-time characteristics and the real-time behavior data to form a real-time sample, wherein the real-time sample is used for training the target model.

In some embodiments of the present disclosure, based on the foregoing scheme, the model is a classification model, and collecting real-time samples of the target model in response to real-time behavior of the user includes:

acquiring real-time characteristics of a user, classifying the user according to the real-time characteristics and displaying a classification result;

collecting real-time behavior data, wherein the real-time behavior data represents real-time behaviors executed by a user on the classification result;

In some embodiments of the present disclosure, based on the foregoing, collecting real-time characteristics of the target model and marking behavior data comprises:

acquiring real-time characteristics of a user;

under the condition that the real-time characteristics meet preset conditions, sending the real-time characteristics to a target end;

and receiving marking behavior data obtained by marking the target end according to the real-time characteristics.

In some embodiments of the present disclosure, based on the foregoing scheme, interpreting the correlation between the real-time behavior data and the real-time characteristic or the correlation between the marking behavior data and the real-time characteristic to obtain a first interpretation result includes:

uploading the real-time samples to an interpretation engine;

and interpreting the correlation between the real-time behavior data and the real-time characteristics or the correlation between the marking behavior data and the real-time characteristics by using an interpretation template in the interpretation engine to obtain a first interpretation result, wherein interpretation indexes are arranged in the interpretation template.

In some embodiments of the present disclosure, based on the foregoing scheme, the interpretation engine includes a real-time interpretation engine and an offline interpretation engine,

accordingly, uploading the real-time samples to an interpretation engine comprises:

if the target model is a real-time training model, uploading the real-time sample to a real-time interpretation engine;

and if the target model is an offline training model, uploading the real-time sample to an offline interpretation engine.

In some embodiments of the present disclosure, based on the foregoing solution, interpreting the target model by using at least one of the target model-related data and the first interpretation result to obtain a second interpretation result includes:

and establishing a data association relation related to at least one of the target model related data and the first interpretation result according to a preset analysis rule, and displaying the data association relation in a visual form.

In some embodiments of the present disclosure, based on the foregoing scheme, before establishing the data association relationship, the method further includes:

detecting abnormal data in the target model related data;

and repairing or deleting the abnormal data.

In some embodiments of the present disclosure, based on the foregoing solution, the method for explaining a model in a service scenario further includes:

receiving a scene binding instruction, wherein the scene binding instruction carries a scene identifier and a user identifier;

and finding a scene corresponding to the scene identification and binding the user identification with the scene identification.

In some embodiments of the present disclosure, based on the foregoing scheme, establishing a data association relation with respect to at least one of the target model-related data and the first interpretation result includes:

acquiring target model related data in a preset time period of the target model;

and establishing an incidence relation between the target model related data and the time point in the preset time period.

In some embodiments of the present disclosure, based on the foregoing scheme, the obtaining target model related data within a preset time period of the target model includes:

receiving a data comparison instruction, wherein the data comparison instruction carries a data identifier and time segment information;

and searching the relevant data of the target model corresponding to the data identification in the time period corresponding to the time period information.

and acquiring the target model related data and/or the custom logic of the real-time characteristics, and sending out alarm information under the condition that the target model related data and/or the real-time characteristics meet preset conditions.

acquiring target model related data and effect change data of the target model within a preset time period;

and establishing an incidence relation among any two or three of the target model related data, the effect change data and the first interpretation result.

In some embodiments of the present disclosure, based on the foregoing scheme, after the interpreting the correlation between the real-time behavior data and the real-time feature or the correlation between the marking behavior data and the real-time feature by using a preset interpretation template in the interpretation engine to obtain a first interpretation result, the method further includes:

and digitizing the first interpretation result and storing the first interpretation result in a data storage module.

In some embodiments of the present disclosure, based on the foregoing scheme, the digitizing the first interpretation result includes:

and converting the first interpretation result into a rule data table by using a preset service configuration table.

and calculating a corresponding algorithm index according to the first interpretation result by using a preset algorithm.

In some embodiments of the present disclosure, based on the foregoing scheme, after repairing or deleting the abnormal data, the method further includes:

verifying whether abnormal data still exist;

and recording the data cleaning information to form a data report and outputting the data report under the condition that abnormal data does not exist.

In some embodiments of the present disclosure, based on the foregoing scheme, the interpretation template is written through an application program interface.

In some embodiments of the present disclosure, based on the foregoing scheme, the target model related data is uploaded by an agreed parsing protocol using a software development kit.

In some embodiments of the present disclosure, based on the foregoing scheme, the environment data includes data related to a processing flow of a server to which the object model is applied in a business scenario.

In some embodiments of the present disclosure, based on the foregoing, the monitoring data includes data related to a failure of feature update of the target model.

In a second aspect of embodiments of the present invention, there is provided a medium having a program stored thereon, the program, when executed by a processor, implementing a method of interpreting a model under a service scenario as described in the above embodiments.

In a third aspect of the embodiments of the present invention, there is provided an apparatus for interpreting a model in a service scenario, including:

the system comprises a first collection module, a second collection module and a third collection module, wherein the first collection module is configured to collect core data related to target model training in a business scene;

the first interpretation module is configured to interpret the core data to obtain a first interpretation result;

a second collection module configured to collect object model related data, the object model related data comprising at least one of: environmental data relating to an application environment of the target model and monitoring data relating to the target model;

and the second interpretation module is configured to interpret the target model by using at least one of the target model related data and the first interpretation result to obtain a second interpretation result.

In some embodiments of the present disclosure, based on the foregoing scheme, the core data includes real-time samples of the target model and intermediate files related to training of the target model, and the first collecting module includes:

a first collecting unit configured to collect real-time samples of the target model in response to real-time behavior of the user, the real-time samples including real-time features and real-time behavior data or marking behavior data;

a second collecting unit configured to collect intermediate files related to the target model training and store the intermediate files to a data storage module;

accordingly, the first interpretation module is further configured to:

In some embodiments of the disclosure, based on the foregoing, the model is a recommendation model, and the first collecting unit is further configured to:

acquiring real-time characteristics of a user;

In some embodiments of the present disclosure, based on the foregoing scheme, the model is a classification model, and the first collection unit is further configured to:

In some embodiments of the present disclosure, based on the foregoing, the first collecting unit is further configured to:

acquiring real-time characteristics of a user;

In some embodiments of the present disclosure, based on the foregoing, the first interpretation module is further configured to:

uploading the real-time samples to an interpretation engine;

accordingly, the first interpretation module is further configured to:

In some embodiments of the present disclosure, based on the foregoing, the second interpretation module is further configured to:

In some embodiments of the present disclosure, based on the foregoing solution, the apparatus further includes:

a detection module configured to detect abnormal data in the target model-related data;

and the repairing module is configured to repair or delete the abnormal data.

the receiving module is configured to receive a scene binding instruction, and the scene binding instruction carries a scene identifier and a user identifier;

a binding module configured to find a scene corresponding to the scene identifier and bind the user identifier with the scene identifier.

an alert module configured to: and acquiring the target model related data and/or the custom logic of the real-time characteristics, and sending out alarm information under the condition that the target model related data and/or the real-time characteristics meet preset conditions.

and the data storage module is used for storing the first interpretation result.

In some embodiments of the present disclosure, based on the foregoing, the data processing module is further configured to:

a verification module configured to verify whether the abnormal data still exists;

and the record output module is configured to record the data cleaning information to form a data report and output the data report under the condition that the abnormal data does not exist.

In a fourth aspect of embodiments of the present invention, there is provided a computing device comprising: the processor is used for calling the executable instructions stored in the memory to execute the method for explaining the model under the service scene in the embodiment.

According to the method, the medium, the device and the computing equipment for explaining the model in the business scene, the method according to the embodiment collects the environment data and the monitoring data in the business scene besides the core data of the target model to participate in the explanation of the target model, so that the model interpretability is improved, and the model interpretability is independent of the model, so that the same explanation method can be used for the business scene of any model training.

Drawings

The above and other objects, features and advantages of exemplary embodiments of the present disclosure will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:

FIG. 1 schematically illustrates a flow diagram of a method of interpreting a model under a business scenario, according to one embodiment of the disclosure;

FIG. 2 schematically illustrates a system architecture diagram implementing a method of interpreting a model under a business scenario, according to one embodiment of the disclosure;

FIG. 3 schematically illustrates a graph of changes in environment data QPS for a method of interpreting a model under a business scenario, according to one embodiment of the present disclosure;

fig. 4 schematically illustrates a variation graph of an environmental data latency of a method of interpreting a model under a business scenario according to one embodiment of the present disclosure;

FIG. 5 schematically illustrates a graph of changes in failure rate of monitoring data features for a method of interpreting a model under a business scenario, according to one embodiment of the disclosure;

FIG. 6 schematically shows a flow diagram of a method of interpreting a recommendation model in a business scenario, according to one embodiment of the present disclosure;

fig. 7 schematically shows a block diagram of an apparatus for explaining a model under a business scenario according to one embodiment of the present disclosure.

In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

Detailed Description

The principles and spirit of the present disclosure will be described with reference to a number of exemplary embodiments. It is understood that these embodiments are given solely for the purpose of enabling those skilled in the art to better understand and to practice the present disclosure, and are not intended to limit the scope of the present disclosure in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

As will be appreciated by one skilled in the art, embodiments of the present disclosure may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.

According to the embodiment of the disclosure, a method, a medium, a device and a computing device for explaining a model in a business scene are provided.

The main terms mentioned in this disclosure are explained as follows:

general machine learning scenario: the internet products are converted from customized content consumption logic in the traditional software industry into personalized matching of the content and consumers, and the traditional rule engine cannot meet the complexity requirement and needs a more personalized mathematical model for content distribution;

interpretability: interpretability is not defined strictly mathematically, in the field of comprehension, interpretability refers to the degree to which one can understand the reason for a decision, the higher the interpretability of a machine learning model, the easier one can understand why certain decisions and predictions are made;

the interpretable model: the interpretable model refers to a mathematical formula which is naturally interpretable, and the reason of decision can be easily explained by model parameters, and common interpretable models include: linear regression, logistic regression, decision trees, and the like;

model interpretable method: aiming at content distribution of internet products, personalized mathematical models, particularly complex deep learning models are increasingly widely applied, and a model interpretability method refers to a technical means for associating the mathematical logic of the complex models with the positive and negative effects of complex business scenes, particularly business, through a certain mathematical means.

Moreover, any number of elements in the drawings are by way of example and not by way of limitation, and any nomenclature is used solely for differentiation and not by way of limitation.

The principles and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.

Summary of The Invention

The model interpretability methods currently popular in the industry mainly focus on models with interpretability themselves, such as linear regression, logistic regression, decision trees, and the like.

Taking logistic regression as an example, logistic regression is a classification method in the field, and is commonly used in a plurality of internet service scenarios including click through rate estimation, and the mathematical formula is as follows:

wherein Y is the final probability of the behavior, X is the behavior characteristic, W is the parameter, and W is assumed to be composed of the characteristic parameter { beta [ ]₀，β₁，β₂，…，β_pAnd if the behavior is aimed at a click rate estimation scene, the probability of clicking the behavior is as follows:

and (3) dividing the behavior click probability and the behavior non-click probability and taking a log:

indicating that the behavior can be characterized by its eigenvalues {1, x }₁，x₂，…，x_pDeciding how likely it is to be clicked, the final probability of the action can be decided by reasonably changing some feature values.

Based on the above mathematical principles, the related art generally performs model attribution analysis on interpretable models, and determines the importance of the model parameters to a scene by taking absolute values of the model parameters.

The inventor finds that the related technology about model interpretability is fallen on a model which is interpretable per se, and after training of the model is completed, interpretation logic of the model is irrelevant to samples and scenes and only relevant to sample characteristic information introduced by the model, so that the interpretation capability is weak. In addition, the interpretable model is generally single in logic and cannot cover the complex modeling requirement under the current industry business scene, the interpretable research on the complex model is in an early black box stage, and the interpretable capacity similar to that of the interpretable model cannot cover the complex model. With the increasing complexity of the service, the dramatic increase of the data volume and the increasing popularity of the complex model, for the service, the interpretable performance deficiency is difficult to explain the reason for the change of the service index.

In one aspect, there are some models that are not interpretable, and it is uncertain whether the model output conforms to the expected variation after the feature is changed. For some models, interpretation by weight is possible, whereas for a rule or random strategy, without model-related information such as weight, traditional interpretable methods cannot be implemented. The inventors have realized that in this case a sample under real traffic scenarios needs to be introduced. Regardless of how the sample is collected, such as a random strategy pushing the youth female anchor in a business scene, but the model does not have any feature representing the youth female anchor, the click probability that the recommended anchor is the youth female can be seen in the sample to be particularly high, and then the recommended youth female anchor conversion rate is known to be high.

On the other hand, model interpretability is not particularly accurate in a business scene, so that besides the model itself is interpreted, the model is put into a specific operation scene, the influence of environmental parameters and some parameters on the model is considered, and meanwhile, some characteristic information and business data information of the interpretable model are utilized. In the method, the model training sample is obtained, and the distribution of the sample on the line is analyzed to assist business personnel. In addition, the environmental data is acquired, and the performance condition after the model is online is considered, for example, whether more traffic generates an operation bottleneck after the model is online or not, and the traffic QPS suddenly increases in the traffic scene, which may cause a problem in the traffic.

Therefore, embodiments of the present disclosure provide a method, medium, apparatus, and computing device for interpreting a model in a business scenario, which can improve interpretability of the model, and implement the same interpretation method for any model-trained business scenario, except that model interpretability is independent of the model.

Having described the general principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

Exemplary method

A method of explaining a model under a business scenario according to an exemplary embodiment of the present disclosure is described below with reference to fig. 1.

Fig. 1 schematically shows a method of interpreting a model under a service scenario, comprising steps 102 to 108.

102: and collecting core data related to target model training in a business scene.

The target model may be a self-interpretable model, an arbitrary machine-learning model, or a mere regular or random strategy. The target model is applied to a business scene to execute the business of the corresponding function. In one embodiment, the target model is a recommendation model, and the service scenario may recommend a song list for the user by the music playing platform, recommend a commodity item for the user by the shopping platform, or recommend a commodity item for the user by the purchasing platform.

In one embodiment, the core data includes real-time samples of the target model, and collecting core data related to training of the target model in a business scenario includes:

real-time samples of the target model are collected in response to real-time behavior of the user, the real-time samples including real-time features and either real-time behavior data or marking behavior data.

In a business scenario applying a target model, a user executes a real-time behavior, collects real-time characteristics and real-time behavior data or real-time characteristics and marking behavior data of the user to form a real-time sample, and performs iterative training on the target model, which will be described below.

In one embodiment, the model is a recommendation model, collecting real-time samples of the target model in response to real-time behavior of the user, comprising:

acquiring real-time characteristics of a user;

For the embodiment that the shopping platform recommends commodity items for the user, behavior data of clicking and browsing the commodity items recommended by the shopping platform by the user is used as a sample, a target model is trained, the trained target model is used for recommending the shopping platform, when the user enters a shopping platform homepage or updates to a certain page, a real-time score is calculated according to user characteristics, the commodity item with the highest score is recommended to the user, and the user executes behaviors aiming at the pushed commodity items, namely clicking or not clicking. For the embodiment that the music playing platform recommends the song list for the user, the user enters the music playing platform or updates to a certain page, calculates the real-time score according to the user characteristics, recommends the song list with the highest score to the user, and the user performs the action of clicking or not clicking on the recommended song list. And describing the collected core data by taking a recommended song list as an example, collecting real-time characteristics and real-time behavior data of a recommended model in response to the clicking or non-clicking real-time behavior of the user after the song list is recommended to the user, and splicing the real-time characteristics and the real-time behavior into a sample to continue iterative training of the recommended model.

Real-time features refer to user-related features or target object-related features when a user uses a platform. For example, the music playing platform recommends the song list, and the real-time characteristics may include the number of times the song list has been played and the number of times the song list has been played by the user in history. The real-time features may further include: the system type of the terminal equipment used by the user is Android or IOS, the network type of the terminal equipment connected is WIFI, 4G or 5G, the gender and age of the user, and the like. The core data related to model training of the model training platform is reported and then analyzed, the problem of no data can be effectively solved by reporting the core data, various different data sources including samples and characteristics can be accurately comprehensively considered by analyzing the core data subsequently, and the support for a complex model is increased.

The case where the real-time sample includes the real-time characteristics and the real-time behavior data is described above, and the case where the real-time sample includes the real-time characteristics and the marking behavior data is described below. For embodiments in which the real-time sample includes real-time features and marking behavior data, collecting the real-time features of the target model and the marking behavior data may include:

acquiring real-time characteristics of a user;

The marking behavior data is behavior data formed by marking by a target terminal, for example, by an operator, according to real-time characteristics. The method is explained by taking the example of discovering cheating users, namely detecting users brushing sings, collecting real-time characteristics of the users, such as the played times of the sings and the historical sings played by the users on the same day, if the played times of the same user ID for a certain singing are larger than a threshold value and/or the historical sings played by the users are larger than a preset threshold value, sending the same user ID to an operator for marking, sending the real-time characteristics to the operator for marking after judgment according to preset rules or templates, namely whether cheating is performed or not. For example, a mark that the model scores high in real time may be flagged as a cheat.

In one embodiment, the target model is a classification model, and collecting real-time samples of the target model in response to real-time behavior of the user includes:

The classification model can be a clustering model using any clustering algorithm or other classification models, the real-time characteristics of the user are obtained under the condition that the user logs in or uses a service platform to which the classification model is applied, the type of the user is judged according to the real-time characteristics, the classification result is displayed to the user, and the user confirms the classification result and confirms whether the classification result meets the self condition or not. The real-time behavior comprises approval or denial behaviors of the classification result, for example, approval behaviors executed by clicking a confirmation control and the like or denial behaviors executed by clicking a denial control and the like, and then real-time characteristics and real-time behavior data are spliced to form a real-time sample.

In one embodiment, the core data includes real-time samples of the target model and an intermediate file related to training of the target model, and collecting the core data related to training of the target model in a business scenario includes:

and collecting intermediate files related to the target model training and storing the intermediate files to a data storage module.

The intermediate files related to the training of the target model are some files for visualizing the model parameters, such as a model parameter histogram, a model weight histogram, and the like. After the intermediate file is stored in the data storage module, the intermediate file can be checked in the training process, and parameters or weights are adjusted according to the intermediate file. For example, it is preferable that the model parameters are subjected to normal distribution with a mean value of 0 and a variance of 1, and the histogram distribution is checked during training to determine whether the model parameters are trained. For example, if all the weights in the histogram are high and the model overflows, it is determined that the model is not reasonable and the parameters need to be adjusted. The model training platform can directly upload the intermediate file and directly store the intermediate file to the data storage module. The data storage module can be any storage device, such as a distributed storage device, optionally, a Driud or MySQL storage device can be used to store related data, especially time-series data, and the data is persisted in a time-series database and used for the change of related reported data when indexes and performance change are compared subsequently, so that the change of model performance and core data indexes at different moments can be effectively compared, the historical change of a scene can be effectively mastered, and reference can also be made for the online of the model.

Step 104: and interpreting the core data to obtain a first interpretation result.

Step 104 may be implemented by: and interpreting the correlation between the real-time behavior data and the real-time characteristics or the correlation between the marking behavior data and the real-time characteristics to obtain a first interpretation result.

Unlike the conventional interpretable method of interpreting the high-low visualization of the weights in the trained model, in the case where the core data at least includes real-time samples of the model, i.e., real-time behavior data and real-time features or marking behavior data and real-time features, the correlation therebetween is interpreted in step 104.

Specifically, the interpreting the correlation between the real-time behavior data and the real-time characteristic or the correlation between the marking behavior data and the real-time characteristic to obtain the first interpretation result can be realized by:

uploading the real-time samples to an interpretation engine;

The interpretation templates can be written through an application program interface, and some templates can be defined by a user or a business person according to requirements. Specifically, the interpretation index in the interpretation template may be set according to a specific application scenario of the target model. The interpretation index can be individual condition dependence, cumulative local effect, feature interaction, replacement feature importance, LIME, Shapley, SHAP, counterfactual interpretation, confrontation sample, heterogeneous point data statistics, influence instance. By combining multiple interpretation indexes to replace the traditional single interpretable model index, the model interpretable capability is stronger.

Taking the recommendation of the song list by the recommendation model as an example, selecting characteristic interaction indexes, and predicting whether to click the song list by using two indexes of the played song list times and the historical song list playing times of the user by the model, assuming that the following four samples exist, as shown in table 1:

TABLE 1 sample data

Each sample has three fields, the first two fields being feature fields and the third field being a behavior field. For example, the first row to the 4 th row in table 1 represent samples of users a to D, and when training the recommendation model, the probability of clicking when the training target is the number of times that the song sheet has been played is small, that is, the probability of the user generating a clicking action is small. The interactive analysis is carried out by the 'song list playing times' and whether the song list is clicked, and the user can not click the song list when the 'song list playing times' is less than 10000 times, the possible reason is that the song list playing times can effectively indicate the popularity of the song list, and the user can not click the song list when the 'user history song list playing times' is less, the possible reason is that the 'user history song list playing times' indicates the interest of the user in playing the song list to a certain extent, and the interest of the user in playing the song list is greater than a certain threshold value, so that the song list recommendation can be accepted more easily. Under the condition that the number of times of playing the song list is set to be less than 10000 times in the explanation template, and the user cannot click the song list, if the number of times of playing the song list in the real-time sample is 8000 times, the correlation between the behavior data and the real-time characteristics can be explained.

In one embodiment, the interpretation engine includes a real-time interpretation engine and an offline interpretation engine,

The explanation engine is divided into a real-time explanation engine and an off-line explanation engine according to business logic, the real-time training model is a real-time effective model, namely a model which is trained in real time by using a real-time sample, the off-line training model is a non-real-time effective model, for example, a training model is trained after a preset time after a new real-time sample is generated, for example, after a song list is recommended to a user, behavior data clicked or not clicked by the user is generated into a real-time sample, the real-time sample is used for training the recommendation model, and for the off-line training model, after the user clicks for a period of time, the real-time sample is used for training the recommendation model after one day or one week.

Fig. 2 schematically shows a system architecture diagram for implementing a method for explaining a model under a service scenario according to an embodiment of the present disclosure, and as shown in fig. 2, the system architecture includes a data reporting system, an explanation engine, and a data storage and visualization system, the data reporting system includes a core data reporting 201, an environment data reporting 202, and a monitoring data reporting 203, the explanation engine is divided into a real-time explanation engine 204 and an offline explanation engine 205 for real-time training or offline training according to a target model, and the data storage and visualization system includes an explanation datamation 206, a data storage module 207, a custom data visualization 208, and a service monitoring system 209.

In the real-time interpretation engine, the core data is reported to the Kafka, namely, the real-time task flow collects and puts the core data on a server and reports the core data to the Kafka. By agreeing on the location of the data deposit for consumption, the downstream business can know the location of the data deposit for production, and the party consuming the data uses, for example, a real-time interpretation engine to consume. And reporting the data to the HDFS for the offline explanation engine, setting an explanation template for explanation, and generating explanation data in batch.

In one embodiment, after the interpreting the correlation between the real-time behavior data and the real-time feature or the correlation between the marking behavior data and the real-time feature by using a preset interpretation template in the interpretation engine to obtain a first interpretation result, the method further includes:

Some data of the interpretation result cannot be directly provided for service personnel to use under the condition of some data, and the data needs to be digitalized by the interpretation and digitization module and then stored in the data storage module.

The digitizing of the first interpretation result may be achieved by: and converting the first interpretation result into a rule data table by using a preset service configuration table. In this case, the interpretation result is displayed as chart information in a personalized manner mainly by using a business configuration module, such as a regularized visual data table.

The digitizing of the first interpretation result may also be achieved by: and calculating a corresponding algorithm index according to the first interpretation result by using a preset algorithm. In this case, the disclosed algorithm index, such as online AUC, online Q distribution, etc., is calculated from the interpretation result. Also like writing interpretation templates, the interpretation results can also be digitized by user-defined writing mathematical logic by providing an interface. In one embodiment, before reporting the core data, abnormal data in the core data is also detected, and the abnormal data is repaired or deleted.

106: collecting object model related data, the object model related data comprising at least one of: environmental data relating to an application environment of the target model and monitoring data relating to the target model.

The environment data mainly comprises access browsing of online services of the application target model, such as query rate per second (QPS), data request delay, model reasoning delay and the like, and also comprises meta information of time consumption of each process, QPS request for a feature warehouse for storing features, load, feature version update time and the like in the real-time sample splicing process, and data directory, hyper-parameters, model architecture, model training timeline and the like of model training, wherein the data are used for assisting in analyzing indexes and performance changes of the model caused by the environment information.

For the environmental data, first consider data related to the processing traffic of the server to which the target model is applied in the business scenario, e.g., whether the traffic is too large. The service is applied to the online business, if the access flow is suddenly increased, the service cannot be supplied, the user feels that the service is blocked, and the flow is large, so that the data request time delay is large. Taking the recommendation model for recommending the song list as an example, if the processing flow for the user to open the music playing platform and push the song list to the user according to the user characteristics is too large, for example, the QPS is too large, the request data delay is large. And reporting the QPS, the request data delay and the like to a data storage module and then carrying out visual display, so that service personnel can conveniently know the current QPS and the like and assist in analyzing the performance change of the target model caused by the QPS and the like.

The feature version update time refers to that features change when samples are spliced, for example, the service life of a user changes when a music playing platform is used in the last year or this year. For another example, the network type used in the current time period is 4g, the network type used in the next time period is WIFI, and a version number and update time are added when the feature is updated. The super parameter is, for example, a learning rate when the target model is trained, and the super parameter is reported as environmental data, so that it can be known what super parameter the target model is trained by, how the target model is represented, and if the super parameter is not represented, the super parameter can be checked to confirm that the set super parameter is not problematic, for example, too large. The model training timeline is used for representing the update condition of the target model, for example, the target model is trained in the morning, but the training resources in the morning are not enough to cause the target model to fail to go on line, for example, the target model goes on line in the afternoon, and then the model training timeline can be used for knowing which time period is used by the old model or the new model. In the model training time line, data acquisition time, sample generation time, model training time and model online time are displayed on a preset page.

The monitoring data may be referred to as monitoring system information, as shown in fig. 2, the monitoring data report 203 mainly includes a feature quality monitoring system and an ABTest monitoring system, the feature quality monitoring system mainly collects data related to feature update failure of the target model, including feature quality after feature version update, such as request failure number, null rate, and the like, and assists in analyzing changes of model performance and indexes caused by the feature quality. The failure of feature update cannot be directly judged, but the existence possibility of the failure is judged according to the time delay of the system. When the features are updated, the program bug or the system reason may be that the update of the up-going features fails or the male and female features in the features are empty. The ABTest monitoring system mainly collects the final effect data of the model, thereby realizing the on-line comparison effect of the effect data and the characteristic quality and assisting in analyzing the index and the performance change degree of the model after the model is on line. The monitoring data may also include data of the underlying architecture on which the deployment service depends, such as whether the underlying architecture is problematic or not, and if the underlying architecture is problematic, the service is terminated. The effect data collected by the abest monitoring system may be correlated and compared with the characteristic quality in the same time period to determine whether the effect data is degraded when the characteristic quality is degraded, or the effect data collected by the abest monitoring system may be correlated and compared with the environmental data in the same time period to determine whether the environmental data is degraded, for example, the effect data is degraded when the QPS is too high, which will be described below.

And uploading the relevant data of the target model, namely the environmental data and the monitoring data by adopting an agreed resolution protocol and adopting a Software Development Kit (SDK). The uploaded data not only comprise environment data indexes and monitoring data indexes, but also comprise corresponding identification information such as scene identification and the like, and the uploaded data are used as unique identification to determine a scene when data are visualized later.

108: and interpreting the target model by using at least one of the target model related data and the first interpretation result to obtain a second interpretation result.

Step 108 may be implemented by:

In one embodiment, establishing a data association relationship with respect to at least one of the target model-related data and the first interpretation result may be achieved by: acquiring target model related data in a preset time period of the target model; and establishing an incidence relation between the target model related data and the time point in the preset time period. Specifically, obtaining target model-related data within a preset time period of the target model may be implemented by: receiving a data comparison instruction, wherein the data comparison instruction carries a data identifier and time segment information; and searching the relevant data of the target model corresponding to the data identification in the time period corresponding to the time period information.

Taking target model related data as environment data QPS as an example, a user inputs a data comparison instruction through the service monitoring system 209, the instruction carries a data identifier for representing QPS and time period information, namely 2021-04-10 full days and 2021-04-13 full days, finds QPS data in the time period, and establishes an association relationship between QPS and each time point in the time period. After visualization, a QPS versus time plot of 2021-04-10 and 2021-04-13 as shown in FIG. 3 is generated. As shown, the horizontal axis represents time, the vertical axis represents how many requests per second, QPS in different time periods has different heights, QPS of 04-10 and QPS of 04-13 are very close in normal time period, but the difference between QPS of 04-13 and QPS of 04-10 in two time periods is relatively large, and business personnel find that the time period is changed and is abnormal compared with the past historical data. The QPS has a high difference, so that the delay difference is large, and the delay is abnormal. As shown in fig. 4, there is an abnormality in the two time period delays corresponding to the two time periods having a large difference in QPS in fig. 3.

In the above, the chart is customized by the user, and the distribution of the chart is seen. In addition, basic data templates, such as a data plane view, namely a graph of horizontal axis time and vertical axis samples, a line graph and the like, can also be directly set for a graph with high utilization rate, if a traffic abnormality is suddenly found, the line graph is used for showing a QPS, the QPS is too high, and the delay is too high, so that a consumer feels stuck.

In another embodiment, establishing the data association relationship between the target model-related data and at least one of the first interpretation results may be further implemented by:

The following description will be given by taking the example of establishing the association between the target model related data and the effect change data. The ABTest monitoring system mainly collects final effect data of the model, so that an on-line comparison effect of the effect data and the characteristic quality is achieved, the effect data collected by the ABTest monitoring system can be correlated and compared with the characteristic quality in the same time period, whether the effect data are poor when the characteristic quality is poor is confirmed, the effect data collected by the ABTest monitoring system can be correlated and compared with environment data in the same time period, and whether the environment data are poor, such as the QPS is too high is confirmed. As shown in fig. 5, the horizontal axis represents time, the vertical axis represents the feature failure rate, and the matching is performed when the model effect data is poor in a period in which the feature failure rate is high. If not, the abnormal condition exists and other reasons are searched. In one embodiment, the gender characteristics of the user, namely male and female distribution, are correlated and visualized with whether the click is finally made, and whether the male and female characteristics reported during sample collection have a problem is verified.

In one embodiment, the method for interpreting a model in a service scenario further comprises:

The self-defined logic is that QPS exceeds 1000, for example, and alarm information is sent out when QPS exceeds 1000, for example, the self-defined logic is that the real-time characteristic 'song list playing times' is less than 10000 times but the recommendation times exceeds a preset threshold value unreasonably, and alarm information is sent out when the real-time characteristic is that the number of times is not reasonable. The user configures the relevant indexes through the service monitoring system 209 to perform a custom logic or a custom monitoring alarm mechanism, thereby realizing the alarm notification for the user, such as mail alarm notification.

Before the data association relationship is established, the method further comprises the following steps: detecting abnormal data in the target model related data; and repairing or deleting the abnormal data. Optionally, after repairing or deleting the abnormal data, the method further includes: verifying whether abnormal data still exist; and recording the data cleaning information to form a data report and outputting the data report under the condition that abnormal data does not exist.

The user inputs a scene binding instruction through the service monitoring system 209 shown in fig. 2, binds the user with the scene identifier, and displays the visual related diagram to the user through the bound scene service.

According to the method for explaining the recommendation model in the service scene, the model performance and the core index of the model in the service scene are explained from multiple aspects of model training samples and service data interpretable indexes; in a general machine learning scene, the data reporting of a plurality of processes is combined, the whole data link of the model of the floor service is communicated, attribution statistics of model performance and indexes is provided, the interpretable indexes of algorithm service personnel are introduced, the black box characteristic of a complex model is solved, and the interpretability of the whole machine learning service scene is greatly improved.

Fig. 6 schematically shows a flowchart of a method for explaining a recommendation model in a business scenario according to an embodiment of the present invention in a specific music recommendation scenario.

Referring to fig. 6, a flowchart of a method for explaining a recommendation model in a business scenario according to an embodiment of the present invention includes the following steps:

602: when a user enters a music playing platform, acquiring the played times of a song list of the user and the historical song list playing times of the user;

604: calculating the score of the song list according to the playing times of the song list and the historical playing times of the song list of the user, and recommending the song list with the highest score to the user;

606: collecting real-time behavior data representing clicking or not clicking performed by a user aiming at the recommended song list;

608: splicing the played times of the song list, the historical song list playing times of the user and the behavior data to form a real-time sample, and training a recommendation model applied to a music playing platform;

610: uploading the real-time sample to an interpretation engine, and interpreting the correlation between the real-time behavior data and the played times of the song list and the historical song list playing times of the user by using an interpretation template in the interpretation engine to obtain a first interpretation result;

taking the 4 samples of table 1, each sample has three fields, the first two fields being features and the third field being behavior. For example, the first row to the 4 th row in table 1 represent samples of users a to D, and when training the recommendation model, the probability of clicking when the training target is the number of times that the song sheet has been played is small, that is, the probability of the user generating a clicking action is small. The interactive analysis is carried out by the 'song list playing times' and whether the song list is clicked, and the user can not click the song list when the 'song list playing times' is less than 10000 times, the possible reason is that the song list playing times can effectively indicate the popularity of the song list, and the user can not click the song list when the 'user history song list playing times' is less, the possible reason is that the 'user history song list playing times' indicates the interest of the user in playing the song list to a certain extent, and the interest of the user in playing the song list is greater than a certain threshold value, so that the song list recommendation can be accepted more easily. Under the condition that the number of times of playing the song list is set to be less than 10000 times in the explanation template, and the user cannot click the song list, if the number of times of playing the song list in the real-time sample is 8000 times, the correlation between the behavior data and the real-time characteristics can be explained.

612: displaying the first interpretation result into chart information by using a preset visual data table and storing the chart information into MySQL;

614: obtaining a QPS and a time delay of a recommended model within a preset time period; establishing an association relation between the QPS and the time delay and a time point in a preset time period and carrying out visual display;

616: obtaining effect change data and data related to feature failure in a preset time period of a recommendation model, and establishing an association relation with a time point for visual display;

618: and acquiring custom logic about the QPS, and sending alarm information when the QPS exceeds a preset value.

Exemplary Medium

Having described the method of the exemplary embodiments of the present invention, the media of the exemplary embodiments of the present invention will be described next.

In some possible embodiments, aspects of the present invention may also be implemented as a medium having stored thereon program code for implementing steps in a method of interpreting a model under a business scenario according to various exemplary embodiments of the present invention described in the above section "exemplary methods" of this specification, when the program code is executed by a processor of a device.

Specifically, the processor of the device, when executing the program code, is configured to implement the following steps:

collecting core data related to target model training in a business scene;

interpreting the core data to obtain a first interpretation result;

The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the method for explaining the model in the service scenario belong to the same concept, and details of the technical solution of the storage medium, which are not described in detail, can be referred to the description of the technical solution of the method for explaining the model in the service scenario.

It should be noted that: the above-mentioned medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take a variety of forms, including, but not limited to: an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device over any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., over the internet using an internet service provider).

Exemplary devices

Having described the media of the exemplary embodiments of the present invention, an apparatus for explaining a model under a business scenario of the exemplary embodiments of the present invention will be described with reference to fig. 7.

Fig. 7 schematically shows a block diagram of an apparatus for explaining a model under a service scenario according to an embodiment of the present invention.

Referring to fig. 7, an apparatus 700 for explaining a model under a service scenario according to an embodiment of the present invention includes:

a first collecting module 702 configured to collect core data related to target model training in a business scenario;

a first interpretation module 704 configured to interpret the core data to obtain a first interpretation result;

a second collection module 706 configured to collect object model related data, the object model related data comprising at least one of: environmental data relating to an application environment of the target model and monitoring data relating to the target model;

a second interpretation module 708 configured to interpret the object model using at least one of the object model related data and the first interpretation result to obtain a second interpretation result.

accordingly, the first interpretation module is further configured to:

acquiring real-time characteristics of a user;

uploading the real-time samples to an interpretation engine;

accordingly, the first interpretation module is further configured to:

and the repairing module is configured to repair or delete the abnormal data.

Exemplary computing device

Having described the method, medium, and apparatus of the exemplary embodiments of the present disclosure, a computing device in accordance with another exemplary embodiment of the present disclosure is described next.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.

In some possible implementations, a computing device according to an embodiment of the invention may include at least one processor, and at least one memory. Wherein the memory stores program code which, when executed by the processor, causes the processor to perform the steps of the method of interpreting a model under a business scenario according to various exemplary embodiments of the present invention described in the "exemplary methods" section above in this specification. For example, the processor may perform the steps as shown in fig. 1:

collecting core data related to target model training in a business scene;

interpreting the core data to obtain a first interpretation result;

The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the method for explaining the model in the service scenario belong to the same concept, and details of the technical solution of the computing device, which are not described in detail, can be referred to the description of the technical solution of the method for explaining the model in the service scenario.

As another example, the processor may also perform the steps as shown in fig. 6.

It should be noted that although in the above detailed description reference is made to a number of units or sub-units of the apparatus being modeled under a traffic scenario, such partitioning is merely exemplary and not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit according to embodiments of the invention. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Moreover, while the operations of the method of the invention are depicted in the drawings in a particular order, this does not require or imply that the operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.

While the spirit and principles of the invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the particular embodiments disclosed, nor is the division of the aspects, which is for convenience only as the features in these aspects may not be combined to benefit from the present disclosure. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method of interpreting models in a business scenario, comprising:

collecting core data related to target model training in a business scene;

interpreting the core data to obtain a first interpretation result;

2. The method for interpreting models in a business scenario of claim 1, wherein the core data includes real-time samples of the object model and intermediate files related to training of the object model, and collecting core data related to training of the object model in the business scenario includes:

3. The method of interpreting a model under a business scenario of claim 2, wherein the model is a recommendation model that collects real-time samples of an object model in response to a user's real-time behavior, comprising:

acquiring real-time characteristics of a user;

4. The method of interpreting models in a business scenario of claim 2, wherein said models are classification models that collect real-time samples of object models in response to real-time behavior of a user, comprising:

5. The method of interpreting models in a business scenario of claim 2, wherein collecting real-time features of the target model and the marking behavior data comprises:

acquiring real-time characteristics of a user;

6. The method for interpreting a model under a business scenario according to claim 2, wherein interpreting the correlation between the real-time behavior data and the real-time feature or the correlation between the marking behavior data and the real-time feature to obtain a first interpretation result includes:

uploading the real-time samples to an interpretation engine;

7. The method for interpreting a model under a business scenario of claim 6, wherein said interpretation engine comprises a real-time interpretation engine and an offline interpretation engine,

8. A medium having stored thereon a program which, when executed by a processor, implements a method of interpreting a model under a service scenario as claimed in any one of claims 1 to 7.

9. An apparatus for interpreting a model under a business scenario, comprising:

10. A computing device, comprising: a processor and a memory, the memory storing executable instructions, the processor being configured to invoke the memory stored executable instructions to perform the method of interpreting a model under a business scenario as claimed in any one of claims 1 to 7.