CN111428882A

CN111428882A - Processing method and computer equipment

Info

Publication number: CN111428882A
Application number: CN202010230859.7A
Authority: CN
Inventors: 穆森·普尔瓦利; 高长安
Original assignee: Lenovo Beijing Ltd
Current assignee: Lenovo Beijing Ltd
Priority date: 2020-03-27
Filing date: 2020-03-27
Publication date: 2020-07-17

Abstract

The processing method computer equipment provides a monitoring environment for the artificial intelligence model, obtains monitoring data of preset monitoring indexes of the artificial intelligence model at different times, determines changes of the model monitoring indexes at different times based on the monitoring data, and finally judges whether the artificial intelligence model needs to be updated according to whether the changes of the monitoring indexes at different times meet the change conditions. The method analyzes the prediction capability and the rationality of the model bottom layer prediction function through the change of the preset monitoring index of the monitoring model at different time, determines whether the artificial intelligence model needs to be updated (representing whether the model prediction capability meets the expectation or not) by further judging whether the change meets the change condition, and judges the proper update time of the artificial intelligence model, thereby providing an intelligent feedback cycle based on the proper time for the model, effectively ensuring the future processing performance of the model and ensuring the credibility of the model.

Description

Processing method and computer equipment

Technical Field

The application belongs to the field of artificial intelligence, and particularly relates to a processing method and computer equipment.

Background

With the development of science and technology, an Artificial Intelligence (AI) new era has been introduced, which uses Machine learning (Machine L earning, M L) as a core, and the AI has been applied to many fields such as science and technology, finance, law, medicine, military and the like.

However, over time, for example, after several days, weeks, months or years, the input data of the model may change greatly, so that the model may not adapt to the input data well, which makes the model possibly become a poor-performance and untrustworthy model in the future.

Disclosure of Invention

In view of the above, the present application provides a processing method and a computer device for determining an update timing of an artificial intelligence model, so that the update of the artificial intelligence model can be triggered at an appropriate timing to ensure the future processing performance of the model and ensure the credibility of the model.

Therefore, the application discloses the following technical scheme:

a method of processing, comprising:

acquiring monitoring data of preset monitoring indexes of the artificial intelligence model at different times;

determining, based on the monitoring data, a change in the predetermined monitoring indicator at the different time;

determining whether the change of the preset monitoring index generated at different time meets a change condition or not to obtain a determination result;

determining whether the artificial intelligence model needs to be updated or not based on the determination result, and generating a first output result; the first output result can be used to indicate whether the artificial intelligence model needs to be updated.

In the above method, preferably, the obtaining of the monitoring data of the predetermined monitoring index of the artificial intelligence model at different times includes at least one of:

obtaining first monitoring data of important features of the artificial intelligence model at different times; the important features include: the input data of the artificial intelligence model comprises characteristics which have influence on the model prediction result and meet influence conditions;

obtaining second monitoring data of the model performance of the artificial intelligence model at different times;

third monitoring data of the modulus input of the artificial intelligence model at different times is obtained.

The above method, preferably, wherein:

the second monitoring data comprises flag rate data of the artificial intelligence model; the flag rate of the model is: the quantity proportion of results with different polarities in different prediction results of different input data by the model, wherein the polarity is the polarity of the prediction result preset aiming at the input data;

the third monitoring data includes: the input data of the artificial intelligence model includes feature values of features.

The method preferably, the determining whether the change of the predetermined monitoring index generated at the different time satisfies a change condition obtains a determination result, and includes at least one of:

determining whether the changes of the important features generated at different times meet a first change condition or not based on the first monitoring data to obtain a first sub-result;

determining whether the change of the model performance generated at different time meets a second change condition or not based on the second monitoring data to obtain a second sub-result;

determining whether the change of the model input at different time meets a third change condition or not based on the third monitoring data to obtain a third sub-result;

the determining whether the artificial intelligence model needs to be updated and generating a first output result includes:

determining whether the artificial intelligence model needs to be updated based on at least one of the first sub-result, the second sub-result, and the third sub-result, and generating a first output result.

The above method, preferably, further comprises:

determining whether target features which do not meet the correlation conditions with the prediction results of the artificial intelligence models exist in the features of the model input data; if so, generating a second output result comprising the target feature; the second output result can be used for indicating that model updating is required according to the target characteristic;

and/or the presence of a gas in the gas,

and generating a model prediction rule according to different incidence relations between different features and the prediction result of the artificial intelligence model, so that the artificial intelligence model is updated based on the model prediction rule.

A computer device, comprising:

a memory for storing at least one set of instructions;

a processor for invoking and executing the set of instructions in the memory, by executing the set of instructions:

The computer device, preferably, the processor obtains monitoring data of the predetermined monitoring index of the artificial intelligence model at different times, and the monitoring data includes at least one of the following:

The computer device described above, preferably, wherein:

The computer device, preferably, the processor determines whether the change of the predetermined monitoring index at the different time satisfies a change condition, and obtains a determination result, where the determination result includes at least one of:

the processor determines whether the artificial intelligence model needs to be updated and generates a first output result, including:

The computer device is preferably configured such that the processor is further configured to:

and/or the presence of a gas in the gas,

According to the scheme, the processing method disclosed by the application provides a monitoring environment for the artificial intelligence model, obtains the monitoring data of the preset monitoring indexes of the artificial intelligence model at different times, determines the changes of the preset monitoring indexes of the model at different times based on the monitoring data, and finally judges whether the artificial intelligence model needs to be updated according to whether the changes of the preset monitoring indexes at different times meet the change conditions. The method analyzes the prediction capability and the rationality of the model bottom layer prediction function through the change of the preset monitoring index of the monitoring model at different time, determines whether the artificial intelligence model needs to be updated (representing whether the model prediction capability meets the expectation or not) by further judging whether the change meets the change condition, and judges the proper update time of the artificial intelligence model, thereby providing an intelligent feedback cycle based on the proper time for the model, effectively ensuring the future processing performance of the model and ensuring the credibility of the model.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

FIG. 1 is a schematic flow chart of a processing method provided by an embodiment of the present application;

FIG. 2 is another schematic flow chart diagram of a processing method provided in an embodiment of the present application;

FIG. 3 is a graphical illustration of feature importance/importance levels provided by embodiments of the present application;

FIG. 4 is a graph illustrating a mark rate curve provided by an embodiment of the present application;

FIG. 5 is a schematic diagram illustrating processing logic for determining a model update timing based on model monitoring according to an embodiment of the present application;

FIG. 6 is a schematic flow chart of a processing method provided in the embodiments of the present application;

FIG. 7 is a schematic flow chart of a processing method provided in an embodiment of the present application;

fig. 8 is an exemplary diagram of an application of the processing method of the present application to determine a model update timing and perform a model update process according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a computer device provided in an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The embodiment of the application discloses a processing method and computer equipment, and the scheme provides a monitoring environment for an artificial intelligent model to be used as some important indexes related to the model, such as model characteristics (important model characteristics), model performance, model input (characteristic values of input data) and the like. The processing method and the computer device of the present application will be described in detail below with specific embodiments.

In an alternative embodiment of the present application, a processing method is disclosed, which can be applied to and run in a computer device such as a portable computer (e.g., a notebook), a desktop computer or a large and medium-sized computer, a background server or a cloud platform server in a general/special purpose computing or configuration environment.

As shown in fig. 1, the processing method disclosed in this embodiment may include the following processing procedures:

step 101, obtaining monitoring data of preset monitoring indexes of the artificial intelligence model at different times.

In this embodiment, the artificial intelligence model refers to an M L model constructed by machine learning, which is a core technology based on artificial intelligence, such as a linear classification model, a support vector machine model, a deep learning model, and the like.

The predetermined monitoring indexes of the artificial intelligence model select some important indexes which are related to the artificial intelligence model and can directly or indirectly reflect the behavior ability of the model bottom layer, wherein the important indexes can include but are not limited to indexes of three aspects of model characteristics (important model characteristics), model performance and model input (characteristic values of input data).

In specific implementation, a monitoring environment can be established for the model to be detected, monitoring and outputting of model index data are performed in the monitoring environment, and based on the monitoring environment, model data in the following aspects can be monitored without limitation:

1) monitoring important characteristics of the model;

here, the model feature specifically refers to a feature of input data of the model; the model input data generally has multidimensional characteristics, and each multidimensional characteristic contributes to a decision result in a prediction decision of the model in a higher or lower way; for example, assuming that the input data of the model is personal information of the website user, in this embodiment, the model features (i.e., the features of the input data of the model) may be features such as gender, age, mailbox, hobbies, and the like, and the feature value of each feature may affect the decision result of the model prediction process more or less.

In the embodiment of the application, for monitoring the model characteristics, the characteristics which can generate the influence meeting the influence condition on the model prediction result in the characteristics included in the input data of the main monitoring model are taken as the important characteristics of the model.

The influence condition may be, for example and without limitation, that the importance or the level of the feature reaches a set importance threshold or a level threshold (for example, the feature is mapped into a plurality of levels according to the influence of the feature on the model decision result, and different levels represent different degrees of influence on the model decision result); alternatively, the influencing condition may also be: the features belong to TOP TOP N features after ranking the features in an importance/rank descending manner, wherein N is a non-0 natural number.

2) Monitoring the performance of the model;

one or more of the models can be monitored specifically for indications of model performance, such as accuracy, precision, and recall of the model.

3) And monitoring model input.

The model input here refers to the characteristic values of the model input data, such as the actual values of the characteristics of gender, age, mailbox, hobby, and the like in the personal information of the website user.

And 102, determining the changes of the preset monitoring indexes at different times based on the monitoring data.

For some selected proper monitoring indexes, the data changes and fluctuations thereof can generally directly or indirectly reflect the decision-making capability of the model bottom layer and the changes of the model behaviors, which correspondingly enables the rationality of the model prediction function to be explained, so that the embodiment analyzes whether the rationality of the potential prediction function of the model can be expected or not and whether the model needs to be updated or not by analyzing the changes of the index data of the selected three monitoring indexes at different times.

The different time may be a different time determined from a certain historical time to the current time based on a fixed set time, and if the fixed set time is one month, three months or one year, the different time is a different time between the previous month/previous three months/previous year and the current time, in this embodiment, the different time may be determined based on a sliding window manner (having a fixed sliding step size, such as the above-mentioned one month, three months or one year) with the current time as the time end.

Alternatively, in another embodiment, the different time may be a different time covered by a time period from a certain set fixed starting time to the current time, for example, assuming that the fixed starting time is a time half a year ago, the different time is a time between the time half a year ago and the current time, the time span of the different time gradually increases with the passage of time, such as increases of 7 months, 8 months, etc., and then the starting time may be updated when the time span increases to an upper limit (such as one year) according to a policy, without limiting the determination manner of the different time.

In implementation, the important characteristics of the model, the performance of the model and the changes of the model input at different times can be analyzed by taking the output of the monitoring environment at different times as a data basis.

And 103, determining whether the changes of the preset monitoring indexes at different times meet change conditions or not, and obtaining a determination result.

And then, whether the important characteristics, the model performance and the changes of the model input at different time meet the change conditions can be further determined, and when the important characteristics, the model performance and the changes of the model input at different time meet the change conditions, the bottom layer decision-making capability and the model behavior of the model are considered to be greatly changed, the rationality of the bottom layer prediction function of the model is correspondingly considered not to be guaranteed, and when the change conditions are not met, the rationality of the model bottom layer prediction function can be considered to be guaranteed.

The change condition may be configured to: the change in the metric exceeds a set maximum allowable change, which may be based on a change in the mapped model metric data that tolerates the maximum change in the model decision making capability.

In implementation, the specific value of the "maximum allowable variation" in the condition may be determined according to the actual application environment of the model engineering, the accuracy/precision requirement of the model, and by combining experience.

104, determining whether the artificial intelligence model needs to be updated or not based on the determination result, and generating a first output result; the first output result can be used to indicate whether the artificial intelligence model needs to be updated.

By performing the condition determination on the change of the monitoring index at different times based on the set change condition, a determination result (i.e., the above determination result) indicating at least whether the change of the monitoring index at different times satisfies the change condition or does not satisfy the change condition can be obtained.

And the judgment result meeting the change condition or not meeting the change condition correspondingly triggers to generate and display an output result, namely the first output result, wherein the first output result at least can be used for indicating whether the artificial intelligence model needs to be updated or not.

The monitoring index meets the change condition, namely the bottom decision making capability and the model behavior of the model are greatly changed, and the model capability and the model credibility are not ensured, so that a first output result which can be used for indicating that the model needs to be updated is generated and displayed, and subsequently, the updating process of the model can be automatically started by the computer equipment which provides the model maintenance function through responding to the first output result, or the updating process of the model is manually started on the computer equipment under the manual intervention; on the contrary, the monitoring index does not satisfy the change condition, that is, the bottom-layer decision-making capability and the model behavior of the model are not greatly changed, and the model capability and the model credibility can be ensured, so that a first output result which can be used for indicating that the model does not need to be updated is generated and displayed, and certainly, no information can be output, and the model does not need to be updated implicitly.

It should be noted that, in practical applications, there may be situations where the machine cannot definitely determine whether the model needs to be updated, for example, an absolute difference between a change of the monitoring index and a set maximum allowable change is smaller than a set threshold value, which increases the determination difficulty, or changes of some indexes in different indexes satisfy a change condition and other indexes do not satisfy the change condition. Whether the monitoring analysis result of whether the model needs to be updated is explicitly given by the model or not, or whether the model needs to be updated is determined by the model triggering and the discussion process of human intelligence and based on the intervention of the human intelligence or not is within the protection scope of the scheme of the application.

The embodiment analyzes the prediction capability and the rationality of the model bottom layer prediction function through the changes of the preset monitoring indexes of the monitoring model at different time, determines whether the artificial intelligence model needs to be updated (representing whether the model prediction capability meets the expectation or not) by further judging whether the changes meet the change condition or not, and judges the proper update time of the artificial intelligence model, thereby providing an intelligent feedback cycle based on the proper time for the model, effectively ensuring the future processing performance of the model and ensuring the credibility of the model.

The following describes in detail a specific implementation process of monitoring and analyzing the model data of the three monitoring indexes to determine the reasonability of the model prediction function and further determine whether the model needs to be updated (the model update time is given correspondingly).

Referring to fig. 2, in this embodiment, based on the three monitoring indexes, the processing method may determine whether the model needs to be updated (i.e. the update time of the model) through the following processing procedures:

step 201, obtaining first monitoring data of important features of the artificial intelligence model at different times, second monitoring data of model performance at different times and third monitoring data of analog-digital input at different times.

For monitoring of important features of the model, L IME is optionally used in the present embodiment to detect important features of the model in each prediction, wherein L IME is an interpretation tool of the machine learning model, which can be used to interpret predictions of any machine learning model, which uses system input based on M L as model input, observes how predictions change through disturbance input, and interprets outputs of the M L model based on mappings between input disturbances and model predicted changes, and in monitoring of important features of the model, L IME tool is used, which can specifically obtain important features of the model (i.e., features that can cause model predictions to change more greatly when disturbed) and observe predicted changes caused by disturbances, and obtain importance monitoring data of the features at the same time.

When monitoring the model performance, because the randomness of data such as the accuracy, precision or recall ratio of the model per day is high, it is difficult to objectively and accurately obtain the model performance based on the data, and accordingly it is difficult to analyze the change of the model performance based on the change of the data, the application proposes to use the flag rate (flag) of the model to reflect the model performance, and to reflect the change of the model performance with the change of the flag rate of the model.

Wherein, the mark rate of the model is as follows: the number proportion of results of different polarities in different predicted results of the model for different input data.

The polarity is a polarity of a prediction result set in advance for input data. The polarity of the prediction result may be divided in different ways, for example, the prediction result may be divided into positive and negative polarities, or good and bad polarities, or 0, 1 polarities, etc. according to different dividing ways.

For example, it is assumed that the predicted results of the model include 3 categories: the

categories

1 and 3 are classified into positive polarity, and the category 2 is classified into negative polarity, etc., and the classification of the prediction result polarity may be flexibly set by a technician, without limitation.

For a particular model operating scenario, the present embodiment considers that the model performance has changed when the model flag rate changes.

In addition, applicants believe that any significant changes in the model input data will likely result in the model's existing underlying decision-making capability no longer being able to better accommodate the input data after the significant changes have occurred, and therefore, any significant changes in the model input data should be studied in order to make a decision as to whether to update the model or discuss it with human intelligence.

For monitoring of model input, feature values of model input data can be detected in real time, for example, for an intelligent recommendation system and an intelligent question-and-answer system of a certain website, when personal information of a user is used as system input or part of the system input to participate in system decision, the personal information of the user using the system can be monitored in real time, and important changes of the system user information are found, for example, the system user changes from most of 'professional one' with a certain technical background to most of 'professional two' without the technical background, or the system user changes from most of women to most of men (all of which can be used as important changes of input data).

Step 202, determining the changes of the important features at different times based on the first monitoring data; determining, based on the second monitoring data, a resulting change in the model performance at the different time; based on the third monitoring data, changes in the model inputs that occur at the different times are determined.

As described above, in order to facilitate observation and analysis, the present embodiment combines all input features of the model using a fusion technique and generates a single-ranked list of the important features in each prediction, so that whether and how much the important features of the model change can be determined based on the single-ranked list of the important features in each prediction.

More specifically, the present embodiment selects TOP N features (i.e., TOP N significant features) from each single-rank list, and generates a significance/significance level curve of the TOP N features in 2-Dim (two-dimensional) space, as shown in fig. 3, in the feature significance/significance level curve, each feature has a curve based on a function fr _ i (t), where i represents different features, t represents time (e.g., day/week/month), corresponding to an X axis (horizontal axis), and fr _ i (t) represents the significance/significance level of the feature, corresponding to a Y axis (vertical axis). Along with the change of the feature importance/importance level in the running process of the model, the height trend of the curve can change, and accordingly the change of the important feature can be obtained by observing and analyzing the change of the feature importance/importance level curve.

Similarly, a signature rate curve is plotted for the model signature rate, as an example of a signature rate curve is provided in fig. 4, with the X-axis (horizontal axis) representing time (e.g., days/weeks/months) and the Y-axis (vertical axis) representing the signature rate, and the model signature rate changes are analyzed by observing the signature rate curve, which correspondingly reflect changes in model performance. In addition, a characteristic value curve is correspondingly drawn for the monitored model input data, and the change generated by the model input is obtained through the observation and analysis of the change of the characteristic value curve.

Thus, three curves may ultimately be output based on model monitoring: characteristic importance curve, mark rate curve, characteristic value curve. The significant change in each curve needs to be interpreted by human intelligence to analyze its possible cause, e.g., a significant change in the curve between

time

8 and 9 in fig. 3, and the percentage of the significant change (e.g., how many feature curves have changed in importance level, the proportion of the changed curve in all curves, etc.) can be calculated for use in the decision of whether to change the model.

Step 203, determining whether the variation of the important features generated at the different time meets a first variation condition, and obtaining a first sub-result.

The first change condition is a condition which can be used for representing that the important characteristic of the model is changed significantly.

One example of the present embodiment is provided below. With reference to fig. 3, the first condition may be set as: the characteristic curves with at least P ═ A% in the curves of the important characteristics are crossed with each other, the crossed state can be maintained for a set time (such as T ═ 1 month) and A is a set constant, such as 10, 20 and the like. The importance/importance level of the original feature I is higher than/lower than that of the feature II, and the importance/importance level of the feature I is lower than/higher than that of the feature II.

Then, whether the important feature has significantly changed at different times can be determined by taking the set first change condition as a criterion to determine the change of the important feature at different times, so as to obtain a first sub-result, which can be but is not limited to a binary result of "0" or "1", where "1" indicates that the change of the important feature satisfies the first change condition, and "0" indicates that the change of the important feature does not satisfy the first change condition.

And 204, determining whether the change of the model performance generated at different time meets a second change condition to obtain a second sub-result.

The second change condition is a condition that can be used to characterize a significant change in the model performance. The change of the mark rate is used for reflecting the change of the model performance.

Alternatively, the second variation condition may be set to any one or more of the following conditions:

1) the marking rate range is increased by at least B1% of the average value of the maximum marking rate values, or the marking rate range is decreased by at least B2% of the average value of the minimum marking rate values; and the state after rising or falling is maintained for a set time;

b1 and B2 are set fixed values, such as 10 and 20.

Referring to fig. 4, the average of the maximum of the mark rate is an average of the respective peak values of the jagged mark rate curve in fig. 4, and the average of the minimum of the mark rate is an average of the respective valley values of the jagged mark rate curve in fig. 4.

2) The minimum/maximum average performance (the average value of the maximum value of the mark rate/the average value of the minimum value of the mark rate) continuously decreases (descending trend), and a set duration is maintained;

3) the minimum/maximum average performance (the average value of the maximum value of the mark rate/the average value of the minimum value of the mark rate) continuously rises (rising trend), and a set time length is maintained;

4) the momentum M1 of N1 days is more than or equal to +/-C1%, and a set time length is maintained;

for example, the momentum M1 is more than or equal to +/-70% in 10 days. Where M1 ═ ((p-p _10)/p _10) × 100, p represents the flag rate of the model at the current time, and p _10 represents the flag rate of the model before N1 days.

If at least one of the above conditions is satisfied, a second sub-result with a result of "1" is obtained, otherwise, a second sub-result with a result of "0" is obtained, where "1" indicates that the variation generated by the model performance at different times satisfies the second variation condition, and "0" indicates that the variation generated by the model performance does not satisfy the second variation condition. Of course, other forms of results are possible, which are not illustrated here, and are within the scope of the present application.

And step 205, determining whether the change of the model input at the different time meets a third change condition, so as to obtain a third sub-result.

The third change condition is a condition that can be used to characterize a significant change in the model input. The change in model input is reflected in particular by a change in the eigenvalues of the model input data.

The feature values of the model input data include two types:

numerical type: such as the user's identification number, the student's school number, etc.;

type of the sample: such as lawyers, nurses, different professions of different users.

For the characteristic value of the numerical type, optionally, the third variation condition may be set to any one or more of the following:

1) the characteristic value is increased by at least D1% of the average value of the maximum value of the characteristic value, or the characteristic value is reduced by at least D2% of the average value of the minimum value of the characteristic value, and the increased or reduced state is maintained for a set time period;

d1 and D2 are fixed values, such as 10 and 20.

The average value of the maximum value of the characteristic value is the average value of the maximum value of the characteristic value in each unit time (such as 1 day) in a period (such as 6 months); the average of the minimum values of the characteristic values is an average of the minimum values of the characteristic values in each unit time (e.g., 1 day) over a period of time (e.g., 6 months).

2) The momentum M2 of N2 days is more than or equal to +/-C2%, and a set time length is maintained;

for example, the momentum M2 is more than or equal to +/-70% in 10 days. Where M2 ═(ni-ni _10)/ni _10) × 100, ni represents the feature value of the feature of the model input data at the current time, and p1_10 represents the feature value of the feature of the model input data before N2 days.

For the feature value of the category type, optionally, the third variation condition may be set to any one or more of the following:

1) a range of categories is at least 10% change and maintained for a period of time;

for example, over 10% of the "professions" in the input user information have changed professions in the sample of user information.

2) The momentum M3 of N3 days is more than or equal to +/-C3%, and a set time length is maintained;

for example, the 10-day momentum M3 is more than or equal to +/-70%, and a set time duration is maintained. Where M3 is ((cl-cl _10)/cl _10) × 100, cl is the number of samples corresponding to the feature (e.g., occupation) category value (e.g., attorney) at the present time, and cl _10 is the number of samples corresponding to the feature (e.g., occupation) category value (e.g., attorney) before N3 days.

And for the characteristic values of the numerical value type or the category type, if the characteristic values are judged to meet at least one of the corresponding conditions, obtaining a third sub-result with a result of 1, and otherwise, obtaining a third sub-result with a result of 0, wherein 1 indicates that the change generated by the model input at different time meets a third change condition, and 0 indicates that the change generated by the model input does not meet the third change condition. Similarly, the method is not limited to "1" and "0", and other result forms can be obtained, and are within the protection scope of the present application.

Step 206, determining whether the artificial intelligence model needs to be updated based on at least one of the first sub-result, the second sub-result and the third sub-result, and generating a first output result.

Finally, the first result of the important model-oriented features, the second sub-result of the model-oriented performance and the third sub-result input to the model can be integrated, and whether the artificial intelligence model needs to be updated or not is determined according to a set strategy.

Illustratively, in an alternative embodiment, if the first sub-result, the second sub-result, and the third sub-result are all "1", the machine may directly present a first output that can be used to indicate that a model update is required; otherwise, if the first sub-result, the second sub-result and the third sub-result are all "0", the first output result indicating that no model update is required can be given by the machine, and of course, the machine may not provide any output in this case, and the behavior that the machine does not provide any output implies that no model update is required; for the first sub-result, the second sub-result and the third sub-result, part of the results is "1" and part of the results is "0", it means that human intelligence is needed to discuss, and the decision of whether to update the model is made under human intelligence intervention, therefore, the machine can trigger the process of discussing with human intelligence, for example, the machine outputs a visual interface with prompt information and available for receiving information input, prompts human intelligence to discuss, and inputs related instruction information to the interface, so as to finally decide whether to update the model. The above is only one example of the present application, and flexible policy setting is possible in implementation.

Referring to fig. 5, a schematic diagram of processing logic for determining whether a model needs to be updated by monitoring three aspects of important features, model performance and model input of the model during the running process of the model to provide proper timing of model update is shown, wherein xAI refers to Explainable Artificial Intelligence in full, which refers to interpretable Artificial Intelligence.

In the embodiment, the model is subjected to data monitoring and analysis of the important features, the model performance and the model input, so that the model is explained, whether the model bottom layer prediction capability is credible or not is obtained through the model explanation, and a proper updating time is determined for the model.

In another alternative implementation of the present application, as shown in fig. 6, the processing method may further include the following processing:

and step 207, determining whether target features which do not meet the correlation conditions with the prediction results of the artificial intelligence model exist in the features of the model input data.

Step 208, if yes, generating a second output result comprising the target feature; the second output result can be used for indicating that model updating is required according to the target characteristic;

in this embodiment, for a feature, if its importance/importance level for model prediction reaches at least a set importance/importance level threshold, the feature is considered to be associated with the model prediction result (essentially having high association), otherwise, if the importance/importance level of the feature for model prediction does not reach the set threshold, the feature is considered to be irrelevant to the model prediction result (essentially having low association).

As an important feature of the model, that is, a feature that has a large influence on the model prediction, it should be at least associated with the model prediction result.

In view of this, the above association condition may be an importance/importance level condition of the feature to the model prediction determined according to the actual logical association between the model feature and the model prediction result.

For example, assuming that the model is used to predict the user occupation as a "lawyer" or "lawyer" type according to the input user information, and assuming that the features in the user information include "gender", "skill (such as professional skill), and" age ", then according to the actual logical association between the different features and the model prediction result, the association condition may be set as a condition characterizing the following information:

the importance/grade of the 'skill' characteristic to the model prediction result reaches an importance/grade threshold value Z;

the importance/ranking of the "gender" feature to the model prediction result should be below the importance/ranking threshold Z.

The above conditions are formulated according to the actual logical relationship between the features and the model prediction results, and accord with the actual association condition between the features and the model prediction results.

Based on this assumption, if it is found that the model uses "gender" as an important feature for predicting whether the user occupation is a "lawyer" in the prediction process through the monitoring of the important features of the model, the feature is necessarily not in accordance with the above-mentioned association condition, and thus, the feature is identified as a target feature which does not satisfy the association condition, which essentially realizes the identification of the deviation feature or the error feature of the model.

And then, a second output result at least comprising the target characteristic can be further generated, the second output result not only indicates that the model needs to be updated, but also further indicates the deviation characteristic/error characteristic of the model, so that the model can be updated by correspondingly indicating that the deviation characteristic/error characteristic of the model is used as cut-in, the importance balance of all the characteristics of the model is realized, and the accuracy or precision of model prediction is improved. It is easy to understand that if there is no target feature that does not satisfy the correlation condition with the prediction result of the artificial intelligence model, the generation and output of the second output result need not be performed.

In another alternative implementation of the present application, as shown in fig. 7, the processing method described above may further include the following processing:

step 209, generating a model prediction rule according to different incidence relations between different features and the prediction result of the artificial intelligence model, so that the artificial intelligence model is updated based on the model prediction rule.

In implementation, different association relations between different features and the prediction result of the artificial intelligence model may be formulated in advance according to actual logical associations between the features of the model input data and the model predictions, and a model prediction rule may be generated, where the model prediction rule may be a rule generated by a machine based on logical associations between features learned by big data and model predictions, may be a rule generated by a machine based on logical associations between features provided manually and model predictions, or may be a rule configured manually and directly, and may of course be a combination of any two or three of the above.

Then, the prediction rule may be further applied to update the model, for example, after the model determines a suitable update timing based on the processing method of the present application, a processing flow for updating the model using the prediction rule is triggered, or after the prediction rule is generated or configured, the model may be updated in real time.

A simple example is provided below.

In combination with practical applications, it can be known that, in the prediction of the user occupation, the prediction contribution degree of the "gender" feature to the "lawyer" occupation is very low, while the prediction contribution degree of the "gender" feature to the "nurse" occupation is higher, that is, the user can be more easily distinguished as "nurse" or "non-nurse" based on a certain gender, such as "woman" or "man", and cannot be easily distinguished as "lawyer" or "non-lawyer", and the prediction contribution degree of the "age" feature to the "student" occupation is higher, such as a certain age "17 years", so that the user can be predicted as "student" with high probability, but not other occupations, such as "lawyer", and the following prediction rules can be generated:

"gender" is an important feature for "nurse" category prediction;

"gender" is an unimportant feature of the "lawyer" category prediction;

"age" is an important characteristic of the "student" category prediction.

Then, the prediction rule can be used to update the model, for example, if "gender" is used as an important feature for professional prediction by a "lawyer" in the artificial intelligence model, the performance of the model is difficult to guarantee, and after the model is updated based on the prediction rule, the "gender" can be changed from the important feature predicted by the model to an unimportant feature, so that the prediction accuracy and precision of the model can be further improved.

Referring to fig. 8, a logic example diagram of determining a model update timing and performing a model update process by using the processing method of the present application is provided, where 801 represents that a person (e.g., a technician or an administrator in a model debugging environment, or a front end user in an actual operating environment) controls the M L model 802 to operate at a device front end, after the model 802 is started, data (e.g., collected and stored personal information of the front end user) is imported from the database 803, the data in the database 803 is used as model input to perform a prediction process, and a prediction result can be fed back to a front end device, and at the same time, a background starts monitoring on the model 802, obtains monitoring data of important features of the model, model performance, and model input, and uses the obtained monitoring data as an input of xAI 804, where xAI 804 performs model interpretation based on the monitoring data, outputs a single-rank list of the important features, an importance/importance level curve, a flag rate curve, a feature value curve, and other analysis information, which is marked as 805 in the diagram, and finally determines whether a current model update is needed to be updated, i.e., whether the intelligent model update timing is needed to be determined by combining with the intelligent model update timing 806, and whether the intelligent model update timing can be determined by the background.

In addition, a Bias feature/error feature selection process may be performed in combination with the above corresponding embodiments of the present application, and a Bias feature/error feature that does not satisfy the association condition in the model is determined through a feature analysis 807, and the model is updated, for example, as a feedback from Bias 808 to model 802 in fig. 8. In addition, generation of the prediction Rule and the model updating process based on the prediction Rule can be performed in combination with the above corresponding embodiments of the present application, such as feedback from the Rule generator 809 to the model 802 in fig. 8.

By judging the update time of the model and carrying out the model update processing, the method can determine a proper update time for the model to trigger the model update, and can combine with the discussion of human intelligence when necessary, thereby not only ensuring the future processing performance of the model and the credibility of the model, but also avoiding unnecessary model update workload caused by improper update time of the model.

Corresponding to the processing method, the embodiment of the present application further provides a computer device, which may be, but is not limited to, a portable computer (such as a notebook), a desktop computer or a large and medium-sized computer, a background server or a cloud platform server in a general/special purpose computing or configuration environment.

As shown in fig. 9, the computer apparatus includes:

a memory 901 for storing at least one set of instructions;

a processor 902 for invoking and executing the set of instructions in the memory, by executing the set of instructions:

In an optional implementation manner of the embodiment of the present application, the processor 902 obtains monitoring data of predetermined monitoring indexes of the artificial intelligence model at different times, where the monitoring data includes at least one of:

In an optional implementation manner of the embodiment of the present application, the second monitoring data includes flag rate data of the artificial intelligence model; the flag rate of the model is: the quantity proportion of results with different polarities in different prediction results of different input data by the model, wherein the polarity is the polarity of the prediction result preset aiming at the input data;

In an optional implementation manner of the embodiment of the present application, the processor 902 determines whether a variation generated by the predetermined monitoring index at the different time satisfies a variation condition, and obtains a determination result, where the determination result includes at least one of:

and determining whether the change of the model input at different time meets a third change condition or not based on the third monitoring data to obtain a third sub-result.

The processor 902 determines whether the artificial intelligence model needs to be updated and generates a first output comprising: determining whether the artificial intelligence model needs to be updated based on at least one of the first sub-result, the second sub-result, and the third sub-result, and generating a first output result.

In an optional implementation manner of the embodiment of the present application, the processor 902 may further be configured to:

and/or the presence of a gas in the gas,

For the computer device disclosed in the embodiments of the present application, since it corresponds to the processing method applied to the computer device disclosed in the corresponding embodiments above, the description is relatively simple, and for the relevant similarities, please refer to the description of the processing method in the corresponding embodiments above, and details are not described here.

It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other.

For convenience of description, the above system or apparatus is described as being divided into various modules or units by function, respectively. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.

From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present application.

Finally, it is further noted that, herein, relational terms such as first, second, third, fourth, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A method of processing, comprising:

2. The method of claim 1, wherein obtaining monitoring data of predetermined monitoring metrics of the artificial intelligence model at different times comprises at least one of:

3. The method of claim 2, wherein:

4. The method according to claim 2 or 3, wherein the determining whether the change of the predetermined monitoring index generated at the different time satisfies a change condition results in a determination result, and the determination result comprises at least one of the following:

5. The method of claim 2, further comprising:

and/or the presence of a gas in the gas,

6. A computer device, comprising:

a memory for storing at least one set of instructions;

7. The computer device of claim 6, the processor obtaining monitoring data for predetermined monitoring metrics of the artificial intelligence model at different times, including at least one of:

8. The computer device of claim 7, wherein:

9. The computer device of claim 7 or 8, wherein the processor determines whether a change in the predetermined monitoring indicator at the different time satisfies a change condition, resulting in a determination result comprising at least one of:

10. The computer device of claim 7, the processor further to:

and/or the presence of a gas in the gas,