CN117540826A - Optimization method and device of machine learning model, electronic equipment and storage medium - Google Patents
Optimization method and device of machine learning model, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117540826A CN117540826A CN202311782927.0A CN202311782927A CN117540826A CN 117540826 A CN117540826 A CN 117540826A CN 202311782927 A CN202311782927 A CN 202311782927A CN 117540826 A CN117540826 A CN 117540826A
- Authority
- CN
- China
- Prior art keywords
- machine learning
- learning model
- target
- contribution
- target machine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010801 machine learning Methods 0.000 title claims abstract description 293
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000005457 optimization Methods 0.000 title claims abstract description 45
- 238000003860 storage Methods 0.000 title claims abstract description 11
- 238000012216 screening Methods 0.000 claims description 15
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 description 21
- 238000004458 analytical method Methods 0.000 description 8
- 239000000654 additive Substances 0.000 description 6
- 230000000996 additive effect Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000013136 deep learning model Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000008092 positive effect Effects 0.000 description 3
- 235000008733 Citrus aurantifolia Nutrition 0.000 description 2
- 235000011941 Tilia x europaea Nutrition 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 239000004571 lime Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000004451 qualitative analysis Methods 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
Abstract
The application provides a machine learning model optimization method, a device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring sample data corresponding to a target machine learning model; calculating the contribution degree of each feature in the sample data to the target machine learning model respectively to obtain z first contribution degrees; determining at least one first target contribution among the z first contributions; and optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree to obtain an optimized target machine learning model. According to the method and the device, the sample data with different prediction results in the training process of the target machine learning model are obtained, the plurality of features are obtained according to the sample data, and the influence degree of the different features on the target machine learning model is calculated, so that at least one feature with larger influence on the target machine learning model is determined, the target machine learning model is optimized through the at least one feature, and the optimization efficiency of the machine learning model is improved.
Description
Technical Field
The present disclosure relates to the field of artificial intelligence technologies, and in particular, to a method and apparatus for optimizing a machine learning model, an electronic device, and a storage medium.
Background
With rapid development of artificial intelligence technology mainly based on machine learning, a machine learning model is applied to more and more fields. Model optimization of the machine learning model plays an important role in modeling, so that the performance and efficiency of the model can be improved, and the resource consumption can be reduced. At present, the common optimization method comprises data preprocessing, data cleaning, standardization, normalization and the like, but the mode generally only selects data with correct prediction to analyze and optimize the machine learning model, but cannot comprehensively consider the influence of all prediction data on the machine learning model, so that the problem of lower optimization efficiency of the machine learning model occurs.
Disclosure of Invention
The embodiment of the application provides a machine learning model optimization method, a device, electronic equipment and a storage medium, which solve the problem of low machine learning model optimization efficiency in the prior art.
To solve the above problems, the present application is realized as follows:
in a first aspect, an embodiment of the present application provides a method for optimizing a machine learning model, the method including:
Acquiring first sample data and second sample data corresponding to a target machine learning model, wherein the target machine learning model is different in prediction result obtained by predicting the first sample data and the second sample data, the first sample data comprises x first features, the second sample data comprises y second features, x is a positive integer, and y is a positive integer;
calculating the contribution degree of each feature to the target machine learning model in the x first features and the y second features respectively to obtain z first contribution degrees, wherein the contribution degrees are used for representing the influence degree of the corresponding feature on the output result of the target machine learning model, z is the sum of the x and the y, and z is a positive integer;
determining at least one first target contribution among the z first contributions;
and optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree to obtain an optimized target machine learning model.
Optionally, the acquiring the first sample data and the second sample data of the target machine learning model in the training process includes:
Determining a category of the target machine learning model, the category including a machine learning model of a classification task and a machine learning model of a regression task;
based on the category, acquiring first sample data and second sample data of a target machine learning model in a training process, wherein the first sample data is sample data of a prediction error of the target machine learning model, and the second sample data is sample data of a prediction correctness of the target machine learning model when the category characterizes the target machine learning model as the machine learning model of the classification task; and under the condition that the category characterizes the target machine learning model as the machine learning model of the regression task, the first sample data is sample data of which the target machine learning model prediction error is larger than a preset error, and the two sample data are sample data of which the target machine learning model prediction error is equal to or smaller than the preset error.
Optionally, the calculating the contribution degree of each feature to the target machine learning model in the x first features and the y second features respectively, to obtain z first contribution degrees includes:
Calculating the contribution degree of each of the x first features and the y second features to the target machine learning model respectively to obtain z second contribution degrees, wherein the z second contribution degrees comprise x second contribution degrees corresponding to the x first features and y second contribution degrees corresponding to the y second features;
optimizing x second contribution degrees corresponding to the x first features to obtain x third contribution degrees;
wherein the z first contribution degrees include: and y second contribution degrees corresponding to the y second features and the x third contribution degrees.
Optionally, the optimizing the x contribution degrees corresponding to the x first features to obtain x third contribution degrees includes:
calculating the contribution degree of the x first features to the target machine learning model respectively to obtain x fourth contribution degrees;
and classifying the x fourth contribution degrees to obtain the x third contribution degrees, wherein the third contribution degrees comprise classification identifiers, and the classification identifiers are used for representing whether the corresponding contribution degrees are positive contribution degrees or negative contribution degrees.
Optionally, the determining at least one first target contribution degree in the z first contribution degrees includes:
Sorting the z first contribution degrees in a descending order sorting manner to obtain a sorting set;
screening the sorting set according to a preset threshold value, and determining at least one first target contribution degree, wherein the first target contribution degree is larger than or equal to the preset threshold value.
Optionally, the optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree, and obtaining the optimized target machine learning model includes:
updating the at least one target contribution based on at least one target feature corresponding to the at least one target contribution to obtain at least one second target contribution, wherein the at least one first target contribution corresponds to the at least one second target contribution one by one;
screening the at least one second target contribution degree and determining at least one third target contribution degree;
and optimizing the target machine learning model based on at least one feature corresponding to the at least one third target contribution degree to obtain an optimized target machine learning model.
Optionally, the optimizing the target machine learning model based on at least one feature corresponding to the at least one third target contribution degree, to obtain an optimized target machine learning model, includes:
Generating optimization information based on the at least one third target contribution;
optimizing the target machine learning model based on the optimization information to obtain an optimized target machine learning model;
wherein the optimization information includes at least one of: adjusting a weight value corresponding to the target feature in the target machine learning model, adding a first sample feature of the target machine learning model, deleting a second sample feature of the target machine learning model, and preprocessing a training sample of the target machine learning model.
In a second aspect, embodiments of the present application further provide an apparatus for optimizing a machine learning model, where the apparatus includes:
the device comprises an acquisition module, a prediction module and a prediction module, wherein the acquisition module is used for acquiring first sample data and second sample data corresponding to a target machine learning model, the target machine learning model is different in prediction result obtained by predicting the first sample data and the second sample data, the first sample data comprises x first features, the second sample data comprises y second features, x is a positive integer, and y is a positive integer;
the computing module is used for respectively computing the contribution degree of each feature to the target machine learning model in the x first features and the y second features to obtain z first contribution degrees, wherein the contribution degrees are used for representing the influence degree of the corresponding feature to the output result of the target machine learning model, z is the sum of the x and the y, and z is a positive integer;
A determining module for determining at least one first target contribution among the z first contributions;
and the optimization module is used for optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree to obtain an optimized target machine learning model.
In a third aspect, embodiments of the present application further provide an electronic device, including: a memory, a processor, and a program stored on the memory and executable on the processor; the processor is configured to read a program in the memory to implement the steps in the method according to the foregoing first aspect.
In a fourth aspect, embodiments of the present application further provide a readable storage medium storing a program, which when executed by a processor implements the steps of the method according to the first aspect.
The application provides a machine learning model optimization method, a device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring first sample data and second sample data corresponding to a target machine learning model, wherein the target machine learning model is different in prediction result obtained by predicting the first sample data and the second sample data, the first sample data comprises x first features, the second sample data comprises y second features, x is a positive integer, and y is a positive integer; calculating the contribution degree of each feature to the target machine learning model in the x first features and the y second features respectively to obtain z first contribution degrees, wherein the contribution degrees are used for representing the influence degree of the corresponding feature on the output result of the target machine learning model, z is the sum of the x and the y, and z is a positive integer; determining at least one first target contribution among the z first contributions; and optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree to obtain an optimized target machine learning model. According to the method and the device, the sample data with different prediction results in the training process of the target machine learning model are obtained, the plurality of features are obtained according to the sample data, and the influence degree of different features on the target machine learning model is calculated, so that at least one feature with larger influence on the target machine learning model is determined, the target machine learning model is optimized through the at least one feature, and the optimization efficiency of the machine learning model is improved.
Drawings
Fig. 1 is a schematic flow chart of a method for optimizing a machine learning model according to an embodiment of the present application;
FIG. 2 is a schematic structural diagram of an optimizing device of a machine learning model according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
The terms "first," "second," and the like in embodiments of the present application are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Furthermore, the use of "and/or" in this application means at least one of the connected objects, such as a and/or B and/or C, is meant to encompass the 7 cases of a alone, B alone, C alone, and both a and B, both B and C, both a and C, and both A, B and C.
Referring to fig. 1, fig. 1 is a schematic flow chart of an optimization method of a machine learning model according to an embodiment of the present application.
Step 101, obtaining first sample data and second sample data corresponding to a target machine learning model, wherein the target machine learning model is different in prediction result obtained by predicting based on the first sample data and the second sample data, the first sample data comprises x first features, the second sample data comprises y second features, x is a positive integer, and y is a positive integer.
In this embodiment, the method provided in the present application is applied to a machine learning platform, where the machine learning platform may deploy a plurality of machine learning models and a deep learning model, where the deep learning is a subset of the machine learning, the machine learning model includes the deep learning model, a generated AI model or a Large Language Model (LLM) belongs to the deep learning model, and the deep learning and other technologies may be used to generate new content. The target machine learning model may be used, for example, to classify and regress structured data or unstructured data (e.g., images, videos, documents, etc.), and other functions may be implemented by training the target machine learning model, which is not specifically limited in this embodiment. Prediction data of different results may be generated during training of the target machine learning model by the sample data or during prediction of the sample data by a subsequent target machine learning model (i.e., model application), where the prediction result may be an accurate prediction result (i.e., the prediction result is consistent with the actual result) or an inaccurate prediction result (i.e., the prediction result is inconsistent with the actual result) as compared to the actual result corresponding to the first sample data or the second sample data. Specifically, for example, classification of the resulting data may be successful or failed. Wherein the first sample data is bad case data, and the second sample data is data which is predicted correctly or more accurately. The definition of badcase data is a sample that the model or specific rule cannot predict well, and the determination method is a sample that the model predicts incorrectly or predicts less accurately. Note that, in different types of machine learning models, the division of the first sample data and the second sample data is also different, and the specific limitation is not made in this embodiment.
Specifically, after first sample data and second sample data are acquired in a training process of a target machine learning model, x first features are determined according to the first sample data, y second features are determined according to the second sample data, x and y are positive integers, wherein the first features and the second features are dimensions of the sample data, and x first features and y second features can be identical or different. For example, where the sample data is structured data, the first feature and the second feature may be in a data format, data size, data length, etc., with different features affecting the machine learning model to different extents.
Step 102, calculating contribution degrees of each feature to the target machine learning model in the x first features and the y second features respectively to obtain z first contribution degrees, wherein the contribution degrees are used for representing influence degrees of the corresponding feature on an output result of the target machine learning model, z is a sum of the x and the y, and z is a positive integer.
In this embodiment, the contribution degree of each first feature and each second feature to the target machine learning model may be calculated by using a target algorithm, where the contribution degree represents the influence degree of the feature in the training process or the prediction process of the target machine learning model. The target algorithm in this embodiment may be a saprolipram additive model interpretation (Shapley Additive explanations, SHAP) algorithm, where the saprolipram additive model interpretation algorithm is a non-parametric machine learning technique that is used to identify and interpret the extent to which a particular input feature affects the model output. And calculating SHAP values corresponding to each first feature and each second feature, namely z first contribution degrees, through a saprolidine additivity model interpretation algorithm, wherein z is the sum of x and y, and z is a positive integer.
The first contribution degree is a specific value, for example, 70, 90, 95, etc., and the larger the value is, the larger the contribution degree of the feature to the target machine learning model is represented, and the smaller the value is, the smaller the contribution degree of the feature to the target machine learning model is represented.
Step 103, determining at least one first target contribution degree in the z first contribution degrees.
In this embodiment, after z first contribution degrees are obtained, the z first contribution degrees are filtered, where the filtering condition may be that the first contribution degrees greater than the target threshold value are determined as the first target contribution degrees by comparing the target threshold value with the z first contribution degrees one by one. For example, when the first contribution is 70, 90, 95, the target threshold is empirically set to 85, where the contribution of 90 and 95 can be determined to be the first target contribution and the contribution of 70 is discarded. After all the first contribution degrees are compared, at least one first target contribution degree is obtained, and the first target contribution degree indicates that the influence of the first target contribution degree on the target machine learning model is large, and the target machine learning model needs to be considered when being optimized.
And step 104, optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree to obtain an optimized target machine learning model.
In this embodiment, after at least one first target contribution degree is obtained, a corresponding optimization measure may be generated according to the at least one first target contribution degree, parameters or results in the target machine learning model are optimized through the optimization measure, and an optimized target machine learning model is obtained, where the target machine learning model may be used to classify and regress structured data, and the optimized target machine learning model is more accurate for classifying and regressing structured data. The method solves the problem that the prior art does not analyze reasons for poor effects from the angles of prediction deviation and prediction error after model training by combining the contribution degree of the first sample data and the second sample data to the machine learning model, and helps analyze the contribution of the characteristics to the model prediction result through the characteristic importance analysis of the correct prediction sample and the incorrect prediction sample, deduces which characteristics are easy to cause the model prediction error, reflects whether the model is reasonable from the side surface, and provides measures for optimizing the model.
The application provides a method for optimizing a machine learning model, which comprises the following steps: acquiring first sample data and second sample data corresponding to a target machine learning model, wherein the target machine learning model is different in prediction result obtained by predicting the first sample data and the second sample data, the first sample data comprises x first features, the second sample data comprises y second features, x is a positive integer, and y is a positive integer; calculating the contribution degree of each feature to the target machine learning model in the x first features and the y second features respectively to obtain z first contribution degrees, wherein the contribution degrees are used for representing the influence degree of the corresponding feature on the output result of the target machine learning model, z is the sum of the x and the y, and z is a positive integer; determining at least one first target contribution among the z first contributions; and optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree to obtain an optimized target machine learning model. According to the method and the device, the sample data with different prediction results in the training process of the target machine learning model are obtained, the plurality of features are obtained according to the sample data, and the influence degree of different features on the target machine learning model is calculated, so that at least one feature with larger influence on the target machine learning model is determined, the target machine learning model is optimized through the at least one feature, and the optimization efficiency of the machine learning model is improved.
In some possible embodiments, optionally, the acquiring the first sample data and the second sample data of the target machine learning model in the training process includes:
determining a category of the target machine learning model, the category including a machine learning model of a classification task and a machine learning model of a regression task;
based on the category, acquiring first sample data and second sample data of a target machine learning model in a training process, wherein the first sample data is sample data of a prediction error of the target machine learning model, and the second sample data is sample data of a prediction correctness of the target machine learning model when the category characterizes the target machine learning model as the machine learning model of the classification task; and under the condition that the category characterizes the target machine learning model as the machine learning model of the regression task, the first sample data is sample data of which the target machine learning model prediction error is larger than a preset error, and the two sample data are sample data of which the target machine learning model prediction error is equal to or smaller than the preset error.
In the present embodiment, since the kinds of the target machine learning models are different, the division of the first sample data and the second sample data is also different, and thus the kinds of the target machine learning models need to be determined before the target machine learning models are optimized. In this embodiment, a machine learning model in which a target machine learning model is a classification task and a machine learning model in which a regression task are used are described as examples.
When the type recognition result of the target machine learning model characterizes the machine learning model with the classification task, the prediction result of the sample data only has two cases, for example, whether the structured data is in JSON data format is judged, and then the prediction result of the target machine learning model is yes or no. At this time, the first sample data is the sample data of the target machine learning model prediction error, and the second sample data is the sample data of the target machine learning model prediction correct.
In the case of a machine learning model in which the type recognition result of the target machine learning model characterizes a regression task of the target machine learning model, the prediction result of the sample data is a value, for example, the similarity between the structured data and the preset data is judged, and then the prediction result of the target machine learning model may be a percentage or fraction, for example, 50%, 80%, or the like.
By accurately determining the type of the target machine learning model before acquiring the data, the first sample data and the second sample data can be better acquired, so that the accuracy of calculating the contribution value through the characteristics is ensured.
Optionally, the calculating the contribution degree of each feature to the target machine learning model in the x first features and the y second features respectively, to obtain z first contribution degrees includes:
calculating the contribution degree of each of the x first features and the y second features to the target machine learning model respectively to obtain z second contribution degrees, wherein the z second contribution degrees comprise x second contribution degrees corresponding to the x first features and y second contribution degrees corresponding to the y second features;
optimizing x second contribution degrees corresponding to the x first features to obtain x third contribution degrees;
wherein the z first contribution degrees include: and y second contribution degrees corresponding to the y second features and the x third contribution degrees.
In this embodiment, the contribution degree of each of the x first features and the y second features to the target machine learning model may be calculated by a first algorithm, and specifically, the first algorithm is used to calculate the influence degree of the input features to the target machine learning model. The first algorithm may be a saprolily additive model interpretation algorithm, and the contribution degree of each of the x first features and the y second features to the target machine learning model, that is, the SHAP value, is calculated by the saprolily additive model interpretation algorithm, and it should be noted that SHAP is a solution for cooperative game, and the SHAP method generates a predicted value for each sample model. The SHAP value of the first sample data is a value assigned to each first feature in the mispredicted or less accurately predicted sample, and is used to characterize the contribution of the first feature to the reasoning error of the erroneous data sample. The SHAP values of the second sample data predict the data assigned to each second feature in the correct or more accurate sample for characterizing the contribution of the second feature to predicting the reasoning of the correct or more correct sample.
In this embodiment, the x second contribution degrees corresponding to the x first features may be optimized by a second algorithm to obtain x third contribution degrees, and specifically, the second algorithm is used to screen among the multiple input features, and output features meeting preset requirements. The second algorithm may be a local interpretable model agnostic interpretation (Local Interpretable Model-agnostic Explanations, LIME) algorithm, where the x second contribution degrees corresponding to the x first features are optimized by the local interpretable model agnostic interpretation algorithm, to obtain x third contribution degrees. The locally interpretable model agnostic interpretation algorithm is a machine learning technique that aims to give a global view of the interpretative model. It interprets the behavior of the training model by interpretable local interpretation so that the user can understand the results of the model. And optimizing the x contribution degrees corresponding to the x first features through a local interpretable model agnostic interpretation algorithm to obtain x third contribution degrees, so that z first contribution degrees can be formed through y second contribution degrees corresponding to the y second features and the x third contribution degrees.
Optionally, the optimizing the x contribution degrees corresponding to the x first features to obtain x third contribution degrees includes:
Calculating the contribution degree of the x first features to the target machine learning model respectively to obtain x fourth contribution degrees;
and classifying the x fourth contribution degrees to obtain the x third contribution degrees, wherein the third contribution degrees comprise classification identifiers, and the classification identifiers are used for representing whether the corresponding contribution degrees are positive contribution degrees or negative contribution degrees.
In this embodiment, first, x first feature corresponding contribution degrees are calculated by using a saprolily additive model interpretation algorithm, so as to obtain x fourth contribution degrees. Specifically, shapley Value, i.e., marginal contribution, of each feature in each sample of the first sample data is calculated based on the SHAP method. The offset of the predicted value of the first sample data from the predicted value mean is the superposition of the contributions of all the features of the sample. For the predicted value of each sample, the contribution of different features in the predicted value can be understood as the predicted value obtained after the positive or negative contribution of each feature on the basis of the "predicted value mean".
And secondly, classifying the x fourth contribution degrees through a local interpretable model agnostic interpretation algorithm to obtain the x third contribution degrees, wherein the classification comprises a classification mark, so that the third contribution degrees are indicated to be positive contribution degrees (positive action) or negative contribution degrees (negative action). The positive effect is that this feature has a positive effect on the model predictive value, i.e., increases the predictive probability. The negative effect is that the feature has a negative effect on some of the predictors, i.e. would be a decrease in the probability of prediction.
Therefore, the analysis of the single sample by combining the SHAP algorithm and the LIME algorithm can visually display the feature importance in each sample and the positive and negative influence on the prediction result, and the limitation of the dependence of the interpretation of the single model on the model is made up by analyzing the feature contribution degree of the prediction error in the single sample data.
Optionally, the determining at least one first target contribution degree in the z first contribution degrees includes:
sorting the z first contribution degrees in a descending order sorting manner to obtain a sorting set;
screening the sorting set according to a preset threshold value, and determining at least one first target contribution degree, wherein the first target contribution degree is larger than or equal to the preset threshold value.
In this embodiment, the z first contribution degrees are sorted in a descending order sorting manner, so that it can be determined which important features have some value intervals that cause some prediction results to be larger. It should be noted that, the preset threshold may be set according to actual experience, and is not specifically limited in this embodiment. And determining at least one first target contribution degree by screening the first contribution degree with the selection degree larger than or equal to a preset threshold value. The sorting set is screened in a descending order sorting mode, so that the screening rate can be improved, and the optimization efficiency of the model is further improved.
Optionally, the optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree, and obtaining the optimized target machine learning model includes:
updating the at least one target contribution based on at least one target feature corresponding to the at least one target contribution to obtain at least one second target contribution, wherein the at least one first target contribution corresponds to the at least one second target contribution one by one;
screening the at least one second target contribution degree and determining at least one third target contribution degree;
and optimizing the target machine learning model based on at least one feature corresponding to the at least one third target contribution degree to obtain an optimized target machine learning model.
In this embodiment, after determining at least one target feature corresponding to at least one first target contribution, the target contribution may be updated in a statistical manner to obtain at least one second target contribution, for example, if a certain first target contribution is obviously greater than values of other first target contributions, this indicates that the first target contribution may be an incorrect value, so it needs to be updated to meet the requirements. Therefore, after screening and updating all the first target contribution degrees, at least one second target contribution degree is determined.
In addition, at least one third target contribution degree is determined by screening the at least one second target contribution degree, wherein a screening mode can quantitatively and qualitatively analyze prediction results in different value ranges of the features through a statistical method, and the influence of a feature interval on the output result of the model is known through analysis. It should be noted that, by quantitative and qualitative analysis, the influence of data quality, the missing condition of a specific value in data, the abnormal condition of a specific value in data, discrete concentration trend, etc. on the model can be known, so as to avoid the problem that the modeling effect is not friendly due to the conditions of data distortion, etc. And (3) carrying out joint analysis on different target column ranges corresponding to different characteristic value ranges by using a statistical method, and analyzing and knowing the influence of a characteristic interval and a target column on a model output result so as to determine target characteristics with larger influence degree.
Optionally, the optimizing the target machine learning model based on at least one feature corresponding to the at least one third target contribution degree, to obtain an optimized target machine learning model, includes:
generating optimization information based on the at least one third target contribution;
Optimizing the target machine learning model based on the optimization information to obtain an optimized target machine learning model;
wherein the optimization information includes at least one of: adjusting a weight value corresponding to the target feature in the target machine learning model, adding a first sample feature of the target machine learning model, deleting a second sample feature of the target machine learning model, and preprocessing a training sample of the target machine learning model.
In this embodiment, finally, optimization information is generated based on the third target contribution degree, and corresponding optimization measures are adopted to improve the machine learning model according to the optimization information. The optimization measures can be adjusting weight values of features in the model, adding or deleting features, improving data preprocessing and the like. For example: and (3) finding data abnormality through characteristic distribution analysis of partial characteristics which are ranked at the front, and suggesting an improved data preprocessing mode in a feedback optimization stage. SHAP importance analysis of feature importance analysis and badcase data helps to infer which features are prone to model prediction errors, and suggest adding or deleting features before model training in the feedback optimization stage. And carrying out joint analysis on different target column ranges corresponding to different characteristic value ranges to know the influence of the characteristic interval and the target column on the model output result. In the feedback optimization stage, the model is forced to pay attention to the samples with wrong classification by changing the sample weights of the training data of the corresponding characteristic interval, so that the performance of the model is improved.
According to the method and the device, the sample data with different prediction results in the training process of the target machine learning model are obtained, the plurality of features are obtained according to the sample data, and the influence degree of different features on the target machine learning model is calculated, so that at least one feature with larger influence on the target machine learning model is determined, the target machine learning model is optimized through the at least one feature, and the optimization efficiency of the machine learning model is improved.
Referring to fig. 2, fig. 2 is a block diagram of an optimizing apparatus of a machine learning model provided in an embodiment of the present application. As shown in fig. 2, the optimizing apparatus 200 of the machine learning model includes:
an obtaining module 210, configured to obtain first sample data and second sample data corresponding to a target machine learning model, where the target machine learning model is based on that a prediction result obtained by predicting the first sample data and the second sample data is different, the first sample data includes x first features, the second sample data includes y second features, x is a positive integer, and y is a positive integer;
a calculating module 220, configured to calculate a contribution degree of each feature to the target machine learning model from the x first features and the y second features, to obtain z first contribution degrees, where the contribution degrees are used to characterize an influence degree of the corresponding feature on an output result of the target machine learning model, and z is a sum of the x and the y, and z is a positive integer;
A determining module 230, configured to determine at least one first target contribution among the z first contributions;
the optimizing module 240 is configured to optimize the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree, to obtain an optimized target machine learning model.
Optionally, the obtaining module 210 includes:
a classification determination sub-module for determining a category of the target machine learning model, the category including a machine learning model of a classification task and a machine learning model of a regression task;
a classification acquisition sub-module, configured to acquire, based on the category, first sample data and second sample data of a target machine learning model in a training process, where, in a case where the category characterizes that the target machine learning model is a machine learning model of the classification task, the first sample data is sample data of the target machine learning model that predicts incorrectly, and the second sample data is sample data of the target machine learning model that predicts correctly; and under the condition that the category characterizes the target machine learning model as the machine learning model of the regression task, the first sample data is sample data of which the target machine learning model prediction error is larger than a preset error, and the two sample data are sample data of which the target machine learning model prediction error is equal to or smaller than the preset error.
Optionally, the computing module 220 includes:
the first calculation sub-module is used for calculating the contribution degree of each of the x first features and the y second features to the target machine learning model respectively to obtain z second contribution degrees, wherein the z second contribution degrees comprise x second contribution degrees corresponding to the x first features and y second contribution degrees corresponding to the y second features;
the second calculation sub-module is used for optimizing x second contribution degrees corresponding to the x first features to obtain x third contribution degrees;
wherein the z first contribution degrees include: and y second contribution degrees corresponding to the y second features and the x third contribution degrees.
Optionally, the second computing submodule includes:
the computing unit is used for respectively computing the contribution degrees of the x first features to the target machine learning model to obtain x fourth contribution degrees;
the classification unit is configured to classify the x fourth contribution degrees to obtain the x third contribution degrees, where the third contribution degrees include a classification identifier, and the classification identifier is used to characterize whether the corresponding contribution degree is a positive contribution degree or a negative contribution degree.
Optionally, the determining module 230 includes:
the sorting sub-module is used for sorting the z first contribution degrees in a descending order sorting mode to obtain a sorting set;
the determining submodule is used for screening the sorting set according to a preset threshold value, and determining at least one first target contribution degree, wherein the first target contribution degree is larger than or equal to the preset threshold value.
Optionally, the optimization module 240 includes:
the updating sub-module is used for updating the at least one target contribution degree based on at least one target feature corresponding to the at least one target contribution degree to obtain at least one second target contribution degree, and the at least one first target contribution degree corresponds to the at least one second target contribution degree one by one;
a screening sub-module, configured to screen the at least one second target contribution, and determine at least one third target contribution;
and the optimization sub-module is used for optimizing the target machine learning model based on at least one feature corresponding to the at least one third target contribution degree to obtain an optimized target machine learning model.
Optionally, the optimizing submodule includes:
a generating unit for generating optimization information based on the at least one third target contribution;
The optimizing unit is used for optimizing the target machine learning model based on the optimizing information to obtain an optimized target machine learning model;
wherein the optimization information includes at least one of: adjusting a weight value corresponding to the target feature in the target machine learning model, adding a first sample feature of the target machine learning model, deleting a second sample feature of the target machine learning model, and preprocessing a training sample of the target machine learning model.
According to the method and the device, the sample data with different prediction results in the training process of the target machine learning model are obtained, the plurality of features are obtained according to the sample data, and the influence degree of different features on the target machine learning model is calculated, so that at least one feature with larger influence on the target machine learning model is determined, the target machine learning model is optimized through the at least one feature, and the optimization efficiency of the machine learning model is improved.
The embodiment of the application also provides electronic equipment. Referring to fig. 3, an electronic device may include a processor 301, a memory 302, and a program 3021 stored on the memory 302 and executable on the processor 301.
Program 3021, when executed by processor 301, may implement any of the steps in the method embodiment corresponding to fig. 1:
acquiring first sample data and second sample data corresponding to a target machine learning model, wherein the target machine learning model is different in prediction result obtained by predicting the first sample data and the second sample data, the first sample data comprises x first features, the second sample data comprises y second features, x is a positive integer, and y is a positive integer;
calculating the contribution degree of each feature to the target machine learning model in the x first features and the y second features respectively to obtain z first contribution degrees, wherein the contribution degrees are used for representing the influence degree of the corresponding feature on the output result of the target machine learning model, z is the sum of the x and the y, and z is a positive integer;
determining at least one first target contribution among the z first contributions;
and optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree to obtain an optimized target machine learning model.
Optionally, the acquiring the first sample data and the second sample data of the target machine learning model in the training process includes:
Determining a category of the target machine learning model, the category including a machine learning model of a classification task and a machine learning model of a regression task;
based on the category, acquiring first sample data and second sample data of a target machine learning model in a training process, wherein the first sample data is sample data of a prediction error of the target machine learning model, and the second sample data is sample data of a prediction correctness of the target machine learning model when the category characterizes the target machine learning model as the machine learning model of the classification task; and under the condition that the category characterizes the target machine learning model as the machine learning model of the regression task, the first sample data is sample data of which the target machine learning model prediction error is larger than a preset error, and the two sample data are sample data of which the target machine learning model prediction error is equal to or smaller than the preset error.
Optionally, the calculating the contribution degree of each feature to the target machine learning model in the x first features and the y second features respectively, to obtain z first contribution degrees includes:
Calculating the contribution degree of each of the x first features and the y second features to the target machine learning model respectively to obtain z second contribution degrees, wherein the z second contribution degrees comprise x second contribution degrees corresponding to the x first features and y second contribution degrees corresponding to the y second features;
optimizing x contribution degrees corresponding to the x first features to obtain x third contribution degrees;
wherein the z first contribution degrees include: and y second contribution degrees corresponding to the y second features and the x third contribution degrees.
Optionally, the optimizing the x contribution degrees corresponding to the x first features to obtain x third contribution degrees includes:
calculating the contribution degree of the x first features to the target machine learning model respectively to obtain x fourth contribution degrees;
and classifying the x fourth contribution degrees to obtain the x third contribution degrees, wherein the third contribution degrees comprise classification identifiers, and the classification identifiers are used for representing whether the corresponding contribution degrees are positive contribution degrees or negative contribution degrees.
Optionally, the determining at least one first target contribution degree in the z first contribution degrees includes:
Sorting the z first contribution degrees in a descending order sorting manner to obtain a sorting set;
screening the sorting set according to a preset threshold value, and determining at least one first target contribution degree, wherein the first target contribution degree is larger than or equal to the preset threshold value.
Optionally, the optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree, and obtaining the optimized target machine learning model includes:
updating the at least one target contribution based on at least one target feature corresponding to the at least one target contribution to obtain at least one second target contribution, wherein the at least one first target contribution corresponds to the at least one second target contribution one by one;
screening the at least one second target contribution degree and determining at least one third target contribution degree;
and optimizing the target machine learning model based on at least one feature corresponding to the at least one third target contribution degree to obtain an optimized target machine learning model.
Optionally, the optimizing the target machine learning model based on at least one feature corresponding to the at least one third target contribution degree, to obtain an optimized target machine learning model, includes:
Generating optimization information based on the at least one third target contribution;
optimizing the target machine learning model based on the optimization information to obtain an optimized target machine learning model;
wherein the optimization information includes at least one of: adjusting a weight value corresponding to the target feature in the target machine learning model, adding a first sample feature of the target machine learning model, deleting a second sample feature of the target machine learning model, and preprocessing a training sample of the target machine learning model.
According to the method and the device, the sample data with different prediction results in the training process of the target machine learning model are obtained, the plurality of features are obtained according to the sample data, and the influence degree of different features on the target machine learning model is calculated, so that at least one feature with larger influence on the target machine learning model is determined, the target machine learning model is optimized through the at least one feature, and the optimization efficiency of the machine learning model is improved.
The embodiment of the present application further provides a computer readable storage medium, on which a computer program is stored, where the computer program when executed by a processor implements each process of the above-mentioned machine learning model optimization method embodiment, and the same technical effects can be achieved, so that repetition is avoided, and no further description is given here. Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), including several instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method described in the embodiments of the present application.
The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those of ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are also within the protection of the present application.
Claims (10)
1. A method of optimizing a machine learning model, the method comprising:
acquiring first sample data and second sample data corresponding to a target machine learning model, wherein the target machine learning model is different in prediction result obtained by predicting the first sample data and the second sample data, the first sample data comprises x first features, the second sample data comprises y second features, x is a positive integer, and y is a positive integer;
calculating the contribution degree of each feature to the target machine learning model in the x first features and the y second features respectively to obtain z first contribution degrees, wherein the contribution degrees are used for representing the influence degree of the corresponding feature on the output result of the target machine learning model, z is the sum of the x and the y, and z is a positive integer;
Determining at least one first target contribution among the z first contributions;
and optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree to obtain an optimized target machine learning model.
2. The method of claim 1, wherein obtaining first sample data and second sample data of the target machine learning model during training comprises:
determining a category of the target machine learning model, the category including a machine learning model of a classification task and a machine learning model of a regression task;
based on the category, acquiring first sample data and second sample data of a target machine learning model in a training process, wherein the first sample data is sample data of a prediction error of the target machine learning model, and the second sample data is sample data of a prediction correctness of the target machine learning model when the category characterizes the target machine learning model as the machine learning model of the classification task; and under the condition that the category characterizes the target machine learning model as the machine learning model of the regression task, the first sample data is sample data of which the target machine learning model prediction error is larger than a preset error, and the two sample data are sample data of which the target machine learning model prediction error is equal to or smaller than the preset error.
3. The method of claim 1, wherein the calculating the contribution of each of the x first features and the y second features to the target machine learning model, respectively, results in z first contributions, comprises:
calculating the contribution degree of each of the x first features and the y second features to the target machine learning model respectively to obtain z second contribution degrees, wherein the z second contribution degrees comprise x second contribution degrees corresponding to the x first features and y second contribution degrees corresponding to the y second features;
optimizing x second contribution degrees corresponding to the x first features to obtain x third contribution degrees;
wherein the z first contribution degrees include: and y second contribution degrees corresponding to the y second features and the x third contribution degrees.
4. The method of claim 3, wherein optimizing the x contribution degrees corresponding to the x first features to obtain x third contribution degrees includes:
calculating the contribution degree of the x first features to the target machine learning model respectively to obtain x fourth contribution degrees;
And classifying the x fourth contribution degrees to obtain the x third contribution degrees, wherein the third contribution degrees comprise classification identifiers, and the classification identifiers are used for representing whether the corresponding contribution degrees are positive contribution degrees or negative contribution degrees.
5. The method of any one of claims 1 to 4, wherein said determining at least one first target contribution among said z first contributions comprises:
sorting the z first contribution degrees in a descending order sorting manner to obtain a sorting set;
screening the sorting set according to a preset threshold value, and determining at least one first target contribution degree, wherein the first target contribution degree is larger than or equal to the preset threshold value.
6. The method according to claim 1 or 5, wherein optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree, the obtaining an optimized target machine learning model includes:
updating the at least one target contribution based on at least one target feature corresponding to the at least one target contribution to obtain at least one second target contribution, wherein the at least one first target contribution corresponds to the at least one second target contribution one by one;
Screening the at least one second target contribution degree and determining at least one third target contribution degree;
and optimizing the target machine learning model based on at least one feature corresponding to the at least one third target contribution degree to obtain an optimized target machine learning model.
7. The method of claim 6, wherein optimizing the target machine learning model based on at least one feature corresponding to the at least one third target contribution, resulting in an optimized target machine learning model, comprises:
generating optimization information based on the at least one third target contribution;
optimizing the target machine learning model based on the optimization information to obtain an optimized target machine learning model;
wherein the optimization information includes at least one of: adjusting a weight value corresponding to the target feature in the target machine learning model, adding a first sample feature of the target machine learning model, deleting a second sample feature of the target machine learning model, and preprocessing a training sample of the target machine learning model.
8. An apparatus for optimizing a machine learning model, the apparatus comprising:
The device comprises an acquisition module, a prediction module and a prediction module, wherein the acquisition module is used for acquiring first sample data and second sample data corresponding to a target machine learning model, the target machine learning model is different in prediction result obtained by predicting the first sample data and the second sample data, the first sample data comprises x first features, the second sample data comprises y second features, x is a positive integer, and y is a positive integer;
the computing module is used for respectively computing the contribution degree of each feature to the target machine learning model in the x first features and the y second features to obtain z first contribution degrees, wherein the contribution degrees are used for representing the influence degree of the corresponding feature to the output result of the target machine learning model, z is the sum of the x and the y, and z is a positive integer;
a determining module for determining at least one first target contribution among the z first contributions;
and the optimization module is used for optimizing the target machine learning model based on at least one feature corresponding to the at least one first target contribution degree to obtain an optimized target machine learning model.
9. An electronic device, comprising: a memory, a processor, and a program stored on the memory and executable on the processor; -characterized in that the processor is arranged to read the program in the memory for implementing the steps in the method for optimizing the machine learning model according to any one of claims 1 to 7.
10. A readable storage medium storing a program, wherein the program, when executed by a processor, implements the steps in the method of optimizing a machine learning model according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311782927.0A CN117540826A (en) | 2023-12-22 | 2023-12-22 | Optimization method and device of machine learning model, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311782927.0A CN117540826A (en) | 2023-12-22 | 2023-12-22 | Optimization method and device of machine learning model, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117540826A true CN117540826A (en) | 2024-02-09 |
Family
ID=89796088
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311782927.0A Pending CN117540826A (en) | 2023-12-22 | 2023-12-22 | Optimization method and device of machine learning model, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117540826A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117877647A (en) * | 2024-03-13 | 2024-04-12 | 苏州创腾软件有限公司 | Recipe generation method and device based on machine learning |
-
2023
- 2023-12-22 CN CN202311782927.0A patent/CN117540826A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117877647A (en) * | 2024-03-13 | 2024-04-12 | 苏州创腾软件有限公司 | Recipe generation method and device based on machine learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9129228B1 (en) | Robust and fast model fitting by adaptive sampling | |
| US20150120263A1 (en) | Computer-Implemented Systems and Methods for Testing Large Scale Automatic Forecast Combinations | |
| CN107168995B (en) | Data processing method and server | |
| CN110991474A (en) | Machine learning modeling platform | |
| CN117540826A (en) | Optimization method and device of machine learning model, electronic equipment and storage medium | |
| CN111797320B (en) | Data processing method, device, equipment and storage medium | |
| US20220092359A1 (en) | Image data classification method, device and system | |
| CN111046930A (en) | Power supply service satisfaction influence factor identification method based on decision tree algorithm | |
| CN114202256B (en) | Architecture upgrading early warning method and device, intelligent terminal and readable storage medium | |
| CN111860698A (en) | Method and device for determining stability of learning model | |
| CN116743637B (en) | Abnormal flow detection method and device, electronic equipment and storage medium | |
| US20210357699A1 (en) | Data quality assessment for data analytics | |
| CN113283673A (en) | Model performance attenuation evaluation method, model training method and device | |
| CN113761193A (en) | Log classification method and device, computer equipment and storage medium | |
| CN115114124A (en) | Host risk assessment method and device | |
| WO2011149608A1 (en) | Identifying and using critical fields in quality management | |
| WO2023179042A1 (en) | Data updating method, fault diagnosis method, electronic device, and storage medium | |
| CN111783883A (en) | Abnormal data detection method and device | |
| CN109460474B (en) | User preference trend mining method | |
| CN114139636B (en) | Abnormal operation processing method and device | |
| CN115994093A (en) | Test case recommendation method and device | |
| CN113570070B (en) | Streaming data sampling and model updating method, device, system and storage medium | |
| JPH09233700A (en) | Method of evaluating reliability on estimation of day maximum demand power | |
| CN114610590A (en) | Method, device and equipment for determining operation time length and storage medium | |
| CN113641823A (en) | Text classification model training method, text classification device, text classification equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |