CN109978179A - Model training method and device, electronic equipment and readable storage medium - Google Patents

Abstract

The embodiment of the disclosure discloses a model training method, a model training device, electronic equipment and a readable storage medium. According to the technical scheme, the base model used in the combined model and the corresponding combination coefficient of the used base model can be automatically determined, the parameter adjusting efficiency in the model training process can be improved, and the accuracy and the objectivity of the model are improved.

Description

Model training method, device, electronic device and readable storage medium

technical field

The present disclosure relates to the field of computer technologies, and in particular, to a model training method, apparatus, electronic device, and readable storage medium.

Background technique

In order to improve the prediction accuracy of the model in machine learning, technicians usually combine multiple base models to improve the generalization ability of the model.

During the process of proposing the present disclosure, the inventor found that the model combination in the prior art often requires technicians to train multiple base models separately, then select and combine the trained multiple base models, and then combine the models. Performing training to adjust the parameters of the model combination makes the existing model training time-consuming and labor-intensive, which seriously affects the efficiency of model training.

SUMMARY OF THE INVENTION

In order to solve the problems in the related art, the embodiments of the present disclosure provide a model training method, an apparatus, an electronic device, and a readable storage medium.

In a first aspect, an embodiment of the present disclosure provides a model training method.

Specifically, the model training method includes:

obtaining first training data and second training data;

Train multiple base models based on the first training data, and determine model parameters of each base model;

Based on the second training data, the base model used in the combined model and the corresponding combining coefficients of the used base model are determined by a greedy algorithm.

In conjunction with the first aspect, in a first implementation manner of the first aspect of the present disclosure, the plurality of base models include at least one linear model and/or at least one nonlinear model.

In conjunction with the first implementation manner of the first aspect, in a second implementation manner of the first aspect of the present disclosure, the linear model includes a logistic regression model; and/or

The nonlinear model includes at least one of an extreme gradient boosting model, a factorization machine, and a random forest.

With reference to the first implementation manner of the first aspect, in a third implementation manner of the first aspect of the present disclosure, the training of multiple base models based on the first training data includes:

using a gradient boosting tree model to process the first training data to obtain intermediate training data;

Remove the low correlation features in the intermediate training data to obtain the third training data;

The plurality of base models are trained based on the third training data.

With reference to the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect of the present disclosure, the training of multiple base models based on the first training data includes training the multiple base models based on the first training data. a nonlinear model in a base model; and/or

The training of the plurality of base models based on the third training data includes training a linear model of the plurality of base models based on the third training data.

With reference to the first aspect, in a fifth implementation manner of the first aspect of the present disclosure, the base model used in the combined model and the corresponding coefficients of the used base model are determined by a greedy algorithm based on the second training data, including :

determining, based on the second training data, a first model with the best performance among the multiple base models, and using the first model as a combined model;

Gradually increase the number of base models in the combined model until adding a new base model no longer improves the performance of the model combination, or the number of base models in the combined model is equal to the total number of the multiple base models, wherein each time The base model added to the combined model is a base model with the best performance of the combined model after adding the base model, and the combination coefficient of each base model in the combined model after adding the base model is determined;

The base model used in the combined model and the corresponding combination coefficients of the used base model are output.

In conjunction with the first aspect, in a sixth implementation manner of the present disclosure, the model training method further includes:

Remove low-correlation features in the original data to obtain preprocessed data;

The first training data, the second training data and the test data are obtained by randomly dividing or dividing the preprocessed data by time.

In conjunction with the sixth implementation manner of the first aspect, in the seventh implementation manner of the first aspect, the present disclosure further includes:

The combined model is validated based on the test data.

With reference to the first aspect, in an eighth implementation manner of the first aspect, the first training data and the second training data include data related to user portraits;

The combined model and the base model are used to make predictions based on data related to user profiles.

In a second aspect, an embodiment of the present disclosure provides a model training device, characterized in that it includes:

an acquisition module, configured to acquire the first training data and the second training data;

a first determination module, configured to train a plurality of base models based on the first training data, and to determine model parameters of each base model;

The second determination module is configured to determine, based on the second training data, a base model used in the combined model and a corresponding combination coefficient of the used base model through a greedy algorithm.

In conjunction with the second aspect, in a first implementation manner of the second aspect, the plurality of base models include at least one linear model and/or at least one nonlinear model.

In conjunction with the first implementation of the second aspect, in a second implementation of the second aspect, the linear model includes a logistic regression model; and/or

The nonlinear model includes at least one of an extreme gradient boosting model, a factorization machine, and a random forest.

With reference to the first implementation manner of the second aspect, in a third implementation manner of the second aspect of the present disclosure, the training of multiple base models based on the first training data includes:

using a gradient boosting tree model to process the first training data to obtain intermediate training data;

Remove the low correlation features in the intermediate training data to obtain the third training data;

The plurality of base models are trained based on the third training data.

With reference to the third implementation manner of the second aspect, in a fourth implementation manner of the second aspect of the present disclosure, the training of multiple base models based on the first training data includes training the multiple base models based on the first training data. a nonlinear model in a base model; and/or

The training of the plurality of base models based on the third training data includes training a linear model of the plurality of base models based on the third training data.

With reference to the fourth implementation manner of the second aspect, in a fifth implementation manner of the second aspect, the base model used in the combined model and the used base model are determined by a greedy algorithm based on the second training data. The corresponding coefficients of the model, including:

determining, based on the second training data, a first model with the best performance among the multiple base models, and using the first model as a combined model;

Gradually increase the number of base models in the combined model until adding a new base model no longer improves the performance of the model combination, or the number of base models in the combined model is equal to the total number of the multiple base models, wherein each time The base model added to the combined model is a base model with the best performance of the combined model after adding the base model, and the combination coefficient of each base model in the combined model after adding the base model is determined;

The base model used in the combined model and the corresponding combination coefficients of the used base model are output.

With reference to the second aspect, in a sixth implementation manner of the second aspect of the present disclosure, the model training further includes: a removal module configured to remove low-correlation features in the original data to obtain preprocessed data;

The segmentation module is configured to randomly segment or segment the preprocessed data to obtain the first training data, the second training data and the test data.

With reference to the sixth implementation manner of the second aspect, in the seventh implementation manner of the second aspect of the present disclosure, the model training further includes:

A verification module is configured to verify the combined model based on the test data.

In conjunction with the second aspect, in an eighth implementation manner of the second aspect, the first training data and the second training data include data related to user portraits;

The combined model and the base model are used to make predictions based on data related to user profiles.

In a third aspect, embodiments of the present disclosure provide an electronic device, including a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are processed by the The controller executes to implement the following method steps:

obtaining first training data and second training data;

Train multiple base models based on the first training data, and determine model parameters of each base model;

Based on the second training data, the base model used in the combined model and the corresponding combining coefficients of the used base model are determined by a greedy algorithm.

In conjunction with the third aspect, in a first implementation manner of the third aspect, the plurality of base models include at least one linear model and/or at least one nonlinear model.

In conjunction with the first implementation manner of the third aspect, in a second implementation manner of the third aspect, the linear model includes a logistic regression model; and/or

The nonlinear model includes at least one of an extreme gradient boosting model, a factorization machine, and a random forest.

With reference to the first implementation manner of the third aspect, in a third implementation manner of the third aspect of the present disclosure, a gradient boosting tree model is used to process the first training data to obtain intermediate training data;

Remove the low correlation features in the intermediate training data to obtain the third training data;

The plurality of base models are trained based on the third training data.

With reference to the third implementation manner of the third aspect, in a fourth implementation manner of the third aspect of the present disclosure, the training of multiple base models based on the first training data includes training the multiple base models based on the first training data. a nonlinear model in a base model; and/or

The training of the plurality of base models based on the third training data includes training a linear model of the plurality of base models based on the third training data.

With reference to the third aspect, in a fifth implementation manner of the third aspect of the present disclosure, the base model used in the combined model and the corresponding coefficients of the used base model are determined by a greedy algorithm based on the second training data, including :

determining, based on the second training data, a first model with the best performance among the multiple base models, and using the first model as a combined model;

Gradually increase the number of base models in the combined model until adding a new base model no longer improves the performance of the model combination, or the number of base models in the combined model is equal to the total number of the multiple base models, wherein each time The base model added to the combined model is a base model with the best performance of the combined model after adding the base model, and the combination coefficient of each base model in the combined model after adding the base model is determined;

The base model used in the combined model and the corresponding combination coefficients of the used base model are output.

In conjunction with the third aspect, in a sixth implementation manner of the third aspect of the present disclosure, the one or more computer instructions are further executed by the processor to implement the following method steps: removing low-correlation features in the original data to obtain preprocessing data;

The first training data, the second training data and the test data are obtained by randomly dividing or dividing the preprocessed data by time.

With reference to the sixth implementation manner of the third aspect, in a seventh implementation manner of the third aspect of the present disclosure, the one or more computer instructions are further executed by the processor to implement the following method steps:

The combined model is validated based on the test data.

With reference to the third aspect, in an eighth implementation manner of the third aspect, the first training data and the second training data include data related to user portraits;

The combined model and the base model are used to make predictions based on data related to user profiles.

In a fourth aspect, an embodiment of the present disclosure provides a readable storage medium on which computer instructions are stored, and when the computer instructions are executed by a processor, implement the first aspect, the first implementation manner of the first aspect to the fourth aspect. The method described in any one of the eight implementation manners.

The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

According to the technical solutions provided by the embodiments of the present disclosure, the base model used in the combined model and the corresponding combination coefficient of the used base model can be automatically determined, the efficiency of parameter adjustment in the model training process can be improved, and the accuracy and objectivity of the model can be improved .

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

Other labels, objects and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the attached image:

1 shows a flowchart of a model training method according to an embodiment of the present disclosure;

2 shows a flowchart of a model training method according to an embodiment of the present disclosure;

3 shows a flowchart of a model training method according to an embodiment of the present disclosure;

4 illustrates a flowchart of training multiple base models according to an embodiment of the present disclosure;

5 shows a schematic diagram of a gradient boosted tree model according to an embodiment of the present disclosure;

6 shows a flowchart of determining a base model used in a combined model and corresponding coefficients of the used base model according to an embodiment of the present disclosure;

7 illustrates an exemplary process of a model training method according to an embodiment of the present disclosure;

8 shows a structural block diagram of a model training apparatus according to an embodiment of the present disclosure;

9 shows a structural block diagram of an electronic device according to an embodiment of the present disclosure;

FIG. 10 shows a schematic structural diagram of a computer system suitable for implementing the model training method according to an embodiment of the present disclosure.

Detailed ways

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Also, for the sake of clarity, parts unrelated to describing the exemplary embodiments are omitted from the drawings.

In the present disclosure, it should be understood that terms such as "comprising" or "having" are intended to indicate the presence of features, numbers, steps, acts, components, parts, or combinations thereof disclosed in this specification, and are not intended to exclude a or multiple other features, numbers, steps, acts, components, parts, or combinations thereof may exist or be added.

In addition, it should be noted that the embodiments of the present disclosure and the features of the embodiments may be combined with each other under the condition of no conflict. The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

As mentioned above, in order to improve the prediction accuracy of the model in machine learning, technicians usually combine multiple base models to improve the generalization ability of the model. During the process of proposing the present disclosure, the inventor found that the model combination in the prior art often requires technicians to train multiple base models separately, then select and combine the trained multiple base models, and then combine the models. Performing training to adjust the parameters of the model combination makes the existing model training time-consuming and labor-intensive, which seriously affects the efficiency of model training.

Considering the above defects, the technical solutions provided by the embodiments of the present disclosure obtain first training data and second training data, train a plurality of base models based on the first training data, determine model parameters of each base model, and determine the model parameters of each base model based on the second training data. Training data, the base model used in the combined model and the corresponding combination coefficients of the used base model are determined by a greedy algorithm. The technical solution can automatically determine the base model used in the combined model and the corresponding combination coefficient of the used base model, which can improve the efficiency of parameter adjustment in the model training process, reduce manual intervention on the model, and improve the generalization ability of the model.

FIG. 1 shows a flowchart of a model training method according to an embodiment of the present disclosure.

As shown in FIG. 1 , the model training method includes the following steps S101-S103.

In step S101, first training data and second training data are acquired.

In step S102, a plurality of base models are trained based on the first training data, and model parameters of each base model are determined.

In step S103, based on the second training data, a base model used in the combined model and a corresponding combination coefficient of the used base model are determined by a greedy algorithm.

According to an embodiment of the present disclosure, for example, assuming that there are multiple base models M1, M2, . The greedy algorithm determines the base model used by the combined model M and the corresponding combining coefficients of the used base model. According to actual needs and the effect of the model, the combined model M may include all or part of the multiple base models M1, M2, . . . , Mn. The combination coefficient of the base model in the combination model can represent the weight of the base model in the combination model. For example, assuming that the combination model M includes base models M1, M2, and M3, and the combination coefficients of the base models M1, M2, and M3 are m1, m2, and m3, respectively, then M=m1*M1+m2*M2+m3*M3.

According to the embodiments of the present disclosure, the base model used in the combined model and the corresponding combination coefficient of the used base model can be automatically determined, which can improve the efficiency of parameter adjustment in the model training process and improve the generalization ability of the model.

The model training method proposed in the present disclosure is widely applicable to the training of various model combinations, the first training data and the second training data may include various data, and the combined model and the base model may be used for Various uses, which are not specifically limited in the present disclosure.

For example, the first training data and the second training data may include data related to user portraits. The combined model and the base model are used to make predictions based on the user profile-related data. For example, the first training data and the second training data may include at least one of the following data related to user portraits: user attribute data (such as age, gender, occupation, etc.), user behavior data (such as consumption preferences, browsing preferences, etc.), The interaction data between the user and other subjects (such as placing an order, saving, returning, etc.), the combined model and the base model can be used to predict the user's behavior based on the data related to the user's portrait, such as the user's click on the recommended item in the recommendation list. probability.

FIG. 2 shows a flowchart of a model training method according to an embodiment of the present disclosure.

As shown in FIG. 2, according to an embodiment of the present disclosure, the model training method further includes steps S104-S105 in addition to steps S101-S103.

In step S104, low correlation features in the original data are removed to obtain preprocessed data.

In step S105, the first training data, the second training data and the test data are obtained by randomly dividing or dividing the preprocessed data by time.

According to the embodiment of the present disclosure, the low correlation feature may be a feature that has nothing to do with the current model training (or a very small relationship), or it may be a redundant feature that can be obtained from other features. For example, if the length and width of a rectangle are known, then The area of the rectangle can be regarded as a redundant feature.

According to an embodiment of the present disclosure, the removing low-correlation features in the original data includes removing low-correlation features in the original data through feature testing methods such as variance testing, correlation testing, etc., for example, removing features with zero variance, removing correlation Features that are below the threshold, etc. In this way, the dimensionality explosion can be effectively avoided. On the one hand, the memory occupation of the data is reduced, and the efficiency of model training is improved. On the other hand, it can also effectively avoid the situation of overfitting caused by too sparse features during the model training process.

FIG. 3 shows a flowchart of a model training method according to an embodiment of the present disclosure.

As shown in FIG. 3, according to an embodiment of the present disclosure, the model training method further includes step S106 in addition to steps S101-S103.

In step S106, the combined model is verified based on the test data.

According to an embodiment of the present disclosure, the test data is different from the first training data and the second training data, that is, the test data has not been used in steps S101-S103. In this way, the fidelity of the verification results can be improved, and the risk of overfitting can be effectively reduced, thereby improving the accuracy of the model.

FIG. 4 shows a flowchart of training multiple base models according to an embodiment of the present disclosure.

As shown in FIG. 4, training a plurality of base models based on the first training data includes steps S201-S203.

In step S201, a gradient boosting tree model is used to process the first training data to obtain intermediate training data.

In step S202, low correlation features in the intermediate training data are removed to obtain third training data.

In step S203, the plurality of base models are trained based on the third training data.

According to an embodiment of the present disclosure, the gradient boosting tree model (Gradient BoostingDecision TreeModel, GBDT model for short) includes multiple regression trees, and is an iterative decision tree model.

According to an embodiment of the present disclosure, the gradient boosting tree model is used to process the first training data to obtain intermediate training data. The gradient boosting tree may be used to process the first training data, and then one-hot encoding (one-hot encoding) is used. The leaf node of each regression tree is recorded, that is, the leaf node where the first training data is located is marked as 1, and the rest are marked as 0, and then all codes are combined to obtain intermediate training data.

An exemplary process of using a gradient boosting tree model to process the first training data to obtain intermediate training data according to an embodiment of the present disclosure will be specifically described below with reference to FIG. 5 .

5 shows a schematic diagram of a gradient boosted tree model according to an embodiment of the present disclosure.

For the convenience of description, in FIG. 5, the gradient boosting tree model T includes three regression trees as an example for explanation and description. It should be understood that this example is only used as an example, and is not a limitation of the present disclosure. A gradient boosted tree model can also consist of two or more regression trees. In addition, the depth and the number of leaf nodes of each regression tree can also be set according to actual needs, which are not specifically limited in the present disclosure.

As shown in FIG. 5 , the gradient boosting tree model T includes three regression trees T1 , T2 and T3 , wherein the regression trees T1 and T3 each have three leaf nodes, and the regression tree T2 has two leaf nodes. Each leaf node corresponds to a feature, and after the first training data S1 is processed by the gradient boosting tree model T, the obtained intermediate training data X1 is an eight-dimensional feature vector.

Assuming that after the gradient boosting tree model T is processed, the first training data S1 falls on the first leaf node of the regression tree T1, then the code a1 can be obtained as [1,0,0], indicating that the first training data S1 falls on the first leaf node of the regression tree T1. The training data S1 includes the feature corresponding to the first leaf node of the regression tree T1; the first training data S1 falls on the second leaf node of the regression tree T2, then the code a2 can be obtained as [0,1]; the If the first training data S1 falls on the third leaf node of the regression tree T3, the code a3 can be obtained as [0, 0, 1]. By combining the encoding a1, the encoding a2 and the encoding a3, the intermediate training data X1 is obtained as an eight-dimensional vector [1,0,0,0,1,0,0,1].

In addition, in the process of processing the first training data using the gradient boosting tree, the parameters of the gradient boosting tree model may be adjusted according to the first training data, thereby improving the performance of the gradient boosting tree model.

According to an embodiment of the present disclosure, the low correlation features in the intermediate training data are removed to obtain third training data. Specifically, low-correlation features in the intermediate training data can be removed through feature testing such as variance testing and correlation testing, for example, features with zero variance and features with a correlation lower than a threshold can be removed.

According to an embodiment of the present disclosure, since the dimension of the intermediate training data is determined by the number of features, the third training data removes low-correlation features compared to the intermediate training data. Therefore, the dimension of the third training data is lower than the dimension of the intermediate training data, thereby effectively avoiding dimensionality explosion. On the one hand, the memory occupation of data can be reduced, and the efficiency of model training can be improved. On the other hand, due to GBDT The intermediate data obtained after processing is a combination of one-hot encoding. The one-hot encoding itself has relatively sparse features, and the combined features are even more sparse. Therefore, by removing low-correlation features from the intermediate data, a third training program with a lower dimension is obtained. It can also effectively avoid overfitting due to too sparse features during model training.

According to an embodiment of the present disclosure, the plurality of base models includes at least one nonlinear model. According to an embodiment of the present disclosure, the nonlinear model includes at least one of an extreme gradient boosting model, a factorization machine, and a random forest.

According to an embodiment of the present disclosure, the training of a plurality of base models based on the first training data includes training a nonlinear model among the plurality of base models based on the first training data.

According to an embodiment of the present disclosure, the first training data can be divided into a first training set and a first test set, and for each base model (eg, a nonlinear model), based on the first training set, combined with greedy Search and grid search methods are used to find multiple sets of parameters, and then based on the first test set, the model parameters of the base model are determined by a cross-validation method.

According to the embodiments of the present disclosure, determining the model parameters of the base model through the cross-validation method can avoid overfitting to a certain extent, and is also beneficial to obtain as much effective information as possible from limited preprocessing data, thereby improving The generalization ability of the base model.

According to the embodiments of the present disclosure, the model parameters of the respective base models are automatically determined through algorithms such as greedy search and grid search combined with the cross-validation method, which can effectively reduce manual intervention, improve the efficiency of model training, and improve the accuracy and performance of the model. objectivity.

According to an embodiment of the present disclosure, the plurality of base models includes at least one linear model. According to an embodiment of the present disclosure, the linear model includes a logistic regression model.

According to an embodiment of the present disclosure, the training of the plurality of base models based on the third training data includes training a linear model of the plurality of base models based on the third training data.

According to an embodiment of the present disclosure, the third training data can be divided into a third training set and a third testing set, and for each base model (eg, a linear model), based on the third training set, the grid A search method is used to find multiple sets of parameters, and based on the third test set, the model parameters of the base model are determined by a cross-validation method.

According to the embodiment of the present disclosure, multiple base models may include multiple base models of different types, which is beneficial to make the prediction errors of the multiple base models independent of each other, thereby reducing the error rate after combining the multiple base models, thereby improving the model Accuracy and reliability of training.

FIG. 6 shows a flowchart of determining a base model used in a combined model and corresponding coefficients of the used base model, according to an embodiment of the present disclosure.

As shown in FIG. 6 , based on the second training data, the base model used in the combined model and the corresponding coefficients of the used base model are determined by a greedy algorithm, including the following steps S301-S303.

In step S301, based on the second training data, a first model with the best performance among the multiple base models is determined, and the first model is used as a combined model.

In step S302, the number of base models in the combined model is gradually increased until adding a new base model no longer improves the performance of the model combination, or the number of base models in the combined model is equal to the total number of the multiple base models , wherein the base model added to the combined model each time is the base model that optimizes the performance of the combined model after adding the base model, and the combined coefficients of each base model in the combined model after adding the base model are determined.

In step S303, the base model used in the combined model and the corresponding combination coefficients of the used base model are output.

According to the embodiment of the present disclosure, the second training data can be divided into a second training set and a second test set. For each combined model, the combined model is first trained based on the second training set, and then based on the second training set. The test set tests the performance of the combined model.

For example, assume that model parameters for 30 base models have been determined based on the first training data. Next, based on the second training data, a first model B1 with the best performance among the 30 base models is determined, and the first model B1 is used as a combined model. Then, determine the second model B2 with the best performance of the combined model obtained by combining with the first model B1 from the remaining 29 base models, and determine the combination coefficient of the first model B1 and the second model B2, according to the The first model B1, the second model B2, and the combined coefficients of the first model B1 and the second model B2 are updated, and the combined model is updated. By analogy, the number of base models in the combined model is gradually increased until adding a new base model no longer improves the performance of the model combination, or the number of base models in the combined model is equal to 30, where each time the combined model is added The base model is the base model with the best performance of the combined model after adding the base model, determines the combination coefficient of each base model in the combined model after adding the base model, and finally outputs the base model and the base model used in the combined model. The corresponding combination coefficients of the base model used.

According to the embodiments of the present disclosure, the greedy algorithm is used to determine the base model used in the combined model and the corresponding coefficients of the used base model, which is beneficial to improve the efficiency of model training and reduce the time complexity.

An exemplary process of the model training method according to an embodiment of the present disclosure will be specifically described below with reference to FIG. 7 .

FIG. 7 illustrates an exemplary process of a model training method according to an embodiment of the present disclosure.

For the convenience of description, only one nonlinear model and one linear model are drawn among the multiple base models in FIG. 7 . It should be understood that this example is only used as an example, and is not intended to limit the present disclosure. The number of linear models and nonlinear models in each base model can be set according to actual needs, which is not specifically limited in the present disclosure.

As shown in FIG. 7 , after obtaining the original data, firstly remove the low-correlation features in the original data to obtain preprocessed data, and then by randomly dividing or dividing the preprocessed data by time, the first training data S1 is obtained. , the second training data S2 and the test data U.

For the first training data S1, first use the GBDT model T to process the first training data S1 to obtain intermediate training data X1, and then remove low correlation features in the intermediate training data X1 to obtain third training data S3.

With continued reference to FIG. 7 , the plurality of base models include linear models and nonlinear models. After the first training data S1, the second training data S2 and the third training data S3 are obtained, the nonlinear models in the multiple base models are trained based on the first training data S1, and based on the third training data S1 The training data trains linear models in the plurality of base models, thereby determining model parameters of each base model.

After the model parameters of the multiple base models are determined, based on the second training data S2, a base model used in the combined model and a corresponding combination system of the used base models are determined by a greedy algorithm.

Then, based on the test data U, the combined model is verified to evaluate the generalization ability of the combined model.

FIG. 8 shows a structural block diagram of a model training apparatus 700 according to an embodiment of the present disclosure. Wherein, the apparatus may be realized by software, hardware or a combination of the two to become part or all of the electronic device.

As shown in FIG. 8 , the model training apparatus 700 includes an acquisition module 701 , a first determination module 702 and a second determination module 703 .

The obtaining module 701 is configured to obtain the first training data and the second training data;

The first determining module 702 is configured to train a plurality of base models based on the first training data, and determine model parameters of each base model;

The second determination module 703 is configured to determine, based on the second training data, a base model used in the combined model and a corresponding combination coefficient of the used base model through a greedy algorithm.

According to an embodiment of the present disclosure, the plurality of base models include at least one linear model and/or at least one nonlinear model.

the linear model comprises a logistic regression model; and/or

The nonlinear model includes at least one of an extreme gradient boosting model, a factorization machine, and a random forest.

According to an embodiment of the present disclosure, the training of multiple base models based on the first training data includes:

using a gradient boosting tree model to process the first training data to obtain intermediate training data;

Remove the low correlation features in the intermediate training data to obtain the third training data;

The plurality of base models are trained based on the third training data.

According to an embodiment of the present disclosure, the training of a plurality of base models based on the first training data includes training a nonlinear model among the plurality of base models based on the first training data; and/or

The training of the plurality of base models based on the third training data includes training a linear model of the plurality of base models based on the third training data.

According to an embodiment of the present disclosure, determining the base model used in the combined model and the corresponding coefficients of the used base model through a greedy algorithm based on the second training data includes:

determining, based on the second training data, a first model with the best performance among the multiple base models, and using the first model as a combined model;

Gradually increase the number of base models in the combined model until adding a new base model no longer improves the performance of the model combination, or the number of base models in the combined model is equal to the total number of the multiple base models, where each The base model added to the combined model for the second time is the base model that makes the combined model performance after adding the base model to be optimal, and the combination coefficient of each base model in the combined model after adding the base model is determined;

The base model used in the combined model and the corresponding combination coefficients of the used base model are output. According to an embodiment of the present disclosure, the apparatus 700 further includes a removal module 704 and a segmentation module 705 .

The removing module 704 is configured to remove low correlation features in the original data to obtain preprocessed data;

The segmenting module 705 is configured to obtain the first training data, the second training data and the test data by randomly segmenting the preprocessed data or segmenting by time.

According to an embodiment of the present disclosure, the apparatus 700 further includes a verification module 706 .

The verification module 706 is configured to verify the combined model based on the test data.

According to an embodiment of the present disclosure, the first training data and the second training data include data related to user portraits;

The combined model and the base model are used to make predictions based on data related to user profiles.

The present disclosure also discloses an electronic device, and FIG. 9 shows a structural block diagram of the electronic device according to an embodiment of the present disclosure.

As shown in FIG. 9 , the electronic device 800 includes a memory 801 and a processor 802 . The memory 801 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 802 to implement the following method steps:

obtaining first training data and second training data;

Train multiple base models based on the first training data, and determine model parameters of each base model;

Based on the second training data, the base model used in the combined model and the corresponding combining coefficients of the used base model are determined by a greedy algorithm.

According to an embodiment of the present disclosure, the plurality of base models include at least one linear model and/or at least one nonlinear model.

According to an embodiment of the present disclosure, the linear model comprises a logistic regression model; and/or

The nonlinear model includes at least one of an extreme gradient boosting model, a factorization machine, and a random forest.

According to an embodiment of the present disclosure, the training of multiple base models based on the first training data includes:

using a gradient boosting tree model to process the first training data to obtain intermediate training data;

Remove the low correlation features in the intermediate training data to obtain the third training data;

The plurality of base models are trained based on the third training data.

According to an embodiment of the present disclosure, the training of a plurality of base models based on the first training data includes training a nonlinear model among the plurality of base models based on the first training data; and/or

The training of the plurality of base models based on the third training data includes training a linear model of the plurality of base models based on the third training data.

According to an embodiment of the present disclosure, determining the base model used in the combined model and the corresponding coefficients of the used base model through a greedy algorithm based on the second training data includes:

determining, based on the second training data, a first model with the best performance among the multiple base models, and using the first model as a combined model;

Gradually increase the number of base models in the combined model, until adding a new base model no longer improves the performance of the model combination, wherein each time the base model of the combined model is added to make the combined model after adding the base model the best performance the base model, determine the combination coefficient of each base model in the combined model after adding the base model;

The base model used in the combined model and the corresponding combination coefficients of the used base model are output.

According to an embodiment of the present disclosure, the one or more computer instructions are also executed by the processor 802 to implement the following method steps:

Remove low-correlation features in the original data to obtain preprocessed data;

The first training data, the second training data and the test data are obtained by randomly dividing or dividing the preprocessed data by time.

According to an embodiment of the present disclosure, the one or more computer instructions are also executed by the processor 802 to implement the following method steps:

The combined model is validated based on the test data.

According to an embodiment of the present disclosure, the first training data and the second training data include data related to user portraits;

The combined model and the base model are used to make predictions based on data related to user profiles.

FIG. 10 shows a schematic structural diagram of a computer system suitable for implementing the model training method according to an embodiment of the present disclosure.

As shown in FIG. 10, a computer system 900 includes a central processing unit (CPU) 901, which can be loaded into a random access memory (RAM) 903 according to a program stored in a read only memory (ROM) 902 or a program from a storage section 909 Instead, various processes in the above-described embodiments are performed. In the RAM 903, various programs and data necessary for the operation of the system 900 are also stored. The CPU 901 , the ROM 902 , and the RAM 903 are connected to each other through a bus 904 . An input/output (I/O) interface 905 is also connected to bus 904 .

The following components are connected to the I/O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 908 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc. ; and a communication section 909 including a network interface card such as a LAN card, a modem, and the like. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I/O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 910 as needed so that a computer program read therefrom is installed into the storage section 908 as needed.

In particular, according to embodiments of the present disclosure, the methods described above may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product including a computer program tangibly embodied on a readable medium thereof, the computer program including program code for performing the above-described object class determination method. In such an embodiment, the computer program may be downloaded and installed from the network via the communication portion 909, and/or installed from the removable medium 911.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the diagram or block diagram may represent a module, segment, or portion of code that contains one or more functions for implementing the specified logical function. executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.

The units or modules involved in the embodiments of the present disclosure may be implemented in a software manner, or may be implemented in a programmable hardware manner. The described units or modules may also be provided in the processor, and the names of these units or modules do not constitute a limitation on the units or modules themselves in certain circumstances.

As another aspect, the present disclosure also provides a readable storage medium. The readable storage medium may be the readable storage medium included in the electronic device or computer system in the above-mentioned embodiments; readable storage medium in the device. The readable storage medium stores one or more programs used by one or more processors to perform the methods described in the present disclosure.

The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, and should also cover the above-mentioned technical features without departing from the inventive concept. Other technical solutions formed by any combination of its equivalent features. For example, a technical solution is formed by replacing the above features with the technical features disclosed in the present disclosure (but not limited to) with similar functions.

Claims (10)

1. a model training method, is characterized in that, comprises: obtaining first training data and second training data; Train multiple base models based on the first training data, and determine model parameters of each base model; Based on the second training data, the base model used in the combined model and the corresponding combining coefficients of the used base model are determined by a greedy algorithm. 2. method according to claim 1, is characterized in that: The plurality of base models include at least one linear model and/or at least one nonlinear model. 3. method according to claim 2, is characterized in that: the linear model comprises a logistic regression model; and/or The nonlinear model includes at least one of an extreme gradient boosting model, a factorization machine, and a random forest. 4. The method according to claim 2, wherein the training of multiple base models based on the first training data comprises: using a gradient boosting tree model to process the first training data to obtain intermediate training data; Remove the low correlation features in the intermediate training data to obtain the third training data; The plurality of base models are trained based on the third training data. 5. method according to claim 4, is characterized in that: The training of multiple base models based on the first training data includes training a nonlinear model in the multiple base models based on the first training data; and/or The training of the plurality of base models based on the third training data includes training a linear model of the plurality of base models based on the third training data. 6. The method according to claim 1, wherein, determining the base model used in the combined model and the corresponding coefficients of the used base model by a greedy algorithm based on the second training data, comprising: determining, based on the second training data, a first model with the best performance among the multiple base models, and using the first model as a combined model; Gradually increase the number of base models in the combined model until adding a new base model no longer improves the performance of the model combination, or the number of base models in the combined model is equal to the total number of the multiple base models, wherein each time The base model added to the combined model is a base model with the best performance of the combined model after adding the base model, and the combination coefficient of each base model in the combined model after adding the base model is determined; The base model used in the combined model and the corresponding combination coefficients of the used base model are output. 7. The method of claim 1, further comprising: Remove low-correlation features in the original data to obtain preprocessed data; The first training data, the second training data and the test data are obtained by randomly dividing or dividing the preprocessed data by time. 8. A model training device, comprising: an acquisition module, configured to acquire the first training data and the second training data; a first determination module, configured to train a plurality of base models based on the first training data, and to determine model parameters of each base model; The second determination module is configured to determine, based on the second training data, a base model used in the combined model and a corresponding combination coefficient of the used base model through a greedy algorithm. 9. An electronic device, comprising a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to achieve The method steps of any one of claims 1-7. 10. A readable storage medium on which computer instructions are stored, wherein the computer instructions implement the method steps of any one of claims 1-7 when the computer instructions are executed by a processor.