WO2020078098A1 - 一种基于梯度提升决策树的模型训练方法及装置 - Google Patents

一种基于梯度提升决策树的模型训练方法及装置 Download PDF

Info

Publication number
WO2020078098A1
WO2020078098A1 PCT/CN2019/101335 CN2019101335W WO2020078098A1 WO 2020078098 A1 WO2020078098 A1 WO 2020078098A1 CN 2019101335 W CN2019101335 W CN 2019101335W WO 2020078098 A1 WO2020078098 A1 WO 2020078098A1
Authority
WO
WIPO (PCT)
Prior art keywords
training
sample set
decision tree
trained
data
Prior art date
Application number
PCT/CN2019/101335
Other languages
English (en)
French (fr)
Inventor
陈超超
周俊
Original Assignee
阿里巴巴集团控股有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴集团控股有限公司 filed Critical 阿里巴巴集团控股有限公司
Priority to SG11202100918SA priority Critical patent/SG11202100918SA/en
Publication of WO2020078098A1 publication Critical patent/WO2020078098A1/zh
Priority to US17/158,451 priority patent/US11157818B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2148—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the process organisation or structure, e.g. boosting cascade
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G06N5/025—Extracting rules from data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/027—Frames

Definitions

  • the embodiments of the present specification relate to the field of information technology, and in particular, to a model training method and device based on a gradient boosting decision tree.
  • the data domain of a business scenario is actually a collection of business data generated based on the business scenario.
  • the embodiments of this specification provide a model training method and device based on a gradient boosting decision tree.
  • the technical solutions are as follows:
  • a model training method based on a gradient boosting decision tree for training a target model applied to a target business scenario.
  • the method includes:
  • the first sample set is a set of labeled samples obtained from the data domain of the source business scene;
  • the source business scene is a business scene close to the target business scene;
  • the second sample set is a set of labeled samples obtained from the data domain of the target business scenario
  • the target model is obtained by integrating the trained decision tree.
  • a prediction method including:
  • the model input feature is input to a prediction model applied to the target business scenario to output a prediction result; the prediction model is obtained according to the method of the first aspect described above.
  • a model training device based on a gradient boosting decision tree for training a target model applied to a target business scenario.
  • the device includes:
  • the first obtaining module obtains a first sample set; the first sample set is a set of labeled samples obtained from the data domain of the source business scene; the source business scene is a business close to the target business scene Scenes;
  • the first training module uses the first sample set, executes the gradient boosting decision tree GBDT algorithm process, and sequentially trains at least one decision tree until the preset training suspension conditions are met;
  • the calculation module determines the training residual according to the decision tree trained using the first sample set
  • a second acquisition module to acquire a second sample set;
  • the second sample set is a set of labeled samples obtained from the data domain of the target business scenario;
  • the second training module uses the second sample set, continues to execute the GBDT algorithm process based on the training residuals, and sequentially trains at least one decision tree until the preset training stop condition is satisfied;
  • the target model is obtained by integrating the trained decision tree.
  • a prediction device including:
  • the obtaining module obtains the data to be predicted from the data domain of the target business scenario
  • a determining module based on the data to be predicted, determining a model input feature corresponding to the data to be predicted
  • the input module inputs the model input features to the prediction model applied to the target business scenario to output the prediction result; the prediction model is obtained according to the method of the first aspect above.
  • the technical solution provided in the embodiment of this specification divides a GBDT algorithm process into two stages.
  • the labeled samples are obtained from the data domain of the business scene close to the target business scene to train several decision trees in sequence and determine The training residuals generated after the training in the previous stage; in the latter stage, the labeled samples are obtained from the data domain of the target business scene, and based on the training residuals, a number of decision trees are continuously trained.
  • the model applied to the target business scenario is actually obtained by integrating the decision tree trained in the previous stage with the decision tree trained in the latter stage.
  • the model applied to the target business scenario can be trained with the data of the business scenario close to the target business scenario. After testing, you can get qualified models.
  • any one of the embodiments of this specification does not need to achieve all the above-mentioned effects.
  • FIG. 1 is a schematic flowchart of a model training method based on a gradient boosting decision tree provided by an embodiment of this specification;
  • FIG. 2 is a schematic diagram of a solution architecture provided by an embodiment of this specification.
  • FIG. 3 is a schematic flowchart of a prediction method provided by an embodiment of this specification.
  • FIG. 4 is a schematic structural diagram of a model training device based on a gradient boosting decision tree provided by an embodiment of this specification;
  • FIG. 5 is a schematic structural diagram of a prediction device provided by an embodiment of this specification.
  • FIG. 6 is a schematic structural diagram of a device for configuring the method of the embodiment of the present specification.
  • the invention draws on the transfer learning ideas in the field of machine learning technology.
  • the data accumulated in the target business scenario is insufficient, the data accumulated in the business scenario close to the target business scenario can be used for model training.
  • the present invention combines the idea of transfer learning with the Gradient Boosting Decision Tree (GBDT) algorithm to improve the GBDT algorithm flow.
  • GBDT Gradient Boosting Decision Tree
  • training is first performed using data generated in a business scenario close to the target business scenario, after certain training suspension conditions are met, the training is suspended, and the current training residual is calculated; then , Using data generated in the target business scenario, continue training based on the training residuals until a certain training stop condition is met. In this way, applying the trained GBDT model to the target business scenario can achieve better prediction results.
  • the business scenarios that are close to the target business scenario are actually business scenarios that are similar or related to the target business scenario.
  • This article refers to the business scenario that is similar to the target business scenario as the source business scenario.
  • the target business scenario is a male product recommendation scenario
  • a model for predicting the age of male users needs to be trained.
  • the accumulated purchase records of male users are small (the purchase records record various characteristics of the purchaser and the age of the purchaser), because sufficient labeled samples cannot be obtained for training. Therefore, the female business product recommendation scene can be used as the source business scene corresponding to the target business scene. Since the female product recommendation function has been launched for a long time, a large number of female user purchase records have been accumulated. Therefore, in the embodiment of this specification, a small number of male user purchase records can be used to train qualified results by virtue of the accumulated large number of female user purchase records. A model used to predict the age of male users.
  • FIG. 1 is a schematic flowchart of a gradient training decision tree-based model training method provided by an embodiment of the present specification, including the following steps:
  • the purpose of this method is to train the target model applied to the target business scenario.
  • the data domain of the source business scenario corresponding to the target business scenario accumulates more data, and a large amount of business data can be obtained from the data domain of the source business scenario for labeling to obtain a sufficient number of labeled samples.
  • the first sample set in step S100 is a set of labeled samples obtained from the data domain of the source business scene.
  • the target business scenario is a male product recommendation scenario
  • the source business scenario is a female product recommendation scenario.
  • S102 Using the first sample set, execute a gradient boosting decision tree GBDT algorithm process, and train at least one decision tree in sequence until a preset training suspension condition is met.
  • the GBDT algorithm is a typical integrated learning algorithm. In the GBDT algorithm process, some labeled samples are used to train more than two decision trees in sequence, and then the trained decision trees are integrated into a model as a training result.
  • the first decision tree trained is actually used to fit the labeled value of each labeled sample.
  • the current training residual can be calculated.
  • the training residual is used to characterize the gap between the predicted value obtained by predicting each labeled sample and the labeled value of each labeled sample as of the current training progress. It can be seen that, in a GBDT algorithm process, the training residual is updated once every decision tree is trained.
  • the training residuals calculated from all decision trees before the other decision tree.
  • the training residual is small enough, it means that the fitting effect of the model parameters of the current model to the labeled values of the labeled samples is up to standard. At this time, the training can be ended.
  • the first decision tree trained is used to fit the labeled values of the samples.
  • the fitting actually means that for the first decision tree, the characteristics of the sample are used as the input of the decision tree, and the labeled value of the sample is used to output the decision tree, and the decision tree parameters are trained to make the decision tree parameters and decision tree input Match the output of the decision tree.
  • the decision tree is used to fit the training residuals calculated from all previous decision trees.
  • the fitting actually means that for each decision tree that is subsequently trained, the characteristics of the sample are used as the input to the decision tree, and the training residuals are used as the output of the decision tree. The input matches the output of the decision tree.
  • Step S102 is a step executed in the previous stage. Specifically, in step S102, the first sample set is used to execute the GBDT algorithm flow, and at least one decision tree is trained in sequence. When the preset training suspension conditions are met, the training is suspended, that is, the previous stage of training is completed.
  • the training suspension condition may be specified according to actual needs.
  • the training suspension condition may be that the number of decision trees trained using the first sample set reaches a first specified number.
  • the timing of suspending the first stage of training is determined by limiting the number of decision trees, which is more intuitive and easy to execute.
  • the training suspension condition may be that the training residual calculated based on the trained decision trees falls within the first specified threshold interval.
  • S104 Determine a training residual according to the decision tree trained using the first sample set.
  • the training is suspended, and the training residual generated during the training process so far is calculated according to the decision tree trained using the first sample set.
  • step S104 the current training residuals are calculated according to the decision trees trained in the previous stage, which also means that the integration of the staged training results in the previous stage is completed.
  • the second sample set is a set of labeled samples obtained from the data domain of the target business scenario.
  • the number of labeled samples in the first sample set is significantly larger than the number of labeled samples in the second sample set.
  • step S106 may be executed before any of steps S100 to S104.
  • S108 Using the second sample set, continue to execute the GBDT algorithm process based on the training residual, and train at least one decision tree in sequence until a preset training stop condition is satisfied.
  • Step S108 is a step executed in the latter stage.
  • the latter stage using the second sample set, based on the training residuals, continue to execute the GBDT algorithm process, when the preset training stop conditions are met, the training is stopped, and the training in the latter stage is completed.
  • step S108 the second sample set is actually used to inherit the staged training results of the previous stage and continue to train the decision tree.
  • the first decision tree trained using the second sample set is used to fit the training residual generated in the previous stage.
  • the decision tree is used to fit all decision trees before the decision tree (including training using the first sample set The decision tree and the training tree calculated using the second sample set).
  • the training stop condition when the preset training stop condition is satisfied, the latter stage will end (ie, stop training).
  • the training stop condition may be specified according to actual needs.
  • the training stop condition may be that the number of decision trees trained using the second sample set reaches a second specified number.
  • the training stopping condition may be that the training residual calculated based on the trained decision trees falls within the second specified threshold interval.
  • the value of the right endpoint of the second threshold interval is smaller than the value of the left endpoint of the first threshold interval.
  • the trained decision trees can be integrated to obtain the target model applied to the target business scenario.
  • the target model may be a sequence of decision trees obtained by sorting the decision trees in order of generation, first to last.
  • a GBDT algorithm process is divided into two stages.
  • the labeled samples are obtained from the data domain of the business scenario close to the target business scenario to train several decision trees in sequence and determine The training residuals generated after the training in the previous stage; in the latter stage, the labeled samples are obtained from the data domain of the target business scene, and based on the training residuals, a number of decision trees are continuously trained.
  • the model applied to the target business scenario is actually obtained by integrating the decision tree trained in the first stage with the decision tree trained in the second stage.
  • the model applied to the target business scenario can be trained with the data of the business scenario close to the target business scenario. After testing, you can get qualified models.
  • the labeled samples may be obtained from the data domain of more than one source business scenario to perform the training in the previous stage.
  • the target business scenario is a male product recommendation scenario
  • the source business scenarios that can be obtained are a female product recommendation scenario and a children's product recommendation scenario, and the data generated under these two source business scenarios are used in turn for the previous stage of the GBDT algorithm process Training.
  • a third sample set may be acquired.
  • the third sample set is a set of labeled samples obtained from the data fields of other source business scenarios.
  • step S108 using the third sample set, continue to execute the GBDT algorithm process based on the training residuals, train at least one decision tree in sequence until the preset training and then pause conditions are met, and according to the use of the third The decision tree trained from the sample set and the decision tree trained using the third sample set re-determine the training residual.
  • the training re-pause condition can be specified according to actual needs.
  • the training suspension condition may be that the number of decision trees trained using the third sample set reaches a third specified number.
  • the training suspension condition may be that the training residual calculated based on the trained decision trees falls within the third specified threshold interval.
  • the GBDT algorithm process is divided into two stages. In the previous stage, the data generated in at least two source business scenarios is used to execute the GBDT algorithm process. In the latter stage, the data generated in the target business scenario is used to continue the GBDT algorithm process. .
  • FIG. 2 is a schematic diagram of a solution architecture provided by an embodiment of this specification.
  • the data in the data domain of the source business scenario is used for more than one (3 in the figure) to perform the training in the previous stage of the GBDT algorithm process, and then, the data in the data domain of the target business scenario is used to start In the first stage of training, the training residuals generated in the previous stage are fitted to finally obtain the model.
  • the data accumulated in the source business scenario A starts to execute the GBDT algorithm process.
  • the training residuals up to the current are calculated, Record as training residual 1.
  • the training up to the current training is calculated
  • the residual is recorded as training residual 2.
  • using the data accumulated in the source business scenario C, based on the training residual 2 continue to execute the GBDT algorithm process.
  • the training residual 3 is actually the training residual to be inherited in the later stage (that is, the training residual described in step S108).
  • the data accumulated in the target business scenario is used to continue the GBDT algorithm process based on the training residual 3.
  • the training is stopped.
  • FIG. 3 is a schematic flowchart of a prediction method provided by an embodiment of this specification, including the following steps:
  • S302 Determine model input features corresponding to the data to be predicted according to the data to be predicted.
  • S304 Input the model input feature to the prediction model applied to the target business scenario to output the prediction result.
  • the prediction model is obtained according to the training method shown in FIG. 1.
  • the embodiment of this specification also correspondingly provides a model training device based on a gradient boosting decision tree for training a target model applied to a target business scenario, as shown in FIG. 4,
  • the device includes:
  • the first obtaining module 401 obtains a first sample set; the first sample set is a set of labeled samples obtained from the data domain of the source business scene; the source business scene is similar to the target business scene Business scene;
  • the first training module 402 uses the first sample set, executes the gradient boosting decision tree GBDT algorithm process, and sequentially trains at least one pre-decision tree until the preset training suspension conditions are met;
  • the calculation module 403 determines the training residual according to the decision tree trained using the first sample set
  • the second obtaining module 404 obtains a second sample set; the second sample set is a set of labeled samples obtained from the data domain of the target business scenario;
  • the second training module 405 uses the second sample set, continues to execute the GBDT algorithm process based on the training residuals, and sequentially trains at least one post-decision tree until the preset training stop condition is satisfied;
  • the target model is obtained by integrating the trained decision tree.
  • the training suspension condition specifically includes:
  • the number of decision trees trained using the first sample set reaches the first specified number.
  • the training stop condition specifically includes:
  • the number of decision trees trained using the first sample set reaches a second specified number.
  • the device also includes:
  • the reprocessing module 406 obtains a third sample set before continuing to execute the GBDT algorithm process based on the training residual using the second sample set; the third sample set is obtained from the data domain of other source business scenarios A set of labeled samples; using the third sample set, continue to execute the GBDT algorithm process based on the training residuals, and train at least one decision tree in sequence until the preset training and then pause conditions are met; according to the use of the first The decision tree trained by this set and the decision tree trained by using the third sample set re-determine the training residual.
  • the training pause condition specifically includes:
  • the number of decision trees trained using the third sample set reaches the third specified number.
  • an embodiment of this specification also correspondingly provides a prediction device, as shown in FIG. 5, including:
  • the obtaining module 501 obtains the data to be predicted from the data domain of the target business scenario
  • the determining module 502 determines the model input characteristics corresponding to the data to be predicted according to the data to be predicted;
  • the input module 503 inputs the model input features to the prediction model applied to the target business scenario to output the prediction result; the prediction model is obtained according to the method shown in FIG. 1.
  • An embodiment of the present specification also provides a computer device, which includes at least a memory, a processor, and a computer program stored on the memory and executable on the processor, where the method shown in FIG. 1 is implemented when the processor executes the program .
  • the device may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050.
  • the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 realize the communication connection between the devices within the device through the bus 1050.
  • the processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. Programs to implement the technical solutions provided by the embodiments of the present specification.
  • the memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory, random access memory), static storage devices, and dynamic storage devices.
  • the memory 1020 may store an operating system and other application programs.
  • related program codes are stored in the memory 1020 and called and executed by the processor 1010.
  • the input / output interface 1030 is used to connect input / output modules to realize information input and output.
  • the input / output / module can be configured as a component in the device (not shown in the figure), or can be externally connected to the device to provide corresponding functions.
  • the input device may include a keyboard, mouse, touch screen, microphone, various sensors, etc.
  • the output device may include a display, a speaker, a vibrator, an indicator light, and the like.
  • the communication interface 1040 is used to connect a communication module (not shown in the figure) to implement communication interaction between the device and other devices.
  • the communication module can realize communication through a wired method (such as USB, network cable, etc.), and can also implement communication through a wireless method (such as mobile network, WIFI, Bluetooth, etc.).
  • the bus 1050 includes a path for transferring information between various components of the device (for example, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).
  • the device may also include the necessary to achieve normal operation Other components.
  • the above-mentioned device may also include only the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figures.
  • Embodiments of the present specification also provide a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method shown in FIG. 1 is implemented.
  • Computer-readable media including permanent and non-permanent, removable and non-removable media, can store information by any method or technology.
  • the information may be computer readable instructions, data structures, modules of programs, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices.
  • computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.
  • the system, method, module or unit explained in the above embodiments may be implemented by a computer chip or entity, or by a product with a certain function.
  • a typical implementation device is a computer, and the specific form of the computer may be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email sending and receiving device, and a game control Desk, tablet computer, wearable device, or any combination of these devices.

Abstract

公开了一种基于梯度提升决策树的模型训练方法及装置。将一个GBDT算法流程划分为两个阶段，在前一阶段，从与目标业务场景相近的业务场景的数据域获取已标注样本依次训练若干决策树，并确定经过前一阶段训练后产生的训练残差；在后一阶段，从目标业务场景的数据域获取已标注样本，并基于所述训练残差，继续训练若干决策树。最终，应用于目标业务场景的模型实际上是由前一阶段训练出的决策树与后一阶段训练出的决策树集成得到的。

Description

一种基于梯度提升决策树的模型训练方法及装置 技术领域
本说明书实施例涉及信息技术领域，尤其涉及一种基于梯度提升决策树的模型训练方法及装置。
背景技术
众所周知，当需要训练应用于某个业务场景的预测模型时，通常需要从该业务场景的数据域获取大量数据进行标注，作为已标注样本，进行模型训练。如果已标注样本的数量较少，则通常无法得到效果合格的模型。需要说明的是，某个业务场景的数据域，实际上是基于该业务场景所产生的业务数据的集合。
然而，实践中，某些特殊业务场景下积累的数据较少。这导致当需要训练应用于某个特殊业务场景的模型时，无法从该特殊业务场景的数据域获取足够的已标注样本，从而无法得到效果合格的模型。
发明内容
为了解决某些特殊业务场景下积累的数据较少导致无法训练出效果合格的模型的问题，本说明书实施例提供一种基于梯度提升决策树的模型训练方法及装置，技术方案如下：
根据本说明书实施例的第1方面，提供一种基于梯度提升决策树的模型训练方法，用于训练应用于目标业务场景的目标模型，所述方法包括：
获取第一样本集合；所述第一样本集合是从源业务场景的数据域获取的已标注样本的集合；所述源业务场景是与所述目标业务场景相近的业务场景；
使用所述第一样本集合，执行梯度提升决策树GBDT算法流程，依次训练至少一个决策树，直至满足预设的训练暂停条件；
根据使用所述第一样本集合训练出的决策树，确定训练残差；
获取第二样本集合；所述第二样本集合是从所述目标业务场景的数据域获取的已标注样本的集合；
使用所述第二样本集合，基于所述训练残差继续执行GBDT算法流程，依次训练至少一个决策树，直至满足预设的训练停止条件；
其中，所述目标模型是由已训练出的决策树集成得到的。
根据本说明书实施例的第2方面，提供一种预测方法，包括：
从目标业务场景的数据域获取待预测数据；
根据所述待预测数据，确定所述待预测数据对应的模型输入特征；
将所述模型输入特征输入到应用于所述目标业务场景的预测模型，以输出预测结果；所述预测模型是根据上述第1方面的方法得到的。
根据本说明书实施例的第3方面，提供一种基于梯度提升决策树的模型训练装置，用于训练应用于目标业务场景的目标模型，所述装置包括：
第一获取模块，获取第一样本集合；所述第一样本集合是从源业务场景的数据域获取的已标注样本的集合；所述源业务场景是与所述目标业务场景相近的业务场景；
第一训练模块，使用所述第一样本集合，执行梯度提升决策树GBDT算法流程，依次训练至少一个决策树，直至满足预设的训练暂停条件；
计算模块，根据使用所述第一样本集合训练出的决策树，确定训练残差；
第二获取模块，获取第二样本集合；所述第二样本集合是从所述目标业务场景的数据域获取的已标注样本的集合；
第二训练模块，使用所述第二样本集合，基于所述训练残差继续执行GBDT算法流程，依次训练至少一个决策树，直至满足预设的训练停止条件；
其中，所述目标模型是由已训练出的决策树集成得到的。
根据本说明书实施例的第4方面，提供一种预测装置，包括：
获取模块，从目标业务场景的数据域获取待预测数据；
确定模块，根据所述待预测数据，确定所述待预测数据对应的模型输入特征；
输入模块，将所述模型输入特征输入到应用于所述目标业务场景的预测模型，以输出预测结果；所述预测模型是根据上述第1方面的方法得到的。
本说明书实施例所提供的技术方案，将一个GBDT算法流程划分为两个阶段，在前一阶段，从与目标业务场景相近的业务场景的数据域获取已标注样本依次训练若干决 策树，并确定经过前一阶段训练后产生的训练残差；在后一阶段，从目标业务场景的数据域获取已标注样本，并基于所述训练残差，继续训练若干决策树。最终，应用于目标业务场景的模型实际上是由前一阶段训练出的决策树与后一阶段训练出的决策树集成得到的。通过本说明书实施例，虽然目标业务场景下积累的数据不足，但是，可以借助与目标业务场景相近的业务场景的数据，训练应用于目标业务场景的模型。经过测试，可以得到效果合格的模型。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本说明书实施例。
此外，本说明书实施例中的任一实施例并不需要达到上述的全部效果。
附图说明
为了更清楚地说明本说明书实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本说明书实施例中记载的一些实施例，对于本领域普通技术人员来讲，还可以根据这些附图获得其他的附图。
图1是本说明书实施例提供的一种基于梯度提升决策树的模型训练方法的流程示意图；
图2是本说明书实施例提供的方案架构示意图；
图3是本说明书实施例提供的一种预测方法的流程示意图；
图4是本说明书实施例提供的一种基于梯度提升决策树的模型训练装置的结构示意图；
图5是本说明书实施例提供的一种预测装置的结构示意图；
图6是用于配置本说明书实施例方法的一种设备的结构示意图。
具体实施方式
本发明借鉴了机器学习技术领域的迁移学习思想。在面对训练应用于目标业务场景的模型的需求时，如果目标业务场景下积累的数据不足，那么可以利用与目标业务场景相近的业务场景下积累的数据进行模型训练。
具体地，本发明将迁移学习思想与梯度提升决策树(Gradient Boosting Decision Tree，GBDT)算法相结合，对GBDT算法流程进行了改进。在本说明书实施例中，针对一个GBDT算法流程，先使用与目标业务场景相近的业务场景下产生的数据进行训练，满足一定的训练暂停条件之后，暂停训练，并计算当前的训练残差；随后，使用目标业务场景下产生的数据，基于所述训练残差继续训练，直到满足一定的训练停止条件。如此，将训练得到的GBDT模型应用于目标业务场景，可以取得较好的预测效果。
需要说明的是，在本文中，与目标业务场景相近的业务场景，实际上是与目标业务场景相类似或相关联的业务场景。本文将与目标业务场景相近的业务场景称为源业务场景。
举例来说，假设目标业务场景是男性商品推荐场景，为了更好的根据男性用户的年龄进行商品推荐，需要训练用于预测男性用户年龄的模型。然而，由于男性商品推荐功能上线不久，积累的男性用户购买记录较少(购买记录中记载了购买者的各种特征信息以及购买者的年龄)，因为无法获得足够的已标注样本进行训练。于是，可以以女性商品推荐场景为目标业务场景对应的源业务场景。由于女性商品推荐功能早已上线，已经积累了大量女性用户购买记录，因此，在本说明书实施例中，可以借助于积累的大量女性用户购买记录，使用少量的男性用户购买记录训练出效果合格的，用于预测男性用户年龄的模型。
为了使本领域技术人员更好地理解本说明书实施例中的技术方案，下面将结合本说明书实施例中的附图，对本说明书实施例中的技术方案进行详细地描述，显然，所描述的实施例仅仅是本说明书的一部分实施例，而不是全部的实施例。基于本说明书中的实施例，本领域普通技术人员所获得的所有其他实施例，都应当属于保护的范围。
以下结合附图，详细说明本说明书各实施例提供的技术方案。
图1是本说明书实施例提供的一种基于梯度提升决策树的模型训练方法的流程示意图，包括以下步骤：
S100：获取第一样本集合。
本方法的目的是训练应用于目标业务场景的目标模型。
在本说明书实施例中，目标业务场景对应的源业务场景的数据域中积累的数据较多，可以从源业务场景的数据域中获取大量业务数据进行标注，得到足量的已标注样本。步骤S100中的第一样本集合是从所述源业务场景的数据域获取的已标注样本的集合。
沿用前文所述的例子，目标业务场景是男性商品推荐场景，源业务场景是女性商品推荐场景。从源业务场景的数据域获取若干已标注样本，具体可以是从女性商品推荐场景积累的购买记录中，获取若干购买记录，然后，针对获取的每个购买记录，从该购买记录中提取购买者的学历、收入、工作类型、身高等特征信息，构建该购买记录对应的用户特征向量，然后，从该购买记录中提取购买者的年龄，作为该用户特征向量的标注值，如此，得到一个已标注样本。
S102：使用所述第一样本集合，执行梯度提升决策树GBDT算法流程，依次训练至少一个决策树，直至满足预设的训练暂停条件。
此处先对GBDT算法的原理进行介绍。
GBDT算法是一种典型的集成学习算法，在GBDT算法流程中，使用一些已标注样本依次训练出两个以上的决策树，然后将训练出的各决策树集成为一个模型，作为训练结果。
其中，训练的第一个决策树实际上用于拟合各已标注样本的标注值。训练出第一个决策树之后，可以计算得到当前的训练残差。所述训练残差用于表征，截至当前的训练进度，对各已标注样本进行预测得到的预测值与各已标注样本的标注值之间的差距。可见，在一个GBDT算法流程中，每训练出一个决策树，所述训练残差会更新一次。
在GBDT算法流程中，训练第一个决策树之后，会继续训练下一个决策树。针对除第一个决策树之外的每个其他决策树，该其他决策树用于拟合根据该其他决策树之前所有决策树计算出的训练残差。如此，随着GBDT算法流程的推进，越来越多的决策树被依次训练出来，所述训练残差会越来越小。当训练残差足够小时，说明当前的模型的模型参数对各已标注样本的标注值的拟合效果达标，此时，便可以结束训练。
还需要说明的是，在GBDT算法中，拟合有两层含义：
其一，训练的第一个决策树用于拟合样本的标注值。此处的拟合实际上是指，针对第一个决策树，以样本的特征为决策树输入，以样本的标注值为决策树输出，训练决策树参数，尽量使得决策树参数、决策树输入与决策树输出相匹配。
其二，针对后续训练的每个决策树，该决策树用于拟合根据之前所有决策树计算得到的训练残差。此处的拟合实际上是指，针对后续训练的每个决策树，以样本的特征为决策树输入，以训练残差为决策树输出，训练决策树参数，尽量使得决策树参数、决策树输入与决策树输出相匹配。
而在本说明书实施例中，将GDBT算法流程拆分为两个阶段(前一阶段与后一阶段)。步骤S102是在前一阶段所执行的步骤。具体地，在步骤S102中，使用第一样本集合，执行GBDT算法流程，依次训练至少一个决策树，当满足预设的训练暂停条件时，暂停训练，也就是完成前一阶段的训练。
需要说明的是，所述训练暂停条件可以根据实际需要指定。例如，所述训练暂停条件可以是，使用所述第一样本集合训练出的决策树的数量达到第一指定数量。在实际应用中，采用对决策树数量进行限定的方式决定暂停第一阶段训练的时机，较为直观，方便执行。
又如，所述训练暂停条件可以是，基于已训练出的各决策树计算得到的训练残差落入第一指定阈值区间。
S104：根据使用所述第一样本集合训练出的决策树，确定训练残差。
当满足所述训练暂停条件时，暂停训练，并根据使用所述第一样本集合训练出的决策树，计算截至当前，训练过程产生的训练残差。
也就是说，在步骤S104中，也就是根据前一阶段训练出的各决策树计算当前的训练残差，这也意味着完成了对前一阶段的阶段性训练结果的整合。
S106：获取第二样本集合。
其中，所述第二样本集合是从所述目标业务场景的数据域获取的已标注样本的集合。
需要说明的是，一般而言，第一样本集合中的已标注样本的数量显著大于第二样本集合中的已标注样本的数量。
还需要说明的，在本说明书实施例中，并不对执行步骤S106的时机进行限制。实际上，可以在步骤S100～S104中任一步骤之前执行步骤S106。
S108：使用所述第二样本集合，基于所述训练残差继续执行所述GBDT算法流程，依次训练至少一个决策树，直至满足预设的训练停止条件。
在前一阶段之后，需要将所述训练残差迁移到后一阶段。步骤S108是在后一阶段所执行的步骤。在后一阶段，使用第二样本集合，基于所述训练残差，继续执行GBDT算法流程，当满足预设的训练停止条件时，停止训练，也就完成了后一阶段的训练。
在步骤S108中，实际上是使用第二样本集合，继承前一阶段的阶段性训练结果， 继续训练决策树。
其中，使用所述第二样本集合训练出的第一个决策树，用于拟合前一阶段产生的训练残差。针对使用所述第二样本集合训练出的第一个决策树之后的每个决策树，该决策树用于拟合根据该决策树之前的所有决策树(包括使用所述第一样本集合训练出的决策树和使用所述第二样本集合训练出的决策树)计算出的训练残差。
需要说明的是，当满足预设的训练停止条件时，所述后一阶段才会结束(即停止训练)。所述训练停止条件可以根据实际需要指定。
例如，所述训练停止条件可以是，使用所述第二样本集合训练出的决策树的数量达到第二指定数量。又如，所述训练停止条件可以是，基于已训练的各决策树计算得到的训练残差落入第二指定阈值区间。通常，所述第二阈值区间的右端点的取值小于所述第一阈值区间的左端点的取值。
通过步骤S100～S108，可以训练出若干决策树。于是，可以将训练出的各决策树进行集成，得到的应用于所述目标业务场景的目标模型。具体地，所述目标模型可以是对各决策树，按产生顺序，由先到后进行排序得到的决策树序列。
通过图1所示的模型训练方法，将一个GBDT算法流程划分为两个阶段，在前一阶段，从与目标业务场景相近的业务场景的数据域获取已标注样本依次训练若干决策树，并确定经过前一阶段训练后产生的训练残差；在后一阶段，从目标业务场景的数据域获取已标注样本，并基于所述训练残差，继续训练若干决策树。最终，应用于目标业务场景的模型实际上是由第一阶段训练出的决策树与第二阶段训练出的决策树集成得到的。通过本说明书实施例，虽然目标业务场景下积累的数据不足，但是，可以借助与目标业务场景相近的业务场景的数据，训练应用于目标业务场景的模型。经过测试，可以得到效果合格的模型。
此外，在本说明书实施例中，可以从不止一个源业务场景的数据域获取已标注样本，进行所述前一阶段的训练。例如，假设目标业务场景是男性商品推荐场景，那么可以获取的源业务场景为女性商品推荐场景和儿童商品推荐场景，依次使用这两个源业务场景下产生的数据进行GBDT算法流程中前一阶段的训练。
具体地，在步骤S108之前，可以获取获取第三样本集合。所述第三样本集合是从其他源业务场景的数据域获取的已标注样本的集合。
接着，在步骤S108之前，使用所述第三样本集合，基于所述训练残差继续执行 GBDT算法流程，依次训练至少一个决策树，直至满足预设的训练再暂停条件，并根据使用所述第一样本集合训练出的决策树和使用所述第三样本集合训练出的决策树，重新确定所述训练残差。
其中，所述训练再暂停条件可以根据实际需要指定。例如，所述训练暂停条件可以是，使用所述第三样本集合训练出的决策树的数量达到第三指定数量。又如，所述训练暂停条件可以是，基于已训练出的各决策树计算得到的训练残差落入第三指定阈值区间。
也就是说，在GBDT算法流程的前一阶段，可以先从一个源业务场景的数据域获取已标注样本的集合，并开始执行GBDT算法流程。当满足训练暂停条件时，先暂停训练，并计算当前的训练残差。随后，更换使用另一个源业务场景对应的已标注样本的结合，基于所述训练残差，继续执行GBDT算法流程。当满足训练再暂停条件时，再次暂停，并重新计算当前的训练残差。
之后，还可以再次更换其他源业务场景，继续进行前一阶段的训练。总之，本领域技术人员应当理解，如下技术方案应在本发明所要求的保护范围之内：
将GBDT算法流程分为两个阶段，在前一阶段，依次使用至少两个源业务场景下产生的数据执行GBDT算法流程，在后一阶段，使用目标业务场景下产生的数据继续执行GBDT算法流程。
图2是本说明书实施例提供的方案架构示意图。如图2所示，使用不止一个(图中以3个为例)源业务场景的数据域的数据执行GBDT算法流程中前一阶段的训练，然后，使用目标业务场景的数据域的数据开始后一阶段的训练，以拟合前一阶段产生的训练残差，最终得到模型。
如图2所示，使用源业务场景A积累的数据开始执行GBDT算法流程，当使用源业务场景A积累的数据所训练出的决策树的数量达到N 1时，计算截至当前的训练残差，记为训练残差1。随后，使用源业务场景B积累的数据，基于训练残差1，继续执行GBDT算法流程，当使用源业务场景B积累的数据所训练出的决策树的数量达到N 2时，计算截至当前的训练残差，记为训练残差2。随后，使用源业务场景C积累的数据，基于训练残差2，继续执行GBDT算法流程，当使用源业务场景C积累的数据所训练出的决策树的数量达到N 3时，计算截至当前的训练残差，记为训练残差3。至此，前一阶段结束。可见，在图2中，训练残差3实际上就是后一阶段所要继承的训练残差(即步骤S108 中所述训练残差)。
在后一阶段，使用目标业务场景积累的数据，基于训练残差3继续执行GBDT算法流程，当使用目标业务场景积累的数据所训练出的决策树的数量达到N 4时，停止训练。
图3是本说明书实施例提供的一种预测方法的流程示意图，包括如下步骤：
S300：从目标业务场景的数据域获取待预测数据。
S302：根据所述待预测数据，确定所述待预测数据对应的模型输入特征。
S304：将所述模型输入特征输入到应用于所述目标业务场景的预测模型，以输出预测结果。
其中，所述预测模型是根据图1所示的训练方法得到的。
基于图1所示的模型训练方法，本说明书实施例还对应提供了一种基于梯度提升决策树的模型训练装置，用于训练应用于目标业务场景的目标模型，如图4所示，所述装置包括：
第一获取模块401，获取第一样本集合；所述第一样本集合是从源业务场景的数据域获取的已标注样本的集合；所述源业务场景是与所述目标业务场景相近的业务场景；
第一训练模块402，使用所述第一样本集合，执行梯度提升决策树GBDT算法流程，依次训练至少一个前决策树，直至满足预设的训练暂停条件；
计算模块403，根据使用所述第一样本集合训练出的决策树，确定训练残差；
第二获取模块404，获取第二样本集合；所述第二样本集合是从所述目标业务场景的数据域获取的已标注样本的集合；
第二训练模块405，使用所述第二样本集合，基于所述训练残差继续执行GBDT算法流程，依次训练至少一个后决策树，直至满足预设的训练停止条件；
其中，所述目标模型是由已训练出的决策树集成得到的。
所述训练暂停条件，具体包括：
使用所述第一样本集合训练出的决策树的数量达到第一指定数量。
所述训练停止条件，具体包括：
使用所述第一样本集合训练出的决策树的数量达到第二指定数量。
所述装置还包括：
再处理模块406，在使用所述第二样本集合，基于所述训练残差继续执行GBDT算法流程之前，获取第三样本集合；所述第三样本集合是从其他源业务场景的数据域获取的已标注样本的集合；使用所述第三样本集合，基于所述训练残差继续执行GBDT算法流程，依次训练至少一个决策树，直至满足预设的训练再暂停条件；根据使用所述第一样本集合训练出的决策树和使用所述第三样本集合训练出的决策树，重新确定所述训练残差。
所述训练再暂停条件，具体包括：
使用所述第三样本集合训练出的决策树的数量达到第三指定数量。
基于图3所示的预测方法，本说明书实施例还对应提供了一种预测装置，如图5所示，包括：
获取模块501，从目标业务场景的数据域获取待预测数据；
确定模块502，根据所述待预测数据，确定所述待预测数据对应的模型输入特征；
输入模块503，将所述模型输入特征输入到应用于所述目标业务场景的预测模型，以输出预测结果；所述预测模型是根据图1所示的方法得到的。
本说明书实施例还提供一种计算机设备，其至少包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其中，处理器执行所述程序时实现图1所示的方法。
图6示出了本说明书实施例所提供的一种更为具体的计算设备硬件结构示意图，该设备可以包括：处理器1010、存储器1020、输入/输出接口1030、通信接口1040和总线1050。其中处理器1010、存储器1020、输入/输出接口1030和通信接口1040通过总线1050实现彼此之间在设备内部的通信连接。
处理器1010可以采用通用的CPU(Central Processing Unit，中央处理器)、微处理器、应用专用集成电路(Application Specific Integrated Circuit，ASIC)、或者一个或多个集成电路等方式实现，用于执行相关程序，以实现本说明书实施例所提供的技术方案。
存储器1020可以采用ROM(Read Only Memory，只读存储器)、RAM(Random Access Memory，随机存取存储器)、静态存储设备，动态存储设备等形式实现。存储 器1020可以存储操作系统和其他应用程序，在通过软件或者固件来实现本说明书实施例所提供的技术方案时，相关的程序代码保存在存储器1020中，并由处理器1010来调用执行。
输入/输出接口1030用于连接输入/输出模块，以实现信息输入及输出。输入输出/模块可以作为组件配置在设备中(图中未示出)，也可以外接于设备以提供相应功能。其中输入设备可以包括键盘、鼠标、触摸屏、麦克风、各类传感器等，输出设备可以包括显示器、扬声器、振动器、指示灯等。
通信接口1040用于连接通信模块(图中未示出)，以实现本设备与其他设备的通信交互。其中通信模块可以通过有线方式(例如USB、网线等)实现通信，也可以通过无线方式(例如移动网络、WIFI、蓝牙等)实现通信。
总线1050包括一通路，在设备的各个组件(例如处理器1010、存储器1020、输入/输出接口1030和通信接口1040)之间传输信息。
需要说明的是，尽管上述设备仅示出了处理器1010、存储器1020、输入/输出接口1030、通信接口1040以及总线1050，但是在具体实施过程中，该设备还可以包括实现正常运行所必需的其他组件。此外，本领域的技术人员可以理解的是，上述设备中也可以仅包含实现本说明书实施例方案所必需的组件，而不必包含图中所示的全部组件。
本说明书实施例还提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现图1所示的方法。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。
通过以上的实施方式的描述可知，本领域的技术人员可以清楚地了解到本说明书实施例可借助软件加必需的通用硬件平台的方式来实现。基于这样的理解，本说明书实 施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本说明书实施例各个实施例或者实施例的某些部分所述的方法。
上述实施例阐明的系统、方法、模块或单元，具体可以由计算机芯片或实体实现，或者由具有某种功能的产品来实现。一种典型的实现设备为计算机，计算机的具体形式可以是个人计算机、膝上型计算机、蜂窝电话、相机电话、智能电话、个人数字助理、媒体播放器、导航设备、电子邮件收发设备、游戏控制台、平板计算机、可穿戴设备或者这些设备中的任意几种设备的组合。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于方法实施例而言，由于其基本相似于方法实施例，所以描述得比较简单，相关之处参见方法实施例的部分说明即可。以上所描述的方法实施例仅仅是示意性的，其中所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，在实施本说明书实施例方案时可以把各模块的功能在同一个或多个软件和/或硬件中实现。也可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。
以上所述仅是本说明书实施例的具体实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本说明书实施例原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本说明书实施例的保护范围。

Claims (14)

  1. 一种基于梯度提升决策树的模型训练方法，用于训练应用于目标业务场景的目标模型，所述方法包括：
    获取第一样本集合；所述第一样本集合是从源业务场景的数据域获取的已标注样本的集合；所述源业务场景是与所述目标业务场景相近的业务场景；
    使用所述第一样本集合，执行梯度提升决策树GBDT算法流程，依次训练至少一个决策树，直至满足预设的训练暂停条件；
    根据使用所述第一样本集合训练出的决策树，确定训练残差；
    获取第二样本集合；所述第二样本集合是从所述目标业务场景的数据域获取的已标注样本的集合；
    使用所述第二样本集合，基于所述训练残差继续执行GBDT算法流程，依次训练至少一个决策树，直至满足预设的训练停止条件；
    其中，所述目标模型是由已训练出的决策树集成得到的。
  2. 如权利要求1所述的方法，所述训练暂停条件，具体包括：
    使用所述第一样本集合训练出的决策树的数量达到第一指定数量。
  3. 如权利要求1所述的方法，所述训练停止条件，具体包括：
    使用所述第二样本集合训练出的决策树的数量达到第二指定数量。
  4. 如权利要求1所述的方法，在使用所述第二样本集合，基于所述训练残差继续执行GBDT算法流程之前，所述方法还包括：
    获取第三样本集合；所述第三样本集合是从其他源业务场景的数据域获取的已标注样本的集合；
    使用所述第三样本集合，基于所述训练残差继续执行GBDT算法流程，依次训练至少一个决策树，直至满足预设的训练再暂停条件；
    根据使用所述第一样本集合训练出的决策树和使用所述第三样本集合训练出的决策树，重新确定所述训练残差。
  5. 如权利要求4所述的方法，所述训练再暂停条件，具体包括：
    使用所述第三样本集合训练出的决策树的数量达到第三指定数量。
  6. 一种预测方法，包括：
    从目标业务场景的数据域获取待预测数据；
    根据所述待预测数据，确定所述待预测数据对应的模型输入特征；
    将所述模型输入特征输入到应用于所述目标业务场景的预测模型，以输出预测结果； 所述预测模型是根据权利要求1～5任一项所述的方法得到的。
  7. 一种基于梯度提升决策树的模型训练装置，用于训练应用于目标业务场景的目标模型，所述装置包括：
    第一获取模块，获取第一样本集合；所述第一样本集合是从源业务场景的数据域获取的已标注样本的集合；所述源业务场景是与所述目标业务场景相近的业务场景；
    第一训练模块，使用所述第一样本集合，执行梯度提升决策树GBDT算法流程，依次训练至少一个前决策树，直至满足预设的训练暂停条件；
    计算模块，根据使用所述第一样本集合训练出的决策树，确定训练残差；
    第二获取模块，获取第二样本集合；所述第二样本集合是从所述目标业务场景的数据域获取的已标注样本的集合；
    第二训练模块，使用所述第二样本集合，基于所述训练残差继续执行GBDT算法流程，依次训练至少一个后决策树，直至满足预设的训练停止条件；
    其中，所述目标模型是由已训练出的决策树集成得到的。
  8. 如权利要求7所述的装置，所述训练暂停条件，具体包括：
    使用所述第一样本集合训练出的决策树的数量达到第一指定数量。
  9. 如权利要求7所述的装置，所述训练停止条件，具体包括：
    使用所述第一样本集合训练出的决策树的数量达到第二指定数量。
  10. 如权利要求7所述的装置，所述装置还包括：
    再处理模块，在使用所述第二样本集合，基于所述训练残差继续执行GBDT算法流程之前，获取第三样本集合；所述第三样本集合是从其他源业务场景的数据域获取的已标注样本的集合；使用所述第三样本集合，基于所述训练残差继续执行GBDT算法流程，依次训练至少一个决策树，直至满足预设的训练再暂停条件；根据使用所述第一样本集合训练出的决策树和使用所述第三样本集合训练出的决策树，重新确定所述训练残差。
  11. 如权利要求10所述的装置，所述训练再暂停条件，具体包括：
    使用所述第三样本集合训练出的决策树的数量达到第三指定数量。
  12. 一种预测装置，包括：
    获取模块，从目标业务场景的数据域获取待预测数据；
    确定模块，根据所述待预测数据，确定所述待预测数据对应的模型输入特征；
    输入模块，将所述模型输入特征输入到应用于所述目标业务场景的预测模型，以输出预测结果；所述预测模型是根据权利要求1～5任一项所述的方法得到的。
  13. 一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行 的计算机程序，其中，所述处理器执行所述程序时实现如权利要求1至5任一项所述的方法。
  14. 一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其中，所述处理器执行所述程序时实现如权利要求6所述的方法。
PCT/CN2019/101335 2018-10-17 2019-08-19 一种基于梯度提升决策树的模型训练方法及装置 WO2020078098A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
SG11202100918SA SG11202100918SA (en) 2018-10-17 2019-08-19 Model Training Method And Apparatus Based On Gradient Boosting Decision Tree
US17/158,451 US11157818B2 (en) 2018-10-17 2021-01-26 Model training method and apparatus based on gradient boosting decision tree

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811210339.9A CN109472296A (zh) 2018-10-17 2018-10-17 一种基于梯度提升决策树的模型训练方法及装置
CN201811210339.9 2018-10-17

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/158,451 Continuation US11157818B2 (en) 2018-10-17 2021-01-26 Model training method and apparatus based on gradient boosting decision tree

Publications (1)

Publication Number Publication Date
WO2020078098A1 true WO2020078098A1 (zh) 2020-04-23

Family

ID=65664073

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/101335 WO2020078098A1 (zh) 2018-10-17 2019-08-19 一种基于梯度提升决策树的模型训练方法及装置

Country Status (5)

Country Link
US (1) US11157818B2 (zh)
CN (1) CN109472296A (zh)
SG (1) SG11202100918SA (zh)
TW (1) TWI719557B (zh)
WO (1) WO2020078098A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112529624A (zh) * 2020-12-15 2021-03-19 北京百度网讯科技有限公司 生成业务预测模型的方法、装置、设备以及存储介质
CN113112311A (zh) * 2021-05-12 2021-07-13 北京百度网讯科技有限公司 训练因果推断模型的方法、信息提示方法以装置
CN113656692A (zh) * 2021-08-17 2021-11-16 中国平安财产保险股份有限公司 基于知识迁移算法的产品推荐方法、装置、设备及介质

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109472296A (zh) 2018-10-17 2019-03-15 阿里巴巴集团控股有限公司 一种基于梯度提升决策树的模型训练方法及装置
JP7196542B2 (ja) * 2018-11-05 2022-12-27 株式会社リコー 学習装置および学習方法
CN110033098A (zh) * 2019-03-28 2019-07-19 阿里巴巴集团控股有限公司 在线gbdt模型学习方法及装置
CN110070117B (zh) * 2019-04-08 2023-04-07 腾讯科技（深圳）有限公司 一种数据处理方法及装置
CN110348580B (zh) * 2019-06-18 2022-05-10 第四范式（北京）技术有限公司 构建gbdt模型的方法、装置及预测方法、装置
CN110942338A (zh) * 2019-11-01 2020-03-31 支付宝(杭州)信息技术有限公司 一种营销赋能策略的推荐方法、装置和电子设备
CN111353600B (zh) * 2020-02-20 2023-12-12 第四范式（北京）技术有限公司 一种异常行为检测方法及装置
US11568317B2 (en) * 2020-05-21 2023-01-31 Paypal, Inc. Enhanced gradient boosting tree for risk and fraud modeling
CN113742052B (zh) * 2020-05-29 2023-09-01 北京京东振世信息技术有限公司 批量任务处理方法和装置
CN112118259B (zh) * 2020-09-17 2022-04-15 四川长虹电器股份有限公司 一种基于提升树的分类模型的越权漏洞检测方法
CN112906905A (zh) * 2021-02-10 2021-06-04 华南师范大学 大数据和小数据相结合的人工智能方法和机器人
CN113705554A (zh) * 2021-08-13 2021-11-26 北京百度网讯科技有限公司 图像识别模型的训练方法、装置、设备及存储介质
CN115795289A (zh) * 2022-12-01 2023-03-14 北京淘友天下技术有限公司 特征识别方法、装置、电子设备及存储介质
CN115879914B (zh) * 2023-02-20 2023-05-12 苏州慧工云信息科技有限公司 设备保养的方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105868847A (zh) * 2016-03-24 2016-08-17 车智互联（北京）科技有限公司 一种购物行为的预测方法及装置
CN106548210A (zh) * 2016-10-31 2017-03-29 腾讯科技（深圳）有限公司 机器学习模型训练方法及装置
WO2017129030A1 (zh) * 2016-01-29 2017-08-03 阿里巴巴集团控股有限公司 磁盘的故障预测方法和装置
CN107944874A (zh) * 2017-12-13 2018-04-20 阿里巴巴集团控股有限公司 基于迁移学习的风控方法、装置及系统
CN109472296A (zh) * 2018-10-17 2019-03-15 阿里巴巴集团控股有限公司 一种基于梯度提升决策树的模型训练方法及装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2694001C2 (ru) * 2017-11-24 2019-07-08 Общество С Ограниченной Ответственностью "Яндекс" Способ и система создания параметра качества прогноза для прогностической модели, выполняемой в алгоритме машинного обучения
RU2720905C2 (ru) * 2018-09-17 2020-05-14 Общество С Ограниченной Ответственностью "Яндекс" Способ и система для расширения поисковых запросов с целью ранжирования результатов поиска

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017129030A1 (zh) * 2016-01-29 2017-08-03 阿里巴巴集团控股有限公司 磁盘的故障预测方法和装置
CN105868847A (zh) * 2016-03-24 2016-08-17 车智互联（北京）科技有限公司 一种购物行为的预测方法及装置
CN106548210A (zh) * 2016-10-31 2017-03-29 腾讯科技（深圳）有限公司 机器学习模型训练方法及装置
CN107944874A (zh) * 2017-12-13 2018-04-20 阿里巴巴集团控股有限公司 基于迁移学习的风控方法、装置及系统
CN109472296A (zh) * 2018-10-17 2019-03-15 阿里巴巴集团控股有限公司 一种基于梯度提升决策树的模型训练方法及装置

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112529624A (zh) * 2020-12-15 2021-03-19 北京百度网讯科技有限公司 生成业务预测模型的方法、装置、设备以及存储介质
CN112529624B (zh) * 2020-12-15 2024-01-09 北京百度网讯科技有限公司 生成业务预测模型的方法、装置、设备以及存储介质
CN113112311A (zh) * 2021-05-12 2021-07-13 北京百度网讯科技有限公司 训练因果推断模型的方法、信息提示方法以装置
CN113112311B (zh) * 2021-05-12 2023-07-25 北京百度网讯科技有限公司 训练因果推断模型的方法、信息提示方法以装置
CN113656692A (zh) * 2021-08-17 2021-11-16 中国平安财产保险股份有限公司 基于知识迁移算法的产品推荐方法、装置、设备及介质
CN113656692B (zh) * 2021-08-17 2023-05-30 中国平安财产保险股份有限公司 基于知识迁移算法的产品推荐方法、装置、设备及介质

Also Published As

Publication number Publication date
TW202036356A (zh) 2020-10-01
SG11202100918SA (en) 2021-03-30
US11157818B2 (en) 2021-10-26
TWI719557B (zh) 2021-02-21
US20210264291A1 (en) 2021-08-26
CN109472296A (zh) 2019-03-15

Similar Documents

Publication Publication Date Title
WO2020078098A1 (zh) 一种基于梯度提升决策树的模型训练方法及装置
JP7181437B2 (ja) 制御されていない照明条件の画像中の肌色を識別する技術
WO2020087979A1 (zh) 生成模型的方法和装置
TWI550417B (zh) 資料遷移方法及裝置
CN111046433A (zh) 一种基于联邦学习的模型训练方法
WO2020056903A1 (zh) 用于生成信息的方法和装置
US9275331B2 (en) Document classification system with user-defined rules
US10394907B2 (en) Filtering data objects
WO2020107625A1 (zh) 视频分类方法、装置、电子设备及计算机可读存储介质
JP7394809B2 (ja) ビデオを処理するための方法、装置、電子機器、媒体及びコンピュータプログラム
CN111079944B (zh) 迁移学习模型解释实现方法及装置、电子设备、存储介质
GB2505938A (en) ETL debugging
WO2020015411A1 (zh) 一种训练改编水平评价模型、评价改编水平的方法及装置
WO2020052062A1 (zh) 检测方法和装置
CN112306447A (zh) 一种界面导航方法、装置、终端和存储介质
US11501209B2 (en) Behavior identification method, behavior identification device, non-transitory computer-readable recording medium recording therein behavior identification program, machine learning method, machine learning device, and non-transitory computer-readable recording medium recording therein machine learning program
US11308273B2 (en) Prescan device activation prevention
WO2023011237A1 (zh) 业务处理
WO2023273621A1 (zh) 脚本生成方法、装置、设备及介质
US10375370B2 (en) Audio capture on mobile client devices
CN113282839B (zh) 一种互联网数据推送处理方法及系统
US10078666B2 (en) Normalizing non-numeric features of files
CN111752912B (zh) 数据处理方法及装置
KR101212845B1 (ko) 동영상 샘플링 방법 및 시스템
US20200252667A1 (en) Apparatuses, systems and methods for sharing content

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19874392

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19874392

Country of ref document: EP

Kind code of ref document: A1