CN112288101A - GBDT and LR fusion method, device, equipment and storage medium based on federal learning - Google Patents
GBDT and LR fusion method, device, equipment and storage medium based on federal learning Download PDFInfo
- Publication number
- CN112288101A CN112288101A CN202011182203.9A CN202011182203A CN112288101A CN 112288101 A CN112288101 A CN 112288101A CN 202011182203 A CN202011182203 A CN 202011182203A CN 112288101 A CN112288101 A CN 112288101A
- Authority
- CN
- China
- Prior art keywords
- gradient
- sample
- group
- gbdt
- passive
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 27
- 238000012549 training Methods 0.000 claims abstract description 38
- 239000011159 matrix material Substances 0.000 claims abstract description 32
- 238000000034 method Methods 0.000 claims abstract description 30
- 238000007477 logistic regression Methods 0.000 claims abstract description 20
- 230000004927 fusion Effects 0.000 claims abstract description 15
- 238000005192 partition Methods 0.000 claims description 41
- 238000004590 computer program Methods 0.000 claims description 12
- 238000010276 construction Methods 0.000 claims description 11
- 230000005540 biological transmission Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 description 9
- 238000007726 management method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000010801 machine learning Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 230000006978 adaptation Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 238000012550 audit Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 108010001267 Protein Subunits Proteins 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003862 health status Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012502 risk assessment Methods 0.000 description 1
- 238000012954 risk control Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Business, Economics & Management (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioethics (AREA)
- Computer Hardware Design (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Computer Security & Cryptography (AREA)
- Accounting & Taxation (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Finance (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a GBDT and LR fusion method, a device, equipment and a storage medium based on federal learning, wherein the method comprises the following steps: calculating the gradient of each first sample, encrypting the gradient and transmitting the gradient to a passive party; acquiring the gradient and the group of each group of the passive side after encryption; decrypting the gradient and the group, selecting optimal feature division according to the gradient and transmitting a division value corresponding to the optimal feature division to a passive party; obtaining a sample space of the passive side divided into a left node or a right node; splitting the first sample according to the sample space to obtain a tree structure corresponding to the GBDT model; and constructing a characteristic matrix according to the tree structure, and performing logistic regression training to obtain an LR model. By the adoption of the GBDT and LR fusion method, the GBDT and LR fusion device and the storage medium based on federal learning, financial data can be directly aggregated to conduct fusion model training of GBDT and LR models.
Description
Technical Field
The application relates to the technical field of model hosting, in particular to a GBDT and LR fusion method, a device, equipment and a storage medium based on federal learning.
Background
In a financial scenario, some wind control models are often constructed, and since the industry needs a model with high interpretability, simple and effective logistic regression is often used for processing classification problems. However, logistic regression is a linear model, and cannot capture nonlinear information, and requires a large amount of feature engineering and consumes manpower and material resources, while GBDT (Gradient Boost Decision Tree) can be used to find differentiated features and feature combinations, thereby reducing the labor cost in feature engineering. But accordingly GBDT is an integrated method and therefore less explanatory. The GBDT and LR (generalized linear model) fusion model just combines the advantages of the GBDT and the LR, and the GBDT is firstly adopted to explore the distinguishing characteristics and the combination characteristics, and then the LR is used to construct a model with high interpretability.
The existing GBDT and LR fusion model is established on the basis of open source data for model training. Nowadays, the financial industry is more and more strictly controlled, and financial data cannot be directly aggregated to perform machine learning model training.
Disclosure of Invention
The application mainly aims to provide a GBDT and LR fusion method, a device, equipment and a storage medium based on federal learning, and aims to solve the technical problem that financial data cannot be directly aggregated to carry out fusion model training of GBDT and LR models.
In order to achieve the above object, the present application provides a GBDT and LR fusion method based on federal learning, which is applied to an active side, and includes the following steps:
calculating the gradient of each first sample, encrypting the gradient and transmitting the gradient to a passive party, wherein the first sample has a label;
acquiring the gradient and the group of the passive side after encryption; the gradient sum group is obtained by calculating the gradient sum of each group after each second sample is grouped according to attributes by the passive party; the first and second swatches correspond to the same user, the second swatch having no label;
decrypting the gradient and the group, selecting the optimal feature partition according to the decrypted gradient and transmitting a partition value corresponding to the optimal feature partition to the passive side;
obtaining a sample space of the passive side divided into a left node or a right node; the sample space is obtained by dividing the second sample according to the division value through the passive party, and the sample space corresponding to a left node or a right node is obtained;
splitting the first sample according to the sample space to obtain a tree structure corresponding to the GBDT model;
and constructing a characteristic matrix according to the tree structure, and carrying out logistic regression training to obtain an LR model.
Further, the step of decrypting the gradient and the group, selecting an optimal feature partition according to the decrypted gradient and transmitting a partition value corresponding to the optimal feature partition to the passive side includes:
decrypting the gradient sum group;
calculating a gain of the first sample from the decrypted gradient sum;
selecting optimal feature division according to the gain;
and transmitting the division value corresponding to the optimal feature division to the passive side.
Further, the step of calculating the gain of the first sample according to the decrypted gradient sum comprises:
by the formula
Calculating a gain of the first sample, wherein the gl、hlLadder split into first samples in left nodeDegree information, said gr、hrThe gradient information of the first sample split into the right node is shown, G and h are the gradient information of the current first sample, and λ is a parameter of formula G.
Further, the step of splitting the first sample according to the sample space to obtain a tree structure corresponding to a GBDT model includes:
according to the sample space, dividing the first samples corresponding to the second samples in the sample space into left nodes;
and dividing the rest of the first samples into right nodes to obtain a tree structure corresponding to the GBDT model.
Further, the step of constructing a feature matrix according to the tree structure and performing logistic regression training to obtain an LR model includes:
performing one-hot coding on leaf nodes in the tree structure;
assigning values to the first sample according to the one-hot codes to construct a sparse matrix;
and taking the sparse matrix as a characteristic matrix, and carrying out logistic regression training to obtain the LR model.
The application also provides another GBDT and LR fusion method based on federal learning, which is applied to a passive side and comprises the following steps:
acquiring age characteristic values of second samples of the passive party;
sorting the second sample according to the age characteristic value;
dividing the second samples according to the sequence and a preset quantile to obtain each group;
calculating the gradient sum of each group to obtain the gradient sum group;
the gradient and the group are encrypted and then transmitted to a master side;
acquiring a division value of the active side; the division value is obtained by decrypting the gradient and the group by the active party and dividing according to the decrypted gradient and the selected optimal characteristic;
dividing the second samples belonging to the division values into left nodes, and dividing the second samples not belonging to the division values into right nodes to obtain sample spaces corresponding to the left nodes or the right nodes;
and transmitting the sample space corresponding to the left node or the right node to the master.
The application also provides a GBDT and LR fusion device based on federal learning, including:
the calculating unit is used for calculating the gradient of each first sample, encrypting the gradient and transmitting the encrypted gradient to a passive party, wherein the first samples have labels;
the first acquisition unit is used for acquiring the encrypted gradient and group of the passive side; the gradient sum group is obtained by calculating the gradient sum of each group after each second sample is grouped according to attributes by the passive party; the first and second swatches correspond to the same user, the second swatch not having a label
The decryption unit is used for decrypting the gradient and the group, selecting the optimal feature partition according to the decrypted gradient and transmitting a partition value corresponding to the optimal feature partition to the passive side;
the second acquisition unit is used for acquiring a sample space of the passive side divided into a left node or a right node; the sample space is obtained by dividing the second sample according to the division value through the passive party, and the sample space corresponding to a left node or a right node is obtained;
the splitting unit is used for splitting the first sample according to the sample space to obtain a tree structure corresponding to the GBDT model;
and the construction unit is used for constructing a characteristic matrix according to the tree structure, and carrying out training of logistic regression to obtain an LR model.
The application also provides another GBDT and LR fusion device based on federal learning, which comprises:
a third obtaining unit, configured to obtain an age characteristic value of each second sample of the passive party;
the sorting unit is used for sorting the second samples according to the age characteristic values;
the grouping unit is used for dividing the second samples according to the sequence and a preset quantile to obtain each group;
the calculating unit is used for calculating the gradient sum of each group to obtain the gradient sum group;
the encryption unit is used for encrypting the gradient and the group and transmitting the encrypted gradient and group to the master side;
a fourth obtaining unit, configured to obtain a division value of the master; the division value is obtained by decrypting the gradient and the group by the active party and dividing according to the decrypted gradient and the selected optimal characteristic;
the dividing unit is used for dividing the second samples belonging to the division values into a left node and dividing the second samples not belonging to the division values into a right node to obtain a sample space corresponding to the left node or the right node;
and the transmission unit is used for transmitting the sample space corresponding to the left node or the right node to the master.
The present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above GBDT and LR fusion methods based on federated learning when executing the computer program.
The present application further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of any of the above-described federate learning based GBDT and LR fusion methods.
According to the GBDT and LR fusion method, the device, the equipment and the storage medium based on the federal learning, a federal model with high interpretability and good effect can be constructed under the condition that model training is not directly carried out on data aggregation. According to the method, when an LR model is trained, a feature matrix is directly constructed according to a tree structure of a GBDT model, and the LR model with high interpretability can be obtained without complicated feature construction. The method only needs to transmit gradient and the like when the GBDT model is constructed, and the construction of the LR model is directly obtained by training on the active side, so that the time efficiency basically depends on the model efficiency of the GBDT, and the time complexity cannot be improved. Meanwhile, the data information of the opposite party is not required to be known between the active party and the passive party, and the respective financial data is not known by other parties, so that the financial data can also be used for machine learning model training.
Drawings
FIG. 1 is a schematic illustration of an implementation environment of an embodiment of the present application;
FIG. 2 is a schematic diagram illustrating steps of a GBDT and LR fusion method based on federated learning according to an embodiment of the present application;
FIG. 3 is a schematic diagram of another exemplary method for fusion of GBDT and LR based on federated learning according to another embodiment of the present application;
FIG. 4 is a block diagram of a GBDT and LR fusion device based on federated learning according to an embodiment of the present application;
FIG. 5 is a block diagram of another exemplary embodiment of a GBDT and LR fusion device based on federated learning;
fig. 6 is a block diagram illustrating a structure of a computer device according to an embodiment of the present application.
The implementation, functional features and advantages of the objectives of the present application will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Referring to fig. 1, under federal study, an active party has a first terminal 1, a passive party has at least one second terminal 2, and the first terminal 1 and the second terminal 2 can perform data communication through a network; the active side and the passive side have the same user, the first terminal 1 has sample data X1 and tag data Y, and the second terminal 2 has sample data X2 and X3 … … XN. The first terminal 1 and the second terminal 2 may each include an independently operating server, or a distributed server, or a server cluster composed of a plurality of servers, where the servers may be cloud servers. Federated learning can build machine learning systems without direct access to sample data, which remains in the original location, helping to ensure privacy and reduce communication costs.
Specifically, the first terminal 1 calculates the gradient of a first sample and transmits the gradient to the second terminal 2 in an encrypted manner, the second terminal 2 groups a second sample and calculates the gradient of the group and forms the gradient and the group to be transmitted to the first terminal 1, the first terminal 1 decrypts the gradient and the group and selects the optimal characteristic according to the decrypted gradient and divides the optimal characteristic, the divided value of the optimal characteristic is transmitted to the second terminal 2, the second terminal 2 divides the second sample according to the division, the divided sample space of the left node or the right node is transmitted to the first terminal 1, the first terminal 1 divides the first sample according to the sample space to obtain a GBDT model, a characteristic matrix is constructed according to the tree structure of the GBDT model, and training of logistic regression is performed to obtain an LR model.
Referring to fig. 2, an embodiment of the present application provides a GBDT and LR fusion method based on federal learning, which is applied to an active side, and includes the following steps:
step S1, firstly, calculating the gradient of each first sample, encrypting the gradient and transmitting the gradient to a passive party, wherein the first samples have labels;
step S2, acquiring the gradient and group of the passive side after encryption; the gradient sum group is obtained by calculating the gradient sum of each group after each second sample is grouped according to attributes by the passive party; the first and second swatches correspond to the same user, the second swatch having no label;
step S3, the gradient and the group are decrypted, and according to the decrypted gradient and the selected optimal characteristic division, the division value corresponding to the optimal characteristic division is transmitted to the passive side;
step S4, obtaining a sample space of the passive side divided into a left node or a right node; the sample space is obtained by dividing the second sample according to the division value through the passive party, and the sample space corresponding to a left node or a right node is obtained;
step S5, splitting the first sample according to the sample space to obtain a tree structure corresponding to the GBDT model;
and step S6, constructing a feature matrix according to the tree structure, and performing logistic regression training to obtain an LR model.
The GBDT and LR fusion method based on federal learning proposed in this embodiment is applied to an active party, where the active party and a passive party have the same batch of users, but the user information of the active party and the passive party is different, for example, a bank a has users: a, B, C; b bank has users A, B, C; in other embodiments, the passive party may have a label, and when the passive party has the label, the label of the active party is taken as the master, and the label of the passive party does not participate in training. In particular, the passive party may include multiple banks. As described in step S1, the gradient of each first sample in the master is calculated, specifically, during the t-th iteration, the calculated gradient is:
wherein h isiIs giThe second derivative of (a); y isiAnd encrypting the gradient to be the ith characteristic of the first sample y and then transmitting the gradient to the passive party, specifically, encrypting the gradient by an addition homomorphic encryption algorithm and then transmitting the gradient to the passive party.
As described in step S2 above, the encrypted gradient and group of each passive packet are obtained. The passive party groups the second samples, specifically, the second samples may be grouped based on some characteristic attributes in the second samples, such as gender, age, and the like, the gradient of the second samples in each group is calculated according to the above equations (1) and (2), the gradient sum of each group is calculated, and the gradient sum of each group is encrypted and then transmitted to the active party. In another embodiment, when training is performed in combination with a multi-party bank, the passive party comprises a plurality of banks, the second sample of each bank is grouped, the gradient sum of the grouping is calculated, then the gradient sum group is formed, and the gradient sum group is encrypted and then transmitted to the active party.
As described in step S3, since the base learner of the GBDT model uses the tree model, after the master decrypts the received gradient sums, the tree model splits according to the decrypted gradient sums, each split node is divided into a left node and a right node, and each first sample falls into a unique leaf node in each tree. And selecting an optimal feature partition, and transmitting a partition value corresponding to the optimal feature partition to a passive side, wherein the optimal feature partition represents an optimal mode of tree model splitting, and the partition value is a hyper parameter, namely a parameter of a set value before training is started, and is not parameter data obtained through training.
As described in step S4, after receiving the feature column and the partition value, the passive side partitions the samples according to the partition value, that is, the samples of the feature column value in the partition value interval are partitioned into the left node and the rest are partitioned into the right node, and returns the sample space partitioned into the left node or the right node to the active side, and the active side obtains the sample space partitioned into the left node or the right node by the passive side.
As described in step S5, after receiving the sample space divided into left nodes, the master can know which second samples fall into the left nodes, so that the first sample can be split equally, the left and right nodes are divided, and finally the corresponding threshold is reached to construct leaf nodes, so as to obtain the tree structure of the GBDT model.
As described in step S6, since the master knows the tree structure of the GBDT model and the sample space that falls into the leaf node, the master assigns a value to the first sample, constructs a sparse matrix, and trains a logistic regression using the sparse matrix as a feature matrix to obtain an LR model.
In this embodiment, the GBDT and LR fusion method based on federal learning can construct a federal model with high interpretability and good effect without model training by directly aggregating data. According to the method, when an LR model is trained, a feature matrix is directly constructed according to a tree structure of a GBDT model, and the LR model with high interpretability can be obtained without complicated feature construction. The method only needs to transmit gradient and the like when the GBDT model is constructed, and the construction of the LR model is directly obtained by training on the active side, so that the time efficiency basically depends on the model efficiency of the GBDT, and the time complexity cannot be improved. Meanwhile, the data information of the opposite party is not required to be known between the active party and the passive party, and the respective financial data is not known by other parties, so that the financial data can also be used for machine learning model training. Specifically, when the method is used for training the wind control model in combination with a multi-party bank, data information of the banks of all parties is not disclosed, so that the method can effectively utilize the multi-party data to complete personal risk assessment as much as possible under the condition of ensuring data safety, and the bank can effectively identify users with poor credit conditions in other banks.
In an embodiment, the step S3 of decrypting the gradient sum, selecting an optimal feature partition according to the gradient sum, and transmitting the feature column and the partition value corresponding to the optimal feature partition to the passive side includes:
decrypting the gradient sum group;
calculating a gain of the first sample from the decrypted gradient sum;
selecting optimal feature division according to the gain;
and transmitting the division value corresponding to the optimal feature division to the passive side.
In this embodiment, the master decrypts the gradient sum groups to obtain the gradient sum of each group, and calculates the gain of the first sample in the master according to the gradient sum.
In an embodiment, the step of calculating the gain of the first sample according to the decrypted gradient sum includes:
by the formula
Calculating a gain of the first sample, wherein the gl、hlFor gradient information split into the first sample in the left node, the gr、hrThe gradient information of the first sample split into the right node is shown, G and h are the gradient information of the current first sample, and λ is a parameter of formula G. In this embodiment, the gain of the first sample is calculated through the above formula, and the value of the gain can represent the splitting quality of the tree model, and when the gain is larger, it indicates that the splitting quality of the tree model is better.
And selecting the optimal characteristic division according to the gain, selecting the corresponding tree model when the gain value is maximum, and transmitting the characteristic column and the division value corresponding to the optimal characteristic division to the passive side so that the passive side knows the splitting of the active side.
In an embodiment, the step S5 of splitting the first sample according to the sample space to obtain a tree structure corresponding to a GBDT model includes:
according to the sample space, dividing the first samples corresponding to the second samples in the sample space into left nodes;
and dividing the rest of the first samples into right nodes to obtain a tree structure corresponding to the GBDT model.
In this embodiment, after knowing the sample space of the passive side divided into the left nodes, the active side divides the first sample corresponding to the second sample at the left node into the left nodes, and divides the remaining first samples into the right nodes, so as to obtain the tree structure of the GBDT model, thereby completing the training of the GBDT model.
In an embodiment, the step S6 of constructing the feature matrix according to the tree structure and performing training of logistic regression to obtain an LR model includes:
performing one-hot coding on leaf nodes in the tree structure;
assigning values to the first sample according to the one-hot codes to construct a sparse matrix;
and taking the sparse matrix as a characteristic matrix, and carrying out logistic regression training to obtain the LR model.
In this embodiment, the GBDT model after training generates a plurality of trees, each first sample falls into a unique leaf node in each tree, each leaf node is regarded as a feature, one-hot encoding is performed on the leaf nodes, the first samples are assigned according to the one-hot encoding, each first sample obtains a corresponding feature vector, all the first samples finally obtain a sparse matrix, each column represents the meaning of the leaf node, and the sparse matrix is put into logistic regression for training to complete construction of the LR model. In this embodiment, no transmission is performed in the construction process of the LR model, so the construction process is very efficient, and the time complexity is not increased.
Referring to fig. 3, an embodiment of the present application provides another GBDT and LR fusion method based on federal learning, which is applied to a passive side, and includes the following steps:
step S10, acquiring age characteristic values of each second sample of the passive side;
step S20, sorting the second sample according to the age characteristic value;
step S30, dividing the second samples according to the sequence and preset quantiles to obtain each group;
step S40, calculating the gradient sum of each group to obtain the gradient sum group;
step S50, the gradient and the group are encrypted and then transmitted to the master side;
step S60, obtaining the division value of the master; the division value is obtained by decrypting the gradient and the group by the active party and dividing according to the decrypted gradient and the selected optimal characteristic;
step S70, dividing the second samples belonging to the division values into left nodes, and dividing the second samples not belonging to the division values into right nodes, to obtain sample spaces corresponding to the left nodes or the right nodes;
step S80, the sample space corresponding to the left node or the right node is transmitted to the master.
The GBDT and LR fusion method based on federal learning proposed in this embodiment is applied to the passive side, and as described in steps S10-S20, the second sample of the passive side has the characteristic of age, and the age characteristic value of the second sample is obtained by the passive side, and the samples are sorted according to the age characteristic value, that is, sorted from small to large according to the age.
As described in step S30, the second samples are grouped according to the sorting, specifically, a quantile is preset, such as a quartile and a quintile, and if a quartile is adopted, the sorted second samples are divided into four equal parts to obtain four groups.
As described in the above steps S40-S50, the gradient sum group is obtained by the passive side summing the gradients of the second samples in each group. Specifically, the gradient sum of each packet is encrypted by an addition homomorphic encryption algorithm and then transmitted to the master. The second samples are grouped according to the age characteristic value through a preset quantile, the number of the second samples in each group is the same, and the proportion of the second samples smaller than a certain value after the second samples are arranged from small to large in the second samples to the total number of the second samples can be directly known through grouping through the quantile.
As described in the above steps S60-S80, after receiving the partition value of the optimal feature partition of the active side, the passive side divides the second sample belonging to the partition value into the left node, divides the second sample not belonging to the partition value into the right node, and transmits the sample space divided into the left node or the right node to the active side, so that the active side can know the sample space of the passive side, and the active side can split the sample space according to half of the sample space of the passive side, thereby obtaining the GBDT model.
The GBDT and LR fusion method based on federal learning provided by the application can be applied to the field of block chains, the trained GBDT model and LR model are stored in a block chain network, and the block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.
The block chain underlying platform can comprise processing modules such as user management, basic service, intelligent contract and operation monitoring. The user management module is responsible for identity information management of all blockchain participants, and comprises public and private key generation maintenance (account management), key management, user real identity and blockchain address corresponding relation maintenance (authority management) and the like, and under the authorization condition, the user management module supervises and audits the transaction condition of certain real identities and provides rule configuration (wind control audit) of risk control; the basic service module is deployed on all block chain node equipment and used for verifying the validity of the service request, recording the service request to storage after consensus on the valid request is completed, for a new service request, the basic service firstly performs interface adaptation analysis and authentication processing (interface adaptation), then encrypts service information (consensus management) through a consensus algorithm, transmits the service information to a shared account (network communication) completely and consistently after encryption, and performs recording and storage; the intelligent contract module is responsible for registering and issuing contracts, triggering the contracts and executing the contracts, developers can define contract logics through a certain programming language, issue the contract logics to a block chain (contract registration), call keys or other event triggering and executing according to the logics of contract clauses, complete the contract logics and simultaneously provide the function of upgrading and canceling the contracts; the operation monitoring module is mainly responsible for deployment, configuration modification, contract setting, cloud adaptation in the product release process and visual output of real-time states in product operation, such as: alarm, monitoring network conditions, monitoring node equipment health status, and the like.
Referring to fig. 4, an embodiment of the present application provides a GBDT and LR fusion device based on federal learning, including:
the calculation unit 10 is configured to calculate a gradient of each first sample, encrypt the gradient, and transmit the encrypted gradient to a passive party, where the first sample has a label;
a first obtaining unit 20, configured to obtain the encrypted gradient and group of the passive side; the gradient sum group is obtained by calculating the gradient sum of each group after each second sample is grouped according to attributes by the passive party; the first and second swatches correspond to the same user, the second swatch not having a label
The decryption unit 30 is configured to decrypt the gradient and the group, select an optimal feature partition according to the decrypted gradient and select the optimal feature partition, and transmit a partition value corresponding to the optimal feature partition to the passive side;
a second obtaining unit 40, configured to obtain a sample space of the passive side divided into a left node or a right node; the sample space is obtained by dividing the second sample according to the division value through the passive party, and the sample space corresponding to a left node or a right node is obtained;
a splitting unit 50, configured to split the first sample according to the sample space to obtain a tree structure corresponding to the GBDT model;
and the constructing unit 60 is configured to construct a feature matrix according to the tree structure, and perform training of logistic regression to obtain an LR model.
In one embodiment, the decryption unit 30 includes:
a decryption subunit, configured to decrypt the gradient sum group;
a second calculating subunit, configured to calculate a gain of the first sample according to the decrypted gradient sum;
the selecting subunit is used for selecting the optimal characteristic division according to the gain;
and the transfer subunit is used for transferring the feature column and the division value corresponding to the optimal feature division to the passive side.
In one embodiment, the second computing subunit includes:
a calculation module for passing the formula
Calculating a gain of the first sample, wherein the gl、hlFor gradient information split into the first sample in the left node, the gr、hrThe gradient information of the first sample split into the right node is shown, G and h are the gradient information of the current first sample, and λ is a parameter of formula G.
In one embodiment, the splitting unit 50 includes:
a third dividing subunit, configured to divide the first sample corresponding to the second sample in the sample space into left nodes according to the sample space;
and the fourth dividing subunit is used for dividing the rest of the first samples into right nodes to obtain the tree structure corresponding to the GBDT model.
In one embodiment, the building unit 60 includes:
the encoding unit is used for performing one-hot encoding on leaf nodes in the tree structure;
an assignment unit, configured to assign a value to the first sample according to the one-hot code, and construct a sparse matrix;
and the training subunit is used for performing logistic regression training by taking the sparse matrix as a characteristic matrix to obtain the LR model.
Referring to fig. 5, an embodiment of the present application provides another GBDT and LR fusion apparatus based on federal learning, including:
a third obtaining unit 1A, configured to obtain an age characteristic value of each second sample of the passive party;
a sorting unit 1B for sorting the second samples according to the age characteristic values;
the grouping unit 1C is configured to divide the second samples according to the sorting and a preset quantile to obtain each group;
a calculating unit 1D, configured to calculate a gradient sum of each group, to obtain the gradient sum group;
the encryption unit 1E is configured to encrypt the gradient sum group and transmit the encrypted gradient sum group to the master;
a fourth obtaining unit 1F, configured to obtain a division value of the master; the division value is obtained by decrypting the gradient and the group by the active party and dividing according to the decrypted gradient and the selected optimal characteristic;
the dividing unit 1G is configured to divide the second sample belonging to the division value into a left node, and divide the second sample not belonging to the division value into a right node, so as to obtain a sample space corresponding to the left node or the right node;
and the transmission unit 1H is configured to transmit the sample space corresponding to the left node or the right node to the master.
In this embodiment, please refer to the above method embodiment for the specific implementation of each unit and sub-unit, which is not described herein again.
Referring to fig. 6, a computer device, which may be a server and whose internal structure may be as shown in fig. 6, is also provided in the embodiment of the present application. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the computer designed processor is used to provide computational and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing first sample data, second sample data and the like. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a federate learning based GBDT and LR fusion method.
Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of some of the structures associated with the present teachings and is not intended to limit the scope of the present teachings as applied to computer devices.
An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for fusion of GBDT and LR based on federal learning is implemented.
In summary, for the GBDT and LR fusion method, apparatus, device, and storage medium based on federal learning provided in this embodiment of the present application, the active side calculates the gradient of each first sample, encrypts the gradient, and transmits the encrypted gradient to the passive side, where the first sample has a tag; acquiring the gradient and the group of the passive side after encryption; the gradient sum group is obtained by calculating the gradient sum of each group after each second sample is grouped according to attributes by the passive party; the first and second swatches correspond to the same user, the second swatch having no label; decrypting the gradient and the group, selecting the optimal feature partition according to the decrypted gradient and transmitting a partition value corresponding to the optimal feature partition to the passive side; obtaining a sample space of the passive side divided into a left node or a right node; the sample space is obtained by dividing the second sample according to the division value through the passive party, and the sample space corresponding to a left node or a right node is obtained; splitting the first sample according to the sample space to obtain a tree structure corresponding to the GBDT model; and constructing a characteristic matrix according to the tree structure, and carrying out logistic regression training to obtain an LR model. The GBDT and LR fusion method based on federal learning can construct a federal model with high interpretability and good effect under the condition of not directly carrying out model training on data aggregation. According to the method, when an LR model is trained, a feature matrix is directly constructed according to a tree structure of a GBDT model, and the LR model with high interpretability can be obtained without complicated feature construction. The method only needs to transmit gradient and the like when the GBDT model is constructed, and the construction of the LR model is directly obtained by training on the active side, so that the time efficiency basically depends on the model efficiency of the GBDT, and the time complexity cannot be improved. Meanwhile, the data information of the opposite party is not required to be known between the active party and the passive party, and the respective financial data is not known by other parties, so that the financial data can also be used for machine learning model training.
It will be understood by those skilled in the art that all or part of the processes of the methods of the above embodiments may be implemented by hardware associated with instructions of a computer program, which may be stored on a non-volatile computer-readable storage medium, and when executed, may include processes of the above embodiments of the methods. Any reference to memory, storage, database, or other medium provided herein and used in the examples may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double-rate SDRAM (SSRSDRAM), Enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.
The above description is only for the preferred embodiment of the present application and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are intended to be included within the scope of the present application.
Claims (10)
1. A GBDT and LR fusion method based on federal learning is applied to an active side and is characterized by comprising the following steps:
calculating the gradient of each first sample, encrypting the gradient and transmitting the gradient to a passive party, wherein the first sample has a label;
acquiring the gradient and the group of the passive side after encryption; the gradient sum group is obtained by calculating the gradient sum of each group after each second sample is grouped according to attributes by the passive party; the first and second swatches correspond to the same user, the second swatch having no label;
decrypting the gradient and the group, selecting the optimal feature partition according to the decrypted gradient and transmitting a partition value corresponding to the optimal feature partition to the passive side;
obtaining a sample space of the passive side divided into a left node or a right node; the sample space is obtained by dividing the second sample according to the division value through the passive party, and the sample space corresponding to a left node or a right node is obtained;
splitting the first sample according to the sample space to obtain a tree structure corresponding to the GBDT model;
and constructing a characteristic matrix according to the tree structure, and carrying out logistic regression training to obtain an LR model.
2. The GBDT and LR fusion method according to claim 1, wherein the step of decrypting the gradient and group, selecting an optimal feature partition according to the decrypted gradient and selecting a partition value corresponding to the optimal feature partition to the passive side includes:
decrypting the gradient sum group;
calculating a gain of the first sample from the decrypted gradient sum;
selecting optimal feature division according to the gain;
and transmitting the division value corresponding to the optimal feature division to the passive side.
3. The method of claim 2, wherein the step of calculating the gain of the first sample based on the decrypted gradient sum comprises:
by the formula
Calculating a gain of the first sample, wherein the gl、hlFor gradient information split into the first sample in the left node, the gr、hrThe gradient information of the first sample split into the right node is shown, G and h are the gradient information of the current first sample, and λ is a parameter of formula G.
4. The method according to claim 1, wherein the step of splitting the first sample according to the sample space to obtain a tree structure corresponding to a GBDT model comprises:
according to the sample space, dividing the first samples corresponding to the second samples in the sample space into left nodes;
and dividing the rest of the first samples into right nodes to obtain a tree structure corresponding to the GBDT model.
5. The GBDT and LR fusion method according to claim 1, wherein the step of constructing a feature matrix from the tree structure and performing logistic regression training to obtain an LR model comprises:
performing one-hot coding on leaf nodes in the tree structure;
assigning values to the first sample according to the one-hot codes to construct a sparse matrix;
and taking the sparse matrix as a characteristic matrix, and carrying out logistic regression training to obtain the LR model.
6. A GBDT and LR fusion method based on federal learning is applied to a passive side and is characterized by comprising the following steps:
acquiring age characteristic values of second samples of the passive party;
sorting the second sample according to the age characteristic value;
dividing the second samples according to the sequence and a preset quantile to obtain each group;
calculating the gradient sum of each group to obtain the gradient sum group;
the gradient and the group are encrypted and then transmitted to a master side;
acquiring a division value of the active side; the division value is obtained by decrypting the gradient and the group by the active party and dividing according to the decrypted gradient and the selected optimal characteristic;
dividing the second samples belonging to the division values into left nodes, and dividing the second samples not belonging to the division values into right nodes to obtain sample spaces corresponding to the left nodes or the right nodes;
and transmitting the sample space corresponding to the left node or the right node to the master.
7. A GBDT and LR fusion device based on federal learning, comprising:
the calculating unit is used for calculating the gradient of each first sample, encrypting the gradient and transmitting the encrypted gradient to a passive party, wherein the first samples have labels;
the first acquisition unit is used for acquiring the encrypted gradient and group of the passive side; the gradient sum group is obtained by calculating the gradient sum of each group after each second sample is grouped according to attributes by the passive party; the first and second swatches correspond to the same user, the second swatch not having a label
The decryption unit is used for decrypting the gradient and the group, selecting the optimal feature partition according to the decrypted gradient and transmitting a partition value corresponding to the optimal feature partition to the passive side;
the second acquisition unit is used for acquiring a sample space of the passive side divided into a left node or a right node; the sample space is obtained by dividing the second sample according to the division value through the passive party, and the sample space corresponding to a left node or a right node is obtained;
the splitting unit is used for splitting the first sample according to the sample space to obtain a tree structure corresponding to the GBDT model;
and the construction unit is used for constructing a characteristic matrix according to the tree structure, and carrying out training of logistic regression to obtain an LR model.
8. A GBDT and LR fusion device based on federal learning, comprising:
a third obtaining unit, configured to obtain an age characteristic value of each second sample of the passive party;
the sorting unit is used for sorting the second samples according to the age characteristic values;
the grouping unit is used for dividing the second samples according to the sequence and a preset quantile to obtain each group;
the calculating unit is used for calculating the gradient sum of each group to obtain the gradient sum group;
the encryption unit is used for encrypting the gradient and the group and transmitting the encrypted gradient and group to the master side;
a fourth obtaining unit, configured to obtain a division value of the master; the division value is obtained by decrypting the gradient and the group by the active party and dividing according to the decrypted gradient and the selected optimal characteristic;
the dividing unit is used for dividing the second samples belonging to the division values into a left node and dividing the second samples not belonging to the division values into a right node to obtain a sample space corresponding to the left node or the right node;
and the transmission unit is used for transmitting the sample space corresponding to the left node or the right node to the master.
9. A computer device comprising a memory and a processor, the memory having a computer program stored therein, wherein the processor when executing the computer program performs the steps of the federal learning based GBDT and LR fusion method of any of claims 1 to 5 or 6.
10. A computer readable storage medium having stored thereon a computer program for implementing the steps of the federate learning based GBDT and LR fusion method of any one of claims 1 to 5 or 6 when executed by a processor.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011182203.9A CN112288101A (en) | 2020-10-29 | 2020-10-29 | GBDT and LR fusion method, device, equipment and storage medium based on federal learning |
| PCT/CN2021/084670 WO2022088606A1 (en) | 2020-10-29 | 2021-03-31 | Gbdt and lr fusion method and apparatus based on federated learning, device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011182203.9A CN112288101A (en) | 2020-10-29 | 2020-10-29 | GBDT and LR fusion method, device, equipment and storage medium based on federal learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112288101A true CN112288101A (en) | 2021-01-29 |
Family
ID=74353220
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011182203.9A Pending CN112288101A (en) | 2020-10-29 | 2020-10-29 | GBDT and LR fusion method, device, equipment and storage medium based on federal learning |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112288101A (en) |
| WO (1) | WO2022088606A1 (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113408668A (en) * | 2021-07-30 | 2021-09-17 | 深圳前海微众银行股份有限公司 | Decision tree construction method and device based on federated learning system and electronic equipment |
| CN113435537A (en) * | 2021-07-16 | 2021-09-24 | 同盾控股有限公司 | Cross-feature federated learning method and prediction method based on Soft GBDT |
| CN113516253A (en) * | 2021-07-02 | 2021-10-19 | 深圳市洞见智慧科技有限公司 | Data encryption optimization method and device in federated learning |
| CN113657614A (en) * | 2021-09-02 | 2021-11-16 | 京东科技信息技术有限公司 | Method and device for updating federal learning model |
| CN113689948A (en) * | 2021-08-18 | 2021-11-23 | 深圳先进技术研究院 | Man-machine asynchronous detection method and device for mechanical ventilation of breathing machine and related equipment |
| WO2022088606A1 (en) * | 2020-10-29 | 2022-05-05 | 平安科技（深圳）有限公司 | Gbdt and lr fusion method and apparatus based on federated learning, device, and storage medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117196776A (en) * | 2023-09-09 | 2023-12-08 | 广东德澳智慧医疗科技有限公司 | Cross-border electronic commerce product credit marking and after-sale system based on random gradient lifting tree algorithm |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180089587A1 (en) * | 2016-09-26 | 2018-03-29 | Google Inc. | Systems and Methods for Communication Efficient Distributed Mean Estimation |
| CN109299728B (en) * | 2018-08-10 | 2023-06-27 | 深圳前海微众银行股份有限公司 | Sample joint prediction method, system and medium based on construction of gradient tree model |
| CN110210233B (en) * | 2019-04-19 | 2024-05-24 | 平安科技（深圳）有限公司 | Combined construction method and device of prediction model, storage medium and computer equipment |
| CN111738359B (en) * | 2020-07-24 | 2020-11-27 | 支付宝(杭州)信息技术有限公司 | Two-party decision tree training method and system |
| CN112288101A (en) * | 2020-10-29 | 2021-01-29 | 平安科技（深圳）有限公司 | GBDT and LR fusion method, device, equipment and storage medium based on federal learning |
-
2020
- 2020-10-29 CN CN202011182203.9A patent/CN112288101A/en active Pending
-
2021
- 2021-03-31 WO PCT/CN2021/084670 patent/WO2022088606A1/en active Application Filing
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022088606A1 (en) * | 2020-10-29 | 2022-05-05 | 平安科技（深圳）有限公司 | Gbdt and lr fusion method and apparatus based on federated learning, device, and storage medium |
| CN113516253A (en) * | 2021-07-02 | 2021-10-19 | 深圳市洞见智慧科技有限公司 | Data encryption optimization method and device in federated learning |
| CN113516253B (en) * | 2021-07-02 | 2022-04-05 | 深圳市洞见智慧科技有限公司 | Data encryption optimization method and device in federated learning |
| CN113435537A (en) * | 2021-07-16 | 2021-09-24 | 同盾控股有限公司 | Cross-feature federated learning method and prediction method based on Soft GBDT |
| CN113408668A (en) * | 2021-07-30 | 2021-09-17 | 深圳前海微众银行股份有限公司 | Decision tree construction method and device based on federated learning system and electronic equipment |
| CN113689948A (en) * | 2021-08-18 | 2021-11-23 | 深圳先进技术研究院 | Man-machine asynchronous detection method and device for mechanical ventilation of breathing machine and related equipment |
| CN113657614A (en) * | 2021-09-02 | 2021-11-16 | 京东科技信息技术有限公司 | Method and device for updating federal learning model |
| CN113657614B (en) * | 2021-09-02 | 2024-03-01 | 京东科技信息技术有限公司 | Updating method and device of federal learning model |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022088606A1 (en) | 2022-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112288101A (en) | GBDT and LR fusion method, device, equipment and storage medium based on federal learning | |
| CN111897673B (en) | Operation and maintenance fault root cause identification method and device, computer equipment and storage medium | |
| Weng et al. | Wolverine: fast, scalable, and communication-efficient zero-knowledge proofs for boolean and arithmetic circuits | |
| US20210409191A1 (en) | Secure Machine Learning Analytics Using Homomorphic Encryption | |
| Zhang et al. | Secure and efficient outsourcing of PCA-based face recognition | |
| CN111598186A (en) | Decision model training method, prediction method and device based on longitudinal federal learning | |
| CN111784001B (en) | Model training method and device and computer readable storage medium | |
| CN110084377A (en) | Method and apparatus for constructing decision tree | |
| CN110601815B (en) | Block chain data processing method and equipment | |
| CN113449048B (en) | Data label distribution determining method and device, computer equipment and storage medium | |
| CN113362048B (en) | Data label distribution determining method and device, computer equipment and storage medium | |
| CN113221153B (en) | Graph neural network training method and device, computing equipment and storage medium | |
| CN111311211A (en) | Data processing method and device based on block chain | |
| CN113378148A (en) | Internet of things equipment identity authentication system and method based on block chain | |
| CN111901554B (en) | Call channel construction method and device based on semantic clustering and computer equipment | |
| CN114760054A (en) | Key management method and device based on digital wallet and storage medium | |
| CN112364059A (en) | Correlation matching method, device, equipment and storage medium under multi-rule scene | |
| CN115276969A (en) | Wireless channel key generation method and device, computer equipment and storage medium | |
| CN106452790A (en) | Multi-party quantum digital signature method without trusted center | |
| Tallapally et al. | Competent multi-level encryption methods for implementing cloud security | |
| Dhanaraj et al. | Probit cryptographic blockchain for secure data transmission in intelligent transportation systems | |
| Yang et al. | Accountable and verifiable secure aggregation for federated learning in IoT networks | |
| CN111343273B (en) | Attribute-based strategy hiding outsourcing signcryption method in Internet of vehicles environment | |
| CN115941351A (en) | Trusted privacy computing system based on cloud service and encryption technology | |
| Wang et al. | Blockchain-Enabled Lightweight Fine-Grained Searchable Knowledge Sharing for Intelligent IoT |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210129 |