CN112766490B - Feature variable learning method, device, equipment and computer readable storage medium - Google Patents
Feature variable learning method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN112766490B CN112766490B CN202110045805.8A CN202110045805A CN112766490B CN 112766490 B CN112766490 B CN 112766490B CN 202110045805 A CN202110045805 A CN 202110045805A CN 112766490 B CN112766490 B CN 112766490B
- Authority
- CN
- China
- Prior art keywords
- modeling
- variable
- learning
- different
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000006870 function Effects 0.000 claims abstract description 52
- 238000003062 neural network model Methods 0.000 claims abstract description 23
- 230000003993 interaction Effects 0.000 claims description 30
- 238000005457 optimization Methods 0.000 claims description 13
- 238000000586 desensitisation Methods 0.000 claims description 12
- 238000005070 sampling Methods 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 7
- 230000000694 effects Effects 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 abstract description 6
- 238000004891 communication Methods 0.000 description 5
- 238000012549 training Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000013526 transfer learning Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012954 risk control Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/03—Credit; Loans; Processing thereof
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Accounting & Taxation (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Finance (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Marketing (AREA)
- Development Economics (AREA)
- General Business, Economics & Management (AREA)
- Economics (AREA)
- Technology Law (AREA)
- Strategic Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Machine Translation (AREA)
Abstract
The invention relates to the technical field of financial science and technology (Fintech). The invention discloses a feature variable learning method, a device, equipment and a computer readable storage medium, wherein modeling feature variable sets corresponding to different modeling tasks of different types of guest groups are firstly obtained and used as input of the same target model, so that modeling for different guest groups and different modeling tasks is not needed; and then building a target model based on a neural network model and a multi-task learning algorithm, and learning a unified input variable expression, namely a final objective function, for different modeling tasks in a manner of sharing hidden layer parameters so as to achieve an optimal effect on the different modeling tasks as far as possible, thereby preventing model parameters from being excessively fitted with a biased guest group distribution, and further avoiding the conditions that the model is difficult to maintain, modeling intermediate information is recycled, or the model cannot be built and the like caused by separate modeling in the conventional manner.
Description
Technical Field
The present invention relates to the technical field of financial science and technology (Fintech), and in particular, to a feature variable learning method, device, apparatus, and computer-readable storage medium.
Background
With the development of computer technology, more and more technologies (big data, distributed, blockchain Blockchain, artificial intelligence, etc.) are applied in the financial field, and the traditional financial industry is gradually changing to financial technology (Fintech), but due to the requirements of security and real-time performance of the financial industry, higher requirements are also put forward on the technologies. For the credit risk modeling field, since a customer often has a plurality of different stages in a credit service, such as account opening, first borrowing, stock, overdue, etc., the existing modeling manner generally builds a plurality of models according to different customer groups of different service stages, but the plurality of models built based on the manner are difficult to maintain due to the large number, and may not even build the plurality of models successfully in some situations. Therefore, the above cases reflect the technical problem that the implementation difficulty of the existing credit risk modeling method is large.
Disclosure of Invention
The invention mainly aims to provide a characteristic variable learning method, a device, equipment and a computer readable storage medium, and aims to solve the technical problem that an existing credit risk modeling mode is difficult to realize.
In order to achieve the above object, the present invention provides a feature variable learning method, including:
Obtaining modeling characteristic variable sets corresponding to modeling sample clients in a plurality of different business stages, wherein the modeling sample clients are obtained based on a plurality of different guest groups, and the modeling characteristic variable sets correspond to a plurality of different modeling tasks;
Building a target model based on a neural network model and a multi-task learning algorithm architecture, and taking the modeling characteristic variable set as input of the target model so as to perform parameter sharing on the modeling characteristic variable set in the target model to obtain a parameter sharing result;
and carrying out parallel learning on the parameter sharing result aiming at a plurality of modeling tasks so as to learn and obtain an objective function, wherein the objective function is used for uniformly representing the mapping relation between the modeling characteristic variable set and the modeling tasks.
Optionally, the step of building a target model based on a neural network model and a multi-task learning algorithm architecture, and taking the modeling feature variable set as an input of the target model to perform parameter sharing on the modeling feature variable set in the target model to obtain a parameter sharing result includes:
Constructing a model comprising an input variable layer, a shared hidden layer, a shared output layer and a task output layer by using a neural network model architecture and a multi-task learning algorithm to serve as the target model;
And inputting the modeling characteristic variable set into an input variable layer of the target model, so that the modeling characteristic variable set is transmitted into the sharing hidden layer from the input variable layer, and depth interaction among variables is performed in the sharing hidden layer, so that the parameter sharing result is obtained.
Optionally, the step of transferring the modeling feature variable set from the input variable layer to the shared hidden layer and performing deep interaction between variables in the shared hidden layer to obtain the parameter sharing result includes:
transmitting the modeling characteristic variable set from the input variable layer to the shared hidden layer so that the modeling characteristic variable set learns multiple groups of general variable interaction information in the shared hidden layer;
and transmitting a plurality of groups of the universal variable interaction information into the sharing output layer so as to collect a plurality of groups of the universal variable interaction information into the parameter sharing result at the sharing output layer.
Optionally, the step of learning the parameter sharing result in parallel for a plurality of modeling tasks to obtain an objective function includes:
the parameter sharing result is transmitted to the task output layer, and a gradient optimization direction is learned at the task output layer based on a plurality of modeling tasks;
And optimizing the parameter sharing result by using the gradient optimization direction so as to obtain the objective function based on the optimized parameter sharing result.
Optionally, the step of obtaining the modeling feature variable set corresponding to the modeling sample clients in the several different service phases includes:
acquiring client information and business stage information corresponding to the client information, and dividing the client information into a plurality of different client groups according to the business stage information;
Sampling the modeling sample clients from a plurality of different guest groups, and obtaining modeling characteristic variable sets corresponding to the modeling sample clients.
Optionally, the step of obtaining the client information and the service stage information corresponding to the client information, and performing the client group division on the client information according to the service stage information to obtain a plurality of different client groups includes:
Acquiring the client information and the business stage information, and performing desensitization processing on the client information to obtain desensitized client information;
And carrying out guest group division on the desensitized client information by utilizing the business stage information to obtain a plurality of different guest groups.
Optionally, after the step of learning the objective function by parallel learning the parameter sharing result for a plurality of modeling tasks, the method further includes:
and determining a modeling task to be predicted in a plurality of modeling tasks, and predicting and outputting a target output result of the modeling task to be predicted based on the target function.
In addition, in order to achieve the above object, the present invention also provides a feature variable learning device including:
the system comprises a feature variable acquisition module, a modeling module and a modeling module, wherein the feature variable acquisition module is used for acquiring modeling feature variable sets corresponding to modeling sample clients in a plurality of different business stages, the modeling sample clients are obtained based on a plurality of different guest groups, and the modeling feature variable sets correspond to a plurality of different modeling tasks;
The model parameter sharing module is used for constructing a target model based on a neural network model and a multi-task learning algorithm architecture, taking the modeling characteristic variable set as the input of the target model, and carrying out parameter sharing on the modeling characteristic variable set in the target model to obtain a parameter sharing result;
And the objective function learning module is used for carrying out parallel learning on the parameter sharing result aiming at a plurality of modeling tasks so as to learn and obtain an objective function, wherein the objective function is used for uniformly representing the mapping relation between the modeling characteristic variable set and the modeling tasks.
Optionally, the model parameter sharing module includes:
the target model construction unit is used for constructing a model comprising an input variable layer, a shared hidden layer, a shared output layer and a task output layer by utilizing a neural network model architecture and a multi-task learning algorithm to serve as the target model;
And the variable depth interaction unit is used for inputting the modeling characteristic variable set into an input variable layer of the target model so as to transfer the modeling characteristic variable set from the input variable layer into the sharing hidden layer, and carrying out depth interaction among variables in the sharing hidden layer to obtain the parameter sharing result.
Optionally, the variable depth interaction unit includes:
transmitting the modeling characteristic variable set from the input variable layer to the shared hidden layer so that the modeling characteristic variable set learns multiple groups of general variable interaction information in the shared hidden layer;
and transmitting a plurality of groups of the universal variable interaction information into the sharing output layer so as to collect a plurality of groups of the universal variable interaction information into the parameter sharing result at the sharing output layer.
Optionally, the objective function learning module includes:
the optimization direction learning unit is used for transmitting the parameter sharing result to the task output layer, and learning the gradient optimization direction based on a plurality of modeling tasks at the task output layer;
And the objective function acquisition unit is used for optimizing the parameter sharing result by utilizing the gradient optimization direction so as to obtain the objective function based on the optimized parameter sharing result.
Optionally, the feature variable obtaining module includes:
the business guest group dividing unit is used for acquiring customer information and business stage information corresponding to the customer information, and carrying out guest group division on the customer information according to the business stage information so as to obtain a plurality of different guest groups;
The feature variable acquisition unit is used for sampling the modeling sample clients from a plurality of different guest groups and acquiring modeling feature variable sets corresponding to the modeling sample clients.
Optionally, the service group dividing unit includes:
the customer information desensitizing unit is used for acquiring the customer information and the business stage information and carrying out desensitization processing on the customer information to obtain desensitized customer information;
and the desensitization information dividing unit is used for dividing the customer groups of the desensitization customer information by utilizing the business stage information to obtain a plurality of different customer groups.
Optionally, the feature variable learning device further includes:
And the target result output unit is used for determining modeling tasks to be predicted in a plurality of modeling tasks, predicting and outputting target output results of the modeling tasks to be predicted based on the target function.
In addition, in order to achieve the above object, the present invention also provides a feature variable learning apparatus including: the system comprises a memory, a processor and a characteristic variable learning program stored on the memory and capable of running on the processor, wherein the characteristic variable learning program realizes the steps of the characteristic variable learning method when being executed by the processor.
In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a feature variable learning program which, when executed by a processor, implements the steps of the feature variable learning method described above.
The invention provides a feature variable learning method, a device, equipment and a computer readable storage medium. According to the invention, modeling characteristic variable sets corresponding to different modeling tasks of different types of guest groups are acquired first and are used as the input of the same target model, so that modeling for different guest groups and different modeling tasks is not needed; and then building a target model based on a neural network model and a multi-task learning algorithm, learning a unified input variable expression, namely a final objective function, for different modeling tasks in a manner of sharing hidden layer parameters so as to achieve an optimal effect on the different modeling tasks as far as possible, and preventing model parameters from being excessively fitted with a biased guest group distribution, thereby avoiding the conditions that the model is difficult to maintain, modeling intermediate information is recycled or the model cannot be built and the like caused by separate modeling in the existing manner, and solving the technical problem that the implementation difficulty of the existing credit risk modeling manner is high.
Drawings
FIG. 1 is a schematic diagram of a device architecture of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart of a first embodiment of a feature variable learning method according to the present invention;
FIG. 3 is a schematic diagram of a model of a second embodiment of the feature variable learning method of the present invention;
fig. 4 is a schematic diagram of functional modules of the feature variable learning device of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic device structure of a hardware running environment according to an embodiment of the present invention.
As shown in fig. 1, the feature variable learning apparatus may include: a processor 1001, such as a CPU, a user interface 1003, a network interface 1004, a memory 1005, a communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
It will be appreciated by those skilled in the art that the device structure shown in fig. 1 is not limiting of the device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a feature variable learning program may be included in a memory 1005 as one type of computer storage medium.
In the device shown in fig. 1, the network interface 1004 is mainly used for connecting to a background server, and performing data communication with the background server; the user interface 1003 is mainly used for connecting a client (programmer end) and communicating data with the client; and the processor 1001 may be configured to call a feature variable learning program stored in the memory 1005 and perform operations in the following feature variable learning method:
Obtaining modeling characteristic variable sets corresponding to modeling sample clients in a plurality of different business stages, wherein the modeling sample clients are obtained based on a plurality of different guest groups, and the modeling characteristic variable sets correspond to a plurality of different modeling tasks;
Building a target model based on a neural network model and a multi-task learning algorithm architecture, and taking the modeling characteristic variable set as input of the target model so as to perform parameter sharing on the modeling characteristic variable set in the target model to obtain a parameter sharing result;
and carrying out parallel learning on the parameter sharing result aiming at a plurality of modeling tasks so as to learn and obtain an objective function, wherein the objective function is used for uniformly representing the mapping relation between the modeling characteristic variable set and the modeling tasks.
Further, the step of constructing a target model based on the neural network model and the multi-task learning algorithm architecture, taking the modeling characteristic variable set as an input of the target model, and performing parameter sharing on the modeling characteristic variable set in the target model to obtain a parameter sharing result includes:
Constructing a model comprising an input variable layer, a shared hidden layer, a shared output layer and a task output layer by using a neural network model architecture and a multi-task learning algorithm to serve as the target model;
And inputting the modeling characteristic variable set into an input variable layer of the target model, so that the modeling characteristic variable set is transmitted into the sharing hidden layer from the input variable layer, and depth interaction among variables is performed in the sharing hidden layer, so that the parameter sharing result is obtained.
Further, the step of transmitting the modeling feature variable set from the input variable layer to the shared hidden layer and performing deep interaction between variables in the shared hidden layer to obtain the parameter sharing result includes:
transmitting the modeling characteristic variable set from the input variable layer to the shared hidden layer so that the modeling characteristic variable set learns multiple groups of general variable interaction information in the shared hidden layer;
and transmitting a plurality of groups of the universal variable interaction information into the sharing output layer so as to collect a plurality of groups of the universal variable interaction information into the parameter sharing result at the sharing output layer.
Further, the step of learning the parameter sharing result in parallel for a plurality of modeling tasks to obtain an objective function includes:
the parameter sharing result is transmitted to the task output layer, and a gradient optimization direction is learned at the task output layer based on a plurality of modeling tasks;
And optimizing the parameter sharing result by using the gradient optimization direction so as to obtain the objective function based on the optimized parameter sharing result.
Further, the step of obtaining the modeling feature variable set corresponding to the modeling sample clients in the different service stages includes:
acquiring client information and business stage information corresponding to the client information, and dividing the client information into a plurality of different client groups according to the business stage information;
Sampling the modeling sample clients from a plurality of different guest groups, and obtaining modeling characteristic variable sets corresponding to the modeling sample clients.
Further, the step of obtaining the client information and the service stage information corresponding to the client information, and performing the client group division on the client information according to the service stage information to obtain a plurality of different client groups includes:
Acquiring the client information and the business stage information, and performing desensitization processing on the client information to obtain desensitized client information;
And carrying out guest group division on the desensitized client information by utilizing the business stage information to obtain a plurality of different guest groups.
Further, after the step of learning the parameter sharing result in parallel for a plurality of modeling tasks to obtain an objective function, the processor 1001 may be configured to invoke a feature variable learning program stored in the memory 1005, and perform operations in the following feature variable learning method:
and determining a modeling task to be predicted in a plurality of modeling tasks, and predicting and outputting a target output result of the modeling task to be predicted based on the target function.
Based on the hardware structure, the embodiment of the characteristic variable learning method is provided.
In order to solve the problems, the invention provides a feature variable learning method, namely, modeling feature variable sets corresponding to different modeling tasks of different types of guest groups are firstly obtained and used as input of the same target model, so that modeling for different guest groups and different modeling tasks is not needed; and then building a target model based on a neural network model and a multi-task learning algorithm, learning a unified input variable expression, namely a final objective function, for different modeling tasks in a manner of sharing hidden layer parameters so as to achieve an optimal effect on the different modeling tasks as far as possible, and preventing model parameters from being excessively fitted with a biased guest group distribution, thereby avoiding the conditions that the model is difficult to maintain, modeling intermediate information is recycled or the model cannot be built and the like caused by separate modeling in the existing manner, and solving the technical problem that the implementation difficulty of the existing credit risk modeling manner is high.
Referring to fig. 2, fig. 2 is a flowchart of a first embodiment of the feature variable learning method according to the present invention. The characteristic variable learning method comprises the following steps of;
Step S10, a modeling characteristic variable set corresponding to a modeling sample client in a plurality of different business stages is obtained, wherein the modeling sample client is obtained based on a plurality of different client groups, and the modeling characteristic variable set corresponds to a plurality of different modeling tasks;
In this embodiment, the present invention is applied to a terminal device. The method is mainly suitable for application scenes of credit risk modeling. For personal credit business, credit risk modeling combines various factors causing the personal to generate default, and utilizes a mathematical model method to identify the risk of the personal to generate default and uses the risk in the whole risk control process. The business stage refers to different stages of the credit business of the client, such as account opening, first borrowing, stock, overdue and the like, and different stages can divide different guest groups and correspond to different risk modeling tasks. Modeling sample clients refer to client objects for which modeling feature variables are to be acquired at this time. The modeling feature variable set contains a plurality of modeling feature variables, which refer to behavior information of a modeling sample client in a digital form before a viewpoint, and is generally represented by x. The viewpoint refers to the point in time when the customer begins to represent risk. A guest group refers to a group of clients that have several common characteristics that are screened out by certain screening criteria. Modeling tasks refer to whether clients become target clients during their performance period at different business stages, and are generally denoted by y, and specific values may be marked by a target client of 1 and a non-target client of 0.
Specifically, if the terminal receives a feature variable learning instruction, determining a modeling sample client needing to acquire modeling feature variables according to the feature variable learning instruction, acquiring modeling feature variables of the modeling sample client in a certain time period from each appointed relevant platform, and collecting the data into the modeling feature variable set.
Step S20, a target model is built based on a neural network model and a multi-task learning algorithm architecture, and the modeling characteristic variable set is used as input of the target model so as to perform parameter sharing on the modeling characteristic variable set in the target model to obtain a parameter sharing result;
in this embodiment, the neural network model refers to a machine learning algorithm, and the model is composed of an input layer, a hidden layer and an output layer, each layer includes a plurality of neurons, the neurons between adjacent layers are connected by weighted edges, the forward calculation is a prediction process, and the backward calculation is a training process. The multi-task learning algorithm is a kind of transfer learning algorithm, and transfer learning can be understood as defining a source domain and a target domain, learning in the source domain, transferring the learned knowledge to the target domain, and improving the learning effect of the target domain. The primary goal of multitasking is to increase generalization capability using domain-specific information implicit in the training signals of multiple related tasks, which is accomplished by training multiple tasks in parallel using a shared representation. The parameter sharing result refers to variable interaction information and parameters which are obtained by learning at a hidden layer of the model and are common to different guest groups and different modeling tasks.
The terminal builds a neural network model sharing parameters, namely the target model, then uniformly takes all modeling characteristic variables in the modeling characteristic variable set as the input of the target model, performs deep interaction among model parameters in the target model by utilizing a multi-target learning algorithm, and learns the model parameters depending on the gradient returned by the upper layer to obtain the parameter sharing result.
And step S30, carrying out parallel learning on the parameter sharing result aiming at a plurality of modeling tasks so as to learn and obtain an objective function, wherein the objective function is used for uniformly representing the mapping relation between the modeling characteristic variable set and the modeling tasks.
In this embodiment, the objective function is used to uniformly characterize the mapping relationship between the modeling feature variable set and the modeling tasks, specifically, the unified feature variable x expression for multiple different modeling tasks y.
And carrying out parameter learning sharing on the feature variables corresponding to different modeling tasks at a hidden layer of the target model, further learning for a plurality of different modeling tasks after obtaining a parameter sharing result, obtaining a loss function finally obtained by weighting and summing target loss functions respectively corresponding to different modeling tasks after learning is completed, and finally obtaining the target function based on the loss function. In the prediction stage after learning is completed, one or more modeling tasks which need to be predicted currently can be selected to be accessed into a target model, and the model can output the prediction results of the modeling tasks which need to be predicted.
The invention provides a characteristic variable learning method. The feature variable learning method comprises the steps of obtaining modeling feature variable sets corresponding to modeling sample clients in a plurality of different business stages, wherein the modeling sample clients are obtained based on a plurality of different guest groups, and the modeling feature variable sets correspond to a plurality of different modeling tasks; building a target model based on a neural network model and a multi-task learning algorithm architecture, and taking the modeling characteristic variable set as input of the target model so as to perform parameter sharing on the modeling characteristic variable set in the target model to obtain a parameter sharing result; and carrying out parallel learning on the parameter sharing result aiming at a plurality of modeling tasks so as to learn and obtain an objective function, wherein the objective function is used for uniformly representing the mapping relation between the modeling characteristic variable set and the modeling tasks. By the method, modeling characteristic variable sets corresponding to different modeling tasks of different types of guest groups are acquired first and used as input of the same target model, so that modeling for different guest groups and different modeling tasks is not needed; and then building a target model based on a neural network model and a multi-task learning algorithm, learning a unified input variable expression, namely a final objective function, for different modeling tasks in a manner of sharing hidden layer parameters so as to achieve an optimal effect on the different modeling tasks as far as possible, and preventing model parameters from being excessively fitted with a biased guest group distribution, thereby avoiding the conditions that the model is difficult to maintain, modeling intermediate information is recycled or the model cannot be built and the like caused by separate modeling in the existing manner, and solving the technical problem that the implementation difficulty of the existing credit risk modeling manner is high.
Further, based on the first embodiment shown in fig. 2 described above, a second embodiment of the feature variable learning method of the present invention is proposed. In this embodiment, step S20 includes:
Constructing a model comprising an input variable layer, a shared hidden layer, a shared output layer and a task output layer by using a neural network model architecture and a multi-task learning algorithm to serve as the target model;
And inputting the modeling characteristic variable set into an input variable layer of the target model, so that the modeling characteristic variable set is transmitted into the sharing hidden layer from the input variable layer, and depth interaction among variables is performed in the sharing hidden layer, so that the parameter sharing result is obtained.
In this embodiment, as shown in fig. 3. The target model architecture comprises an input variable layer, a shared hidden layer, a shared output layer and a task output layer. It should be noted that the two layers of the shared hidden layer in the figure do not represent the actual number of shared hidden layers. The input variable layer is used for inputting all modeling characteristic variables in the modeling characteristic variable set as input of the input variable layer in the target model by the input variable information terminal, then the modeling characteristic variable set is transmitted into the sharing hidden layer from the input variable layer, the depth interaction between variables is carried out in the sharing hidden layer, and the model parameters are learned by depending on the gradient returned by the upper layer, so that the parameter sharing result is obtained.
Further, the step of transmitting the modeling feature variable set from the input variable layer to the shared hidden layer and performing deep interaction between variables in the shared hidden layer to obtain the parameter sharing result includes:
transmitting the modeling characteristic variable set from the input variable layer to the shared hidden layer so that the modeling characteristic variable set learns multiple groups of general variable interaction information in the shared hidden layer;
and transmitting a plurality of groups of the universal variable interaction information into the sharing output layer so as to collect a plurality of groups of the universal variable interaction information into the parameter sharing result at the sharing output layer.
In this embodiment, after all modeling feature variables in the modeling feature variable set are transmitted from the input variable layer to the shared hidden layer, modeling feature variables corresponding to different modeling tasks learn multiple groups of general variable interaction information in the shared hidden layer, and then are transmitted to the shared output layer for summarization, and the summarized result is used as the parameter sharing result.
Further, step S30 includes:
the parameter sharing result is transmitted to the task output layer, and a gradient optimization direction is learned at the task output layer based on a plurality of modeling tasks;
And optimizing the parameter sharing result by using the gradient optimization direction so as to obtain the objective function based on the optimized parameter sharing result.
In this embodiment, as shown in fig. 3, the task output layer accesses a plurality of different modeling tasks (e.g., task object 1 to task object n in fig. 3) based on the parameter sharing result of the shared output layer, so as to learn further about the modeling tasks, and at the same time, the gradient value between the output result and the modeling tasks is transferred downwards by the loss function at this layer, where the loss function is obtained by weighting and summing the target loss functions of the modeling tasks, and the weight is generally set by manual experience. The loss function is used to measure the degree of difference between the model f (x) and the modeling task y, and its derivative is used as the gradient direction of the parameter optimization. In the learning stage in the task output layer, different target output results and training targets are input into the loss function to obtain a gradient optimization direction, and meanwhile, partial shared parameters of the lower layer are optimized to obtain a final target function.
Further, in this embodiment, a neural network model with shared parameters is constructed for different guest groups and business targets in different business stages, the parameters are shared, the output is shared, and the parameter learning is affected by accessing multiple targets, so that the model parameters can reach a better effect on each target as much as possible, and the model parameters are prevented from being excessively fitted to a biased guest group distribution.
Further, based on the first embodiment shown in fig. 2 described above, a third embodiment of the feature variable learning method of the present invention is proposed. In this embodiment, step S10 includes:
acquiring client information and business stage information corresponding to the client information, and dividing the client information into a plurality of different client groups according to the business stage information;
Sampling the modeling sample clients from a plurality of different guest groups, and obtaining modeling characteristic variable sets corresponding to the modeling sample clients.
In this embodiment, the service stage information refers to identification information of different stages where the client is in the credit service, such as account opening, first borrowing, stock, overdue, etc., and different groups may be divided in different stages. The terminal classifies the whole existing client sources by taking different business stages as classification basis, and divides a plurality of client groups. And then selecting part of clients from a plurality of client groups through sample sampling to serve as modeling sample clients, and finally correspondingly acquiring modeling characteristic variables of the modeling sample clients on a designated platform to serve as the modeling characteristic variable set. The specific sampling mode may be random sampling or other sampling modes, and the embodiment is not limited in particular.
Further, the step of obtaining the client information and the service stage information corresponding to the client information, and performing the client group division on the client information according to the service stage information to obtain a plurality of different client groups includes:
Acquiring the client information and the business stage information, and performing desensitization processing on the client information to obtain desensitized client information;
And carrying out guest group division on the desensitized client information by utilizing the business stage information to obtain a plurality of different guest groups.
In this embodiment, data desensitization refers to the deformation of data of some sensitive information through a desensitization rule, so as to realize reliable protection of sensitive privacy data. Under the condition of involving client security data or some commercial sensitive data, under the condition of not violating system rules, the real data is modified and tested, and personal information such as an identity card number, a mobile phone number, a card number, a client number and the like needs to be subjected to data desensitization. Because the client information may contain sensitive data, the terminal needs to perform desensitization processing on the client information after acquiring the client information to obtain desensitized client information, and then performs client group division.
Further, after step S30, the method further includes:
and determining a modeling task to be predicted in a plurality of modeling tasks, and predicting and outputting a target output result of the modeling task to be predicted based on the target function.
In this embodiment, in the prediction stage after learning is completed, the user may select one or more target output nodes according to the requirement, and after receiving the currently determined modeling task to be predicted (i.e., the business target), the terminal may output a prediction result for the user to apply, so that the scheme is more convenient and flexible.
As shown in fig. 4, the present invention also provides a feature variable learning device, including:
The feature variable obtaining module 10 is configured to obtain a modeling feature variable set corresponding to a modeling sample client in a plurality of different service phases, where the modeling sample client is obtained based on a plurality of different client groups, and the modeling feature variable set corresponds to a plurality of different modeling tasks;
The model parameter sharing module 20 is configured to build a target model based on a neural network model and a multi-task learning algorithm architecture, and take the modeling feature variable set as an input of the target model, so as to perform parameter sharing on the modeling feature variable set in the target model to obtain a parameter sharing result;
And the objective function learning module 30 is configured to learn the parameter sharing result in parallel for a plurality of modeling tasks to learn to obtain an objective function, where the objective function is used to uniformly characterize a mapping relationship between the modeling feature variable set and the plurality of modeling tasks.
The method executed by each program module may refer to each embodiment of the feature variable learning method of the present invention, and will not be described herein.
The invention also provides a characteristic variable learning device.
The characteristic variable learning device comprises a processor, a memory and a characteristic variable learning program stored on the memory and capable of running on the processor, wherein the characteristic variable learning program realizes the steps of the characteristic variable learning method when being executed by the processor.
The method implemented when the feature variable learning program is executed may refer to various embodiments of the feature variable learning method of the present invention, which will not be described herein.
The invention also provides a computer readable storage medium.
The computer-readable storage medium of the present invention stores thereon a feature variable learning program which, when executed by a processor, implements the steps of the feature variable learning method as described above.
The method implemented when the feature variable learning program is executed may refer to various embodiments of the feature variable learning method of the present invention, which is not described herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (8)
1. A feature variable learning method, characterized in that the feature variable learning method comprises:
Obtaining modeling characteristic variable sets corresponding to modeling sample clients in a plurality of different business stages, wherein the modeling sample clients are obtained based on a plurality of different guest groups, and the modeling characteristic variable sets correspond to a plurality of different modeling tasks;
Building a target model based on a neural network model and a multi-task learning algorithm architecture, and taking the modeling characteristic variable set as input of the target model so as to perform parameter sharing on the modeling characteristic variable set in the target model to obtain a parameter sharing result;
Parallel learning is carried out on the parameter sharing result aiming at a plurality of modeling tasks so as to learn to obtain an objective function, wherein the objective function is used for uniformly representing the mapping relation between the modeling characteristic variable set and the modeling tasks;
The step of obtaining modeling characteristic variable sets corresponding to modeling sample clients in a plurality of different service stages comprises the following steps:
acquiring client information and business stage information corresponding to the client information, and dividing the client information into a plurality of different client groups according to the business stage information;
Sampling the modeling sample clients from a plurality of different guest groups, and obtaining modeling characteristic variable sets corresponding to the modeling sample clients;
The step of obtaining the client information and the business stage information corresponding to the client information, and dividing the client information into a plurality of different client groups according to the business stage information comprises the following steps:
Acquiring the client information and the business stage information, and performing desensitization processing on the client information to obtain desensitized client information;
And carrying out guest group division on the desensitized client information by utilizing the business stage information to obtain a plurality of different guest groups.
2. The method for learning feature variables according to claim 1, wherein the step of constructing a target model based on a neural network model and a multi-task learning algorithm architecture, taking the modeling feature variable set as an input of the target model, and performing parameter sharing on the modeling feature variable set in the target model to obtain a parameter sharing result comprises:
Constructing a model comprising an input variable layer, a shared hidden layer, a shared output layer and a task output layer by using a neural network model architecture and a multi-task learning algorithm to serve as the target model;
And inputting the modeling characteristic variable set into an input variable layer of the target model, so that the modeling characteristic variable set is transmitted into the sharing hidden layer from the input variable layer, and depth interaction among variables is performed in the sharing hidden layer, so that the parameter sharing result is obtained.
3. The feature variable learning method of claim 2, wherein the step of transferring the modeled feature variable set from the input variable layer to the shared hidden layer and performing deep interaction between variables in the shared hidden layer to obtain the parameter sharing result comprises:
transmitting the modeling characteristic variable set from the input variable layer to the shared hidden layer so that the modeling characteristic variable set learns multiple groups of general variable interaction information in the shared hidden layer;
and transmitting a plurality of groups of the universal variable interaction information into the sharing output layer so as to collect a plurality of groups of the universal variable interaction information into the parameter sharing result at the sharing output layer.
4. The feature variable learning method of claim 2, wherein the step of learning the parameter sharing result in parallel for a plurality of modeling tasks to learn to obtain an objective function includes:
the parameter sharing result is transmitted to the task output layer, and a gradient optimization direction is learned at the task output layer based on a plurality of modeling tasks;
And optimizing the parameter sharing result by using the gradient optimization direction so as to obtain the objective function based on the optimized parameter sharing result.
5. The method for learning feature variables according to any one of claims 1 to 4, wherein after the step of learning the parameter sharing result in parallel for a plurality of modeling tasks to obtain an objective function, further comprising:
and determining a modeling task to be predicted in a plurality of modeling tasks, and predicting and outputting a target output result of the modeling task to be predicted based on the target function.
6. A feature variable learning device, characterized by comprising:
the system comprises a feature variable acquisition module, a modeling module and a modeling module, wherein the feature variable acquisition module is used for acquiring modeling feature variable sets corresponding to modeling sample clients in a plurality of different business stages, the modeling sample clients are obtained based on a plurality of different guest groups, and the modeling feature variable sets correspond to a plurality of different modeling tasks;
The model parameter sharing module is used for constructing a target model based on a neural network model and a multi-task learning algorithm architecture, taking the modeling characteristic variable set as the input of the target model, and carrying out parameter sharing on the modeling characteristic variable set in the target model to obtain a parameter sharing result;
The objective function learning module is used for carrying out parallel learning on the parameter sharing result aiming at a plurality of modeling tasks so as to learn and obtain an objective function, wherein the objective function is used for uniformly representing the mapping relation between the modeling characteristic variable set and the modeling tasks;
The characteristic variable acquisition module is also used for acquiring client information and business stage information corresponding to the client information, and carrying out guest group division on the client information according to the business stage information so as to obtain a plurality of different guest groups; sampling the modeling sample clients from a plurality of different guest groups, and obtaining modeling characteristic variable sets corresponding to the modeling sample clients;
the characteristic variable acquisition module is also used for acquiring the client information and the business stage information, and performing desensitization processing on the client information to obtain desensitized client information; and carrying out guest group division on the desensitized client information by utilizing the business stage information to obtain a plurality of different guest groups.
7. A characteristic variable learning apparatus, characterized by comprising: memory, a processor and a feature variable learning program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the feature variable learning method of any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a feature variable learning program which, when executed by a processor, implements the steps of the feature variable learning method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110045805.8A CN112766490B (en) | 2021-01-13 | 2021-01-13 | Feature variable learning method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110045805.8A CN112766490B (en) | 2021-01-13 | 2021-01-13 | Feature variable learning method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112766490A CN112766490A (en) | 2021-05-07 |
| CN112766490B true CN112766490B (en) | 2024-05-24 |
Family
ID=75700461
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110045805.8A Active CN112766490B (en) | 2021-01-13 | 2021-01-13 | Feature variable learning method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112766490B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110414624A (en) * | 2019-08-06 | 2019-11-05 | 广东工业大学 | Disaggregated model construction method and device based on multi-task learning |
| CN110443364A (en) * | 2019-06-21 | 2019-11-12 | 深圳大学 | A kind of deep neural network multitask hyperparameter optimization method and device |
| CN110598842A (en) * | 2019-07-17 | 2019-12-20 | 深圳大学 | Deep neural network hyper-parameter optimization method, electronic device and storage medium |
| CN110796166A (en) * | 2019-09-25 | 2020-02-14 | 浙江大学 | Attention mechanism-based multitask image processing method |
| CN111325357A (en) * | 2020-03-03 | 2020-06-23 | 深圳前海微众银行股份有限公司 | Machine learning modeling method, device, equipment and computer readable storage medium |
| CN112001484A (en) * | 2020-08-22 | 2020-11-27 | 哈尔滨工业大学 | Safety defect report prediction method based on multitask deep learning |
| CN112163676A (en) * | 2020-10-13 | 2021-01-01 | 北京百度网讯科技有限公司 | Multitask service prediction model training method, device, equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11657322B2 (en) * | 2018-08-30 | 2023-05-23 | Nec Corporation | Method and system for scalable multi-task learning with convex clustering |
| US12008459B2 (en) * | 2019-04-19 | 2024-06-11 | Microsoft Technology Licensing, Llc | Multi-task machine learning architectures and training procedures |
-
2021
- 2021-01-13 CN CN202110045805.8A patent/CN112766490B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110443364A (en) * | 2019-06-21 | 2019-11-12 | 深圳大学 | A kind of deep neural network multitask hyperparameter optimization method and device |
| CN110598842A (en) * | 2019-07-17 | 2019-12-20 | 深圳大学 | Deep neural network hyper-parameter optimization method, electronic device and storage medium |
| CN110414624A (en) * | 2019-08-06 | 2019-11-05 | 广东工业大学 | Disaggregated model construction method and device based on multi-task learning |
| CN110796166A (en) * | 2019-09-25 | 2020-02-14 | 浙江大学 | Attention mechanism-based multitask image processing method |
| CN111325357A (en) * | 2020-03-03 | 2020-06-23 | 深圳前海微众银行股份有限公司 | Machine learning modeling method, device, equipment and computer readable storage medium |
| CN112001484A (en) * | 2020-08-22 | 2020-11-27 | 哈尔滨工业大学 | Safety defect report prediction method based on multitask deep learning |
| CN112163676A (en) * | 2020-10-13 | 2021-01-01 | 北京百度网讯科技有限公司 | Multitask service prediction model training method, device, equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| Geometry Embedded Multi-Task Network for Lane Detection and Classification;Zhendong Zhao 等;《Journal of Physics: Conference Series》;20200823;第1-13页 * |
| 可扩展机器学习的并行与分布式优化算法综述;亢良伊;王建飞;刘杰;叶丹;;软件学报;20171009(01);第113-134页 * |
| 多阶段并发的协同物流网络任务-资源集成调配模型;徐小峰;刘靖;;运筹与管理;20180725(07);第47-52页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112766490A (en) | 2021-05-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111310814A (en) | Method and device for training business prediction model by utilizing unbalanced positive and negative samples | |
| CN111461226A (en) | Countermeasure sample generation method, device, terminal and readable storage medium | |
| CN109766454A (en) | A kind of investor's classification method, device, equipment and medium | |
| CN112633962B (en) | Service recommendation method and device, computer equipment and storage medium | |
| AU2019201382A1 (en) | Determining optimal workforce types to fulfill occupational roles in an organization based on occupational attributes | |
| CN111582500A (en) | Method and system for improving model training effect | |
| CN110222838B (en) | Document sorting method and device, electronic equipment and storage medium | |
| CN112785005B (en) | Multi-objective task assistant decision-making method and device, computer equipment and medium | |
| Barry-Straume et al. | An evaluation of training size impact on validation accuracy for optimized convolutional neural networks | |
| CN110110233A (en) | Information processing method, device, medium and calculating equipment | |
| CN112819024B (en) | Model processing method, user data processing method and device and computer equipment | |
| C. Coelho et al. | MASE-BDI: agent-based simulator for environmental land change with efficient and parallel auto-tuning | |
| CN114091554A (en) | Training set processing method and device | |
| CN110969441A (en) | Anti-fraud model processing method and device based on deep learning | |
| CN111814117A (en) | Model interpretation method, device and readable storage medium | |
| CN112328646A (en) | Multitask course recommendation method and device, computer equipment and storage medium | |
| CN116257363A (en) | Resource scheduling method, device, equipment and storage medium | |
| CN110263136B (en) | Method and device for pushing object to user based on reinforcement learning model | |
| CN115130573A (en) | Data processing method, device, storage medium, equipment and product | |
| CN115130536A (en) | Training method of feature extraction model, data processing method, device and equipment | |
| CN117253287B (en) | Action prediction model training method based on domain generalization, related method and product | |
| CN112766490B (en) | Feature variable learning method, device, equipment and computer readable storage medium | |
| CN106997472B (en) | User feature classification method of user credit model, user credit evaluation method and device | |
| CN112070205A (en) | Multi-loss model obtaining method and device | |
| CN110889635A (en) | Method for performing emergency drilling on food safety event processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |