CN110955828B - Multi-factor embedded personalized package recommendation method based on deep neural network - Google Patents
Multi-factor embedded personalized package recommendation method based on deep neural network Download PDFInfo
- Publication number
- CN110955828B CN110955828B CN201911134269.8A CN201911134269A CN110955828B CN 110955828 B CN110955828 B CN 110955828B CN 201911134269 A CN201911134269 A CN 201911134269A CN 110955828 B CN110955828 B CN 110955828B
- Authority
- CN
- China
- Prior art keywords
- package
- user
- time sequence
- attribute
- call
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Telephonic Communication Services (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a multi-factor embedded personalized package recommendation method based on a deep neural network, which specifically comprises the following steps: firstly, data acquisition and pretreatment are carried out; extracting natural attribute characteristic representation of a user by using a full connection layer; secondly, extracting time sequence characteristic representations of user call, flow use and telephone charge use behaviors by utilizing an LSTM network, and splicing the time sequence characteristic representations to serve as user characteristic representations; extracting package attribute characteristic representation by using a full connection layer; the user characteristic representation and the package characteristic representation are spliced and then sent to a fully-connected network, and the probability of handling packages by the user is predicted; and calculating the probability of the target user handling all packages, and selecting M packages with the highest handling probability as a package recommendation list of the user. The method for realizing multi-factor embedded personalized package recommendation based on the deep neural network is beneficial to the targeted package recommendation of operators facing users, thereby being beneficial to the expansion of market scale and the popularization of 5G business.
Description
Technical Field
The invention relates to the technical field of recommendation systems, in particular to a multi-factor embedded personalized package recommendation method based on a deep neural network.
Background
In recent years, with the rapid increase in the amount of mobile services and the number of users, the competition of customers among telecom operators is also getting stronger. To attract users, telecom operators have released rich packages to meet the diversified user needs. It is a challenging task for the operator how to quickly and efficiently market packages to truly demanding users. The user handles packages under a variety of factors, such as: how to utilize these complex factors presents a great challenge to package recommendation, such as the user's revenue, consumption habits, conversation behavior, traffic usage behavior, telephone fee usage behavior, etc.
Existing data-driven package recommendation methods can be divided into two categories: first, package recommendation methods based on user behavior rely on the similarity between users, packages and packages to recommend packages to users that other users with similar behavior transact, or packages that are similar to packages transacted by users. Such methods ignore attributes of the user and package and generally do not provide a valid recommendation. In the second category, content-based package recommendation methods generally mine user preferences according to attributes (such as price, call duration, flow, short messages, etc.) of packages handled by users, and further recommend packages that conform to the user preferences. However, such methods ignore the user's own attributes and behavioral characteristics (e.g., call, traffic usage behavior similarity characteristics) and fail to model the dynamics of the user's preferences.
Disclosure of Invention
The invention aims to overcome the defects, and aims to provide a multi-factor embedded personalized package recommendation method based on a deep neural network, which aims to solve the technical problem of how to fully mine package attributes and user attributes (including user natural attributes, call, flow use and telephone charge use behavior data) for personalized package recommendation; the invention constructs user characteristic representation by using the user natural attribute, the call behavior data, the flow use behavior data and the telephone charge use behavior data, constructs package characteristic representation by using the package attribute, further digs the implicit relativity between the package and the user, and further constructs a personalized package recommendation model with high generalization capability. Firstly, collecting package attributes, related user natural attributes, call behavior data, flow use behavior data, telephone charge use behavior data and package handling records, and preprocessing; extracting user natural attribute characteristic representation by using a full connection layer, extracting user call, flow use and telephone charge use behavior time sequence characteristic representation by using an LSTM network, and splicing the user natural attribute characteristic representation and the user call, flow use and telephone charge use behavior time sequence characteristic representation to serve as user characteristic representation; extracting package attribute characteristic representation by using the full connection layer; then, the user characteristic representation and the package characteristic representation are spliced and then sent into a fully-connected network, and the probability of handling packages by the user is predicted; and finally, calculating the probability of transacting all packages by the target user, and selecting M packages with the highest transacting probability as a package recommendation list of the user. The method for realizing multi-factor embedded personalized package recommendation based on the deep neural network is beneficial to the targeted package recommendation of operators facing users, thereby being beneficial to the expansion of market scale and the popularization of 5G business.
The invention achieves the aim through the following technical scheme: a multi-factor embedded personalized package recommendation method based on a deep neural network comprises three stages of data acquisition and preprocessing, model training and package recommendation, and specifically comprises the following steps:
(1) Data acquisition and preprocessing: collecting package attributes, user natural attributes and package handling records, and preprocessing; collecting daily call behavior data, daily flow usage behavior data and monthly telephone charge usage behavior data of a user, and carrying out missing value completion and outlier elimination treatment; constructing a call behavior time sequence and a flow using behavior time sequence, performing coarsening processing and normalization processing, constructing a telephone charge using behavior time sequence and performing normalization processing;
(2) Model training stage: extracting natural attribute characteristic representation of a user by using a full connection layer, constructing two multi-layer LSTM networks with the same structure, respectively inputting call behavior time sequence and flow use behavior time sequence into the two LSTM networks, and extracting call behavior time sequence characteristics and flow use behavior time sequence characteristics; constructing a multilayer LSTM network, inputting the telephone charge using behavior time sequence into the LSTM network, and extracting the telephone charge using behavior time sequence characteristics; splicing the user natural attribute characteristic representation with the call, flow use and telephone charge use behavior time sequence characteristic representation to serve as a user characteristic representation; extracting package attribute feature representation by using a full connection layer, splicing the user feature representation and the package feature representation, and then sending the package attribute feature representation and the package feature representation into a full connection network to predict the probability of handling packages by a user;
(3) Package recommendation phase: and calculating the probability of the target user handling all packages, and selecting M packages with the highest handling probability as a package recommendation list of the user.
Preferably, the data acquisition and preprocessing stage specifically comprises the following steps:
(1.1) collecting relevant package attributes, comprising: package price, subsidy amount, monthly promised consumption amount, pre-stored telephone charge and telephone charge identification, pre-stored telephone charge and physical identification, call duration, short message, common flow and directional flow;
(1.2) coding the package attribute to obtain a package attribute code l;
(1.3) collecting relevant user natural attributes, including: gender, age, time of network access, client star class, whether to use fusion package, whether to be a family network user, family number, use terminal price, CDMA number under the same client and App use preference;
(1.4) coding the natural attribute of the user to obtain a natural attribute code v of the user;
(1.5) collecting related user call behavior data, flow use behavior data and telephone charge use behavior data, and carrying out missing value complementation and abnormal value elimination treatment; the user call behavior data comprise daily calling times, daily calling time, daily calling times and daily calling time; the flow use behavior data comprise daily flow use times, daily flow use duration, daily uplink flow and daily downlink flow; the telephone charge using behavior data comprise a total monthly telephone charge amount, a monthly extra voice charge, a monthly extra flow charge and a monthly extra short message charge;
(1.6) constructing a call behavior time sequence xi with a time interval of 1 day and a span of T days according to the call and flow use behavior data of the user after the completion of the missing value and the elimination of the abnormal value call And traffic usage behavior timing ζ data And coarsening with particle size gNormalizing; constructing telephone fee use behavior time sequence xi with time interval of 1 month and span of p months cost And carrying out normalization treatment;
(1.7) collecting a package handling record of the user; wherein the package transacting record is represented in the form of triples < u, c, t >, where u is the user id, c is the package id, and t is the date on which the user u transacts package c.
Preferably, the method for obtaining the package attribute code l comprises the following steps:
(1.2.1) direct one-hot encoding of discrete attributes;
(1.2.2) performing maximum and minimum normalization processing on the continuous attribute, and normalizing the processed attribute value to [0,1]]X is the original attribute value, x max For the maximum value of the attribute, x min For the minimum value of this attribute, the formula is as follows:
(1.2.3) splicing all package attribute codes to obtain a package attribute code l.
Preferably, the method for obtaining the user natural attribute code v is as follows:
(1.4.1) direct one-time thermal encoding of discrete attributes;
(1.4.2) dividing the continuous attribute except age into 5 sections through an equal frequency division box, and then performing single-heat coding;
(1.4.3) for age attributes, dividing the age into 8 intervals, and then performing one-hot encoding; wherein 8 intervals are under 16 years old, 16 to 21 years old, 22 to 27 years old, 28 to 33 years old, 34 to 39 years old, 40 to 45 years old, 46 to 51 years old and over 51 years old, respectively;
and (1.4.4) splicing the single-hot codes of all the user natural attributes to obtain the user natural attribute codes v.
Preferably, in the step (1.5), missing values and abnormal values are detected for user call, traffic usage and call charge usage behavior data, and the missing values and abnormal values are filled by using a linear interpolation method.
Preferably, the specific steps of the roughening and normalizing treatment are as follows:
(1.6.1) performing zero-mean normalization processing on each sequence, so that the processed data are normalized to the range of [0,1], wherein the formula is as follows:
wherein x is an original numerical value, mu is the mean value of the sequence in which the numerical value is located, and sigma is the standard deviation of the sequence in which the numerical value is located;
(1.6.2) calculating an average value of the behavior sequence with the time span of T days every g days, and coarsening the sequence with the original step length of T to the step length of T/g; due to the telephone charge use behavior time sequence xi cost The time span of (2) is 1 month, and coarsening is not required.
Preferably, the model training stage specifically comprises:
(2.1) constructing a training data set, and batching the training data set according to a fixed batch size, wherein the total number is N;
(2.2) sequentially selecting a set of training samples with index i from the training dataset, wherein i e {0,1,., N }; repeating steps (2.3) -step (2.10) for each training sample in the batch;
(2.3) taking the sample s in the training data set as a positive sample, randomly sampling k other packages, and replacing packages in the sample s to obtain k negative samples s' 1 ，s′ 2 ，…，s′ k ；
(2.4) mapping the user natural attribute code v to the feature space to obtain a user natural attribute feature representation f attr The method comprises the steps of carrying out a first treatment on the surface of the Wherein, the user natural attribute code v is mapped to the feature space by using a linear transformation mode, and the formula is as follows:
f attr ＝W a T v
wherein W is a Is a mapping matrix;
(2.5) building two identical-structure multilayer LSTM networks, wherein each layerComprises T/g LSTM units; coarsening and normalizing call behavior time sequence xi call And traffic usage behavior timing ζ data Respectively inputting into two LSTM networks, extracting the time sequence characteristic f of the call behavior call And traffic usage behavior timing feature f data ；
(2.6) constructing a multilayer LSTM network, wherein each layer comprises p LSTM units, and the normalized telephone charge use behavior time sequence xi cost Inputting into LSTM network, extracting time sequence characteristic f of telephone charge using behavior cost ；
(2.7) representing the user's natural attribute characteristics by f attr Time sequence characteristic f of call behavior call Flow usage behavior time sequence feature f data And telephone charge use behavior time sequence characteristic f cost Splicing to obtain a user characteristic representation f;
(2.8) mapping the package attribute codes l in the positive and negative samples to feature spaces respectively to obtain a package attribute feature representation r; wherein, the package attribute code/is mapped to the feature space by using a linear transformation mode, and the formula is as follows:
r＝W t T l
wherein W is t For mapping matrix, l ε { l pos ，l′ 1 ，l′ 2 ，…，l′ k }；
(2.9) splicing the user characteristic representation f and the package attribute characteristic representation r in the positive and negative samples, then sending the spliced user characteristic representation f and the package attribute characteristic representation r into a fully-connected network, and outputting the probability of handling packages by the user;
(2.10) calculating a loss functionWherein logarithmic loss with negative sampling is used as a loss function +.>The calculation formula is as follows:
wherein h is θ (. Cndot.) represents the probability values of model outputs, s and s' i Positive and negative samples, respectively;
(2.11) according to loss of all samples in the batchAdjusting network parameters in the whole model;
(2.12) repeating steps (2.2) -step (2.11) until all batches of the training dataset have participated in model training;
(2.13) repeating steps (2.2) -step (2.12) until a specified number of iterations is reached.
Preferably, the step (2.1) specifically includes: each package handling record is constructed into a training sample, and the format of each sample is as follows: s= [ v, ζ ] call ，ξ data ，ξ cost ；l pos ]All training samples form a training data set; the training data set is batched according to the batch size M manually set by experience, and the total number of the batches is N; the specific calculation mode is as follows:
wherein N is Samples For the total number of samples in the training dataset,is a top-rounding function.
Preferably, in the step (2.5), three layers of LSTM networks are adopted to extract time sequence characteristics of user call and traffic usage; the LSTM network is a cyclic neural network, and each LSTM unit comprises a memory unit c t And three gates: input gate i t Output gate o t And forget door f t Respectively controlling the input, output and update of data; in x t For input at time t, h t-1 And c t-1 For the hidden state and the memory cell state at the previous moment, the calculation formula is as follows:
i t ＝sigm(W xi x t +W hi h t-1 +b i )
f t ＝sigm(W xf x t +W hf h t-1 +b f )
o t ＝sigm(W xo x t +W ho h t-1 +W co c t-1 +b o )
wherein the operatorRepresenting the point multiplication operation, W and b represent the weight matrix and the bias vector, respectively, and sigma and tanh represent the sigmoid function and the hyperbolic tangent function, respectively.
Preferably, in step (2.11), the loss of all samples in the batchThe calculation formula of (2) is as follows:
wherein, thereinFor the loss of the mth sample in the batch, M is the number of samples in each batch; according to the loss->The network parameters in the whole model are adjusted, and the updating formula is as follows:
wherein eta is the learning rate and theta is all the learnable parameters of the model.
Preferably, the package recommendation stage specifically includes:
the natural attribute codes after the pretreatment of the target user are spliced with the attribute codes of all packages in sequence, and the probability of handling each package by the target user is calculated by sending the natural attribute codes into a trained model;
and (3.2) selecting M packages with highest handling probability as a package recommendation list of the user.
The invention has the beneficial effects that: (1) The invention effectively utilizes the user characteristics (including natural user attributes, call, flow use and telephone charge use behavior data) and package attributes, thereby fully excavating the invisible correlation between the user and the package, and leading the model to have stronger performance and higher generalization capability; (2) The LSTM network is utilized to extract the time sequence characteristics of user call, flow use and telephone charge use behaviors, the depth characteristics are automatically obtained in a data driving mode, the change trend of modeling user behavior sequences in a modeling mode can be displayed, and the problem that the user behavior characteristics cannot be fully utilized in the existing package recommendation method is solved.
Drawings
FIG. 1 is a schematic flow diagram of the method of the present invention;
FIG. 2 is a schematic diagram of a model framework of an embodiment of the present invention;
FIG. 3 is a schematic diagram of call and traffic usage behavior sequence feature extraction according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of feature fusion of an embodiment of the present invention.
Detailed Description
The invention will be further described with reference to the following specific examples, but the scope of the invention is not limited thereto:
examples: as shown in fig. 1, a multi-factor embedded personalized package recommendation method based on a deep neural network is divided into three stages of data acquisition and preprocessing, model training and package recommendation, wherein a model framework diagram is shown in fig. 2, and is specifically as follows:
(1) A data acquisition and processing stage; the specific steps of data acquisition and preprocessing are as follows:
step 1, collecting related package attributes, including: package price, subsidy amount, monthly promised consumption amount, pre-charge identification, pre-charge physical identification, call duration (minutes), short message (bar), normal flow (MB) and directional flow (MB).
And step 2, coding the package attribute to obtain a package attribute code l.
The specific steps for constructing the package attribute codes comprise:
a) And directly performing single-heat coding on discrete attributes (such as a pre-charge identification and a pre-charge physical identification).
b) Maximum and minimum normalization processing is carried out on continuous attributes (such as package price and call duration), so that the processed attribute values are normalized to [0,1]]X is the original attribute value, x max For the maximum value of the attribute, x min For the minimum value of this attribute, the formula is as follows:
c) And splicing all package attribute codes to obtain a package attribute code l.
Step 3, collecting relevant user natural attributes, including: gender, age, time of network entry (month), client star rating, whether to use converged packages, whether to be a family network user, number of families, price (element) of terminal to use, number of CDMA to be downloaded with clients, and App usage preference (4 categories of social, video, game and reading).
And 4, coding the natural attribute of the user to obtain a natural attribute code v of the user.
The specific steps for constructing the user natural attribute codes comprise:
a) The discrete attributes (e.g., gender, client star, app usage preference) are directly unithermally encoded.
b) And dividing continuous attributes (such as network access time length, CDMA number under the same client and using terminal price) except age into 5 sections through an equal frequency division box, and then performing independent heat coding.
c) For the age attribute, considering that the preference of the user for packages is different for different age groups, the ages are divided into 8 sections (under 16 years old, 16 to 21 years old, 22 to 27 years old, 28 to 33 years old, 34 to 39 years old, 40 to 45 years old, 46 to 51 years old and over 51 years old) and then the independent heat codes are performed.
d) And splicing the single-hot codes of all the user natural attributes to obtain the user natural attribute codes v.
And step 5, collecting relevant user call behavior data (comprising daily calling times, daily calling time, daily calling times and daily calling time), flow use behavior data (comprising daily flow use times, daily flow use time, daily uplink flow and daily downlink flow) and telephone charge use behavior data (comprising monthly telephone charge total amount, monthly extra voice charge, monthly extra flow charge and monthly extra short message charge), and carrying out missing value completion and outlier elimination processing.
And detecting missing values and abnormal values of the call, the flow use and the telephone charge use behavior data of the user, and filling the missing values and the abnormal values by using a linear interpolation method.
Step 6, constructing a call behavior time sequence xi with a time interval of 1 day and a span of T days according to the user call and flow use behavior data after the missing value completion and the abnormal value elimination call And traffic usage behavior timing ζ data Coarsening and normalizing the mixture with the granularity of g; constructing telephone fee use behavior time sequence xi with time interval of 1 month and span of p months cost And carrying out normalization processing.
The specific steps of roughening and normalizing treatment include:
a) Zero mean normalization processing is carried out on each sequence, so that the processed data are normalized to the range of [0,1], and the formula is as follows:
where x is the original value, μ is the mean of the sequence in which the value is located, and σ is the standard deviation of the sequence in which the value is located.
b) In model training, sequences with too many steps can significantly increase the complexity of model computation. The average value is calculated every g days for a behavior sequence with a time span of T days, and the sequence with the original step length of T is coarsened to the step length of T/g. Due to the telephone charge use behavior time sequence xi cost The time span of (2) is 1 month, the total step length is short, and coarsening is not needed.
And 7, collecting package handling records of the user.
The package transaction record is represented in the form of triples < u, c, t >, where u is the user id, c is the package id, and t is the date on which user u transacts package c.
(2) Model training stage; the specific steps of the model training stage are as follows:
step 1, constructing a training data set, and batching the training data set according to a fixed batch size, wherein the total batch number is N.
Each package handling record is constructed into a training sample, and the format of each sample is as follows: s= [ v, ζ ] call ，ξ data ，ξ cost ；l pos ]All training samples constitute a training dataset. The training data set is batched according to an empirically artificially set batch size M, with the total number of batches being N. The specific calculation mode is as follows:
wherein N is Samples For the total number of samples in the training dataset,is a top-rounding function.
Step 2, a batch of training samples with index i is sequentially selected from the training dataset, wherein i is {0,1,... Steps 3-10 are repeated for each training sample in the batch.
Step 3, taking a sample s in the training data set as a positive sample, randomly sampling k other packages, and replacing the packages in the s with the positive samples to obtain k negative samples s' 1 ，s′ 2 ，…，s′ k 。
S= [ v, ζ ] in training data set call ，ξ data ，ξ cost ；l pos ]For positive samples, randomly sampling k other packages, and coding the corresponding package attribute of the k other packages into l' 1 ，l′ 2 ，…，l′ k Respectively replacing package attribute codes I in s pos Obtaining k negative samples s' 1 ，s′ 2 ，…，s′ k 。
Step 4, mapping the user natural attribute codes v to the feature space to obtain the user natural attribute feature representation f attr 。
The user natural attribute code v is mapped to the feature space by using a linear transformation mode, and the formula is as follows:
f attr ＝W a T v (4)
wherein W is a Is a mapping matrix.
And 5, constructing two multi-layer LSTM networks with the same structure, wherein each layer comprises T/g LSTM units. Coarsening and normalizing call behavior time sequence xi call And traffic usage behavior timing ζ data Respectively inputting into two LSTM networks, extracting the time sequence characteristic f of the call behavior call And traffic usage behavior timing feature f data 。
To efficiently extract features of the time series data, a multi-layer LSTM network is generally employed to enhance the non-linear capabilities of the model. In order to balance the fitting capacity and complexity of the model, three layers of LSTM networks are adopted to extract time sequence characteristics of user communication and traffic use behaviors.
The LSTM network is a cyclic neural network, and each LSTM unit comprises a memory unit c t And three gates: input gate i t Output gate o t And forget door f t And respectively controlling the input, output and update of the data. In x t For input at time t, h t-1 And c t-1 For the hidden state and the memory cell state at the previous moment, the calculation formula is as follows:
i t ＝sigm(W xi x t +W hi h t-1 +b i ) (5)
f t ＝sigm(W xf x t +W hf h t-1 +b f ) (6)
o t ＝sigm(W xo x t +W ho h t-1 +W co c t-1 +b o ) (8)
wherein the operatorRepresenting the point multiplication operation, W and b represent the weight matrix and the bias vector, respectively, and sigma and tanh represent the sigmoid function and the hyperbolic tangent function, respectively.
As shown in fig. 3, the coarsened and normalized call behavior timing ζ c And traffic usage behavior timing ζ d S=t/g values are included, and corresponding LSTM cells are input, respectively. In the LSTM network, the state at the last moment is input into the next LSTM unit, the time sequence information of the data is reserved, the vector output by the last LSTM layer is activated by using a Sigmoid function, and the call behavior time sequence characteristic f is extracted c And traffic usage behavior timing feature f d 。
Step 6, constructing a multi-layer LSTM network, wherein each layer comprises p LSTM units, and the normalized telephone charge using behavior time sequence xi cost Inputting into LSTM network, extracting time sequence characteristic f of telephone charge using behavior cost 。
With conversation behavior timing xi c And traffic usage behavior timing ζ d Similarly, the normalized telephone charge use behavior time sequence xi cost Inputting into three layers of LSTM network, each layer containing p LSTM units, extracting time sequence characteristic f of telephone charge using behavior cost 。
Step 7, representing the natural attribute characteristics of the user to f attr Time sequence characteristic f of call behavior call Flow usage behavior time sequence feature f data And telephone charge use behavior time sequence characteristic f cost And splicing to obtain the user characteristic representation f.
And 8, mapping the package attribute codes l in the positive and negative samples to the feature space respectively to obtain a package attribute feature representation r.
The package attribute code/is mapped to the feature space using a linear transformation, the formula is as follows:
r＝W t T l (10)
wherein W is t For mapping matrix, l ε { l pos ，l′ 1 ，l′ 2 ，…，l′ k }。
And 9, splicing the user characteristic representation f and the package attribute characteristic representation r in the positive and negative samples, and then sending the spliced user characteristic representation f and the package attribute characteristic representation r into a fully-connected network, and outputting the package handling probability of the user.
As shown in fig. 4, after the user characteristic representation f and the package attribute characteristic representation r in the positive and negative samples are spliced, the user characteristic representation f and the package attribute characteristic representation r are sent into a fully-connected network, and the probability p of handling packages by the user is output. The activation function of the hidden layer of the fully-connected network is ReLU, the number of neurons of the output layer is 1, the activation function is Sigmoid, the range of the output result is [0,1], and the probability of handling packages by users is represented. The nonlinear capability of the model can be enhanced by superposing a plurality of fully-connected layers, and a three-layer fully-connected network is adopted for balancing the fitting capability and complexity of the model. Wherein the fully connected layer maps the input h using a nonlinear function ReLU activation as follows:
z＝ReLU(W T h) (11)
wherein W is the weight matrix of the full connection layer, and z is the output of the full connection layer.
Step 10, calculating a loss function
Logarithmic loss with negative sampling is used as a loss function in the present inventionThe calculation formula is as follows:
wherein h is θ (. Cndot.) represents the probability values of model outputs, s and s' i Positive and negative samples, respectively.
Step 11, according to the loss of all samples in the batchAnd adjusting network parameters in the whole model.
Loss of all samples in the batchThe calculation formula of (2) is as follows:
wherein, thereinFor the loss of the mth sample in the batch, M is the number of samples in each batch; according to the loss->The network parameters in the whole model are adjusted, and the updating formula is as follows:
wherein eta is the learning rate and theta is all the learnable parameters of the model.
Step 12, repeating steps 2-11 until all batches of the training dataset are involved in model training.
Step 13, repeating steps 2-12 until the specified iteration number is reached.
(3) Package recommendation phase:
step 1, the natural attribute codes preprocessed by the target user, call, flow use and telephone charge use behavior data are spliced with the attribute codes of all packages in sequence, and the attribute codes are sent to a trained model to calculate the probability of handling each package by the target user.
And 2, selecting M packages with highest handling probability as a package recommendation list of the user.
The foregoing is considered as illustrative of the principles of the present invention, and has been described herein before with reference to the accompanying drawings, in which the invention is not limited to the specific embodiments shown.
Claims (11)
1. The multi-factor embedded personalized package recommendation method based on the deep neural network is characterized by comprising three stages of data acquisition and preprocessing, model training and package recommendation, and specifically comprises the following steps:
(1) Data acquisition and preprocessing: collecting package attributes, user natural attributes and package handling records, and preprocessing; collecting daily call behavior data, daily flow usage behavior data and monthly telephone charge usage behavior data of a user, and carrying out missing value completion and outlier elimination treatment; constructing a call behavior time sequence and a flow using behavior time sequence, performing coarsening processing and normalization processing, constructing a telephone charge using behavior time sequence and performing normalization processing;
(2) Model training stage: extracting natural attribute characteristic representation of a user by using a full connection layer, constructing two multi-layer LSTM networks with the same structure, respectively inputting call behavior time sequence and flow use behavior time sequence into the two LSTM networks, and extracting call behavior time sequence characteristics and flow use behavior time sequence characteristics; constructing a multilayer LSTM network, inputting the telephone charge using behavior time sequence into the LSTM network, and extracting the telephone charge using behavior time sequence characteristics; splicing the user natural attribute characteristic representation with the call, flow use and telephone charge use behavior time sequence characteristic representation to serve as a user characteristic representation; extracting package attribute feature representation by using a full connection layer, splicing the user feature representation and the package feature representation, and then sending the package attribute feature representation and the package feature representation into a full connection network to predict the probability of handling packages by a user;
(3) Package recommendation phase: and calculating the probability of the target user handling all packages, and selecting M packages with the highest handling probability as a package recommendation list of the user.
2. The depth neural network-based multi-factor embedded personalized package recommendation method as claimed in claim 1, wherein: the data acquisition and preprocessing stage specifically comprises the following steps:
(1.1) collecting relevant package attributes, comprising: package price, subsidy amount, monthly promised consumption amount, pre-stored telephone charge and telephone charge identification, pre-stored telephone charge and physical identification, call duration, short message, common flow and directional flow;
(1.2) coding the package attribute to obtain a package attribute code l;
(1.3) collecting relevant user natural attributes, including: gender, age, time of network access, client star class, whether to use fusion package, whether to be a family network user, family number, use terminal price, CDMA number under the same client and App use preference;
(1.4) coding the natural attribute of the user to obtain a natural attribute code v of the user;
(1.5) acquiring related daily call behavior data, daily flow usage behavior data and monthly telephone charge usage behavior data of a user, and performing missing value completion and outlier elimination treatment; the daily call behavior data comprise daily call times, daily call duration, daily call times and daily call duration; daily flow use behavior data comprise daily flow use times, daily flow use duration, daily uplink flow and daily downlink flow; the monthly telephone charge usage behavior data comprises monthly telephone charge total amount, monthly extra voice charge, monthly extra flow charge and monthly extra short message charge;
(1.6) constructing a call behavior time sequence xi with a time interval of 1 day and a span of T days according to the daily call behavior data and the flow use behavior data after the completion of the missing value and the elimination of the abnormal value call And traffic usage behavior timing ζ data Coarsening and normalizing the mixture with the granularity of g; constructing telephone fee use behavior time sequence xi with time interval of 1 month and span of p months cost And carrying out normalization treatment;
(1.7) collecting a package handling record of the user; wherein the package transacting record is represented in the form of triples < u, c, t >, where u is the user id, c is the package id, and t is the date on which the user u transacts package c.
3. The depth neural network-based multi-factor embedded personalized package recommendation method as claimed in claim 2, wherein: in the step (1.2), the method for obtaining the package attribute code l is as follows:
(1.2.1) direct one-hot encoding of discrete attributes;
(1.2.2) performing maximum and minimum normalization processing on the continuous attribute, and normalizing the processed attribute value to [0,1]]X is the original attribute value, x max For the maximum value of the attribute, x min For the minimum value of this attribute, the formula is as follows:
(1.2.3) splicing all package attribute codes to obtain a package attribute code l.
4. The depth neural network-based multi-factor embedded personalized package recommendation method as claimed in claim 2, wherein: in the step (1.4), the method for obtaining the user natural attribute code v is as follows:
(1.4.1) direct one-time thermal encoding of discrete attributes;
(1.4.2) dividing the continuous attribute except age into 5 sections through an equal frequency division box, and then performing single-heat coding;
(1.4.3) for age attributes, dividing the age into 8 intervals, and then performing one-hot encoding; wherein 8 intervals are under 16 years old, 16 to 21 years old, 22 to 27 years old, 28 to 33 years old, 34 to 39 years old, 40 to 45 years old, 46 to 51 years old and over 51 years old, respectively;
and (1.4.4) splicing the single-hot codes of all the user natural attributes to obtain the user natural attribute codes v.
5. The depth neural network-based multi-factor embedded personalized package recommendation method as claimed in claim 2, wherein: in the step (1.5), the missing value and the abnormal value are detected for the daily call behavior data, the daily flow rate usage behavior data and the monthly telephone fee usage behavior data, and the missing value and the abnormal value are filled by using a linear interpolation method.
6. The depth neural network-based multi-factor embedded personalized package recommendation method as claimed in claim 2, wherein: in the step (1.6), the specific steps of coarsening and normalizing treatment are as follows:
(1.6.1) performing zero-mean normalization processing on each sequence, so that the processed data are normalized to the range of [0,1], wherein the formula is as follows:
wherein x is an original numerical value, mu is the mean value of the sequence in which the numerical value is located, and sigma is the standard deviation of the sequence in which the numerical value is located;
(1.6.2) calculating an average value of the behavior sequence with the time span of T days every g days, and coarsening the sequence with the original step length of T to the step length of T/g; due to the telephone charge use behavior time sequence xi cost The time span of (2) is 1 month, and coarsening is not required.
7. The depth neural network-based multi-factor embedded personalized package recommendation method as claimed in claim 1, wherein: the model training stage specifically comprises the following steps:
(2.1) constructing a training data set, and batching the training data set according to a fixed batch size, wherein the total number is N;
(2.2) sequentially selecting a collection of training samples with index i from the training dataset, wherein i e {0,1, …, N }; repeating steps (2.3) -step (2.10) for each training sample in the batch;
(2.3) taking the sample s in the training data set as a positive sample, randomly sampling k other packages, and replacing packages in the sample s to obtain k negative samples s' 1 ,s′ 2 ,…,s′ k ；
(2.4) mapping the user natural attribute code v to the feature space to obtain a user natural attribute feature representation f attr The method comprises the steps of carrying out a first treatment on the surface of the Wherein, the user natural attribute code v is mapped to the feature space by using a linear transformation mode, and the formula is as follows:
f attr ＝W a T v
wherein W is a Is a mapping matrix;
(2.5) constructing two multi-layer LSTM networks with the same structure, wherein each layer comprises T/g LSTM units; coarsening and normalizing call behavior time sequence xi call And traffic usage behavior timing ζ data Respectively inputting the call behavior time sequence relation and the traffic use behavior time sequence relation into two LSTM networks to extract the call behavior time sequence characteristic f call And traffic usage behavior timing feature f data ；
(2.6) constructing a multilayer LSTM network, wherein each layer comprises p LSTM units, and the normalized telephone charge use behavior time sequence xi cost Inputting into LSTM network, modeling telephone charge use behavior time sequence relation to extract telephone charge use behavior time sequence characteristic f cost ；
(2.7) characterizing the user's natural attributesSign representation f attr Time sequence characteristic f of call behavior call Flow usage behavior time sequence feature f data And telephone charge use behavior time sequence characteristic f cost Splicing to obtain a user characteristic representation f;
(2.8) mapping the package attribute codes l in the positive and negative samples to feature spaces respectively to obtain a package attribute feature representation r; wherein, the package attribute code/is mapped to the feature space by using a linear transformation mode, and the formula is as follows:
r＝W t T l
wherein W is t For mapping matrix, l ε { l pos ，l′ 1 ，l′ 2 ，…，l′ k }；
(2.9) splicing the user characteristic representation f and the package attribute characteristic representation r in the positive and negative samples, then sending the spliced user characteristic representation f and the package attribute characteristic representation r into a fully-connected network, and outputting the probability of handling packages by the user;
(2.10) calculating a loss functionWherein logarithmic loss with negative sampling is used as a loss function +.>The calculation formula is as follows:
wherein h is θ (. Cndot.) represents the probability values of model outputs, s and s' i Positive and negative samples, respectively;
(2.11) according to loss of all samples in the batchAdjusting network parameters in the whole model;
(2.12) repeating steps (2.2) -step (2.11) until all batches of the training dataset have participated in model training;
(2.13) repeating steps (2.2) -step (2.12) until a specified number of iterations is reached.
8. The depth neural network-based multi-factor embedded personalized package recommendation method as claimed in claim 7, wherein: the step (2.1) is specifically as follows: each package handling record is constructed into a training sample, and the format of each sample is as follows: s= [ v, ζ ] call ，ξ data ,ξ cost ；l pos ]All training samples form a training data set; the training data set is batched according to the batch size M manually set by experience, and the total number of the batches is N;
the specific calculation mode is as follows:
wherein N is Samples For the total number of samples in the training dataset,is a top-rounding function.
9. The depth neural network-based multi-factor embedded personalized package recommendation method as claimed in claim 7, wherein: in the step (2.5), modeling a call behavior time sequence relation and a flow use behavior time sequence relation by adopting a three-layer LSTM network so as to extract call behavior time sequence characteristics and flow use behavior time sequence characteristics; the LSTM network is a cyclic neural network, and each LSTM unit comprises a memory unit c t And three gates: input gate i t Output gate o t And forget door f t Respectively controlling the input, output and update of data; in x t For input at time t, h t-1 And c t-1 For the hidden state and the memory cell state at the previous moment, the calculation formula is as follows:
i t ＝sigm(W xi x t +W hi h t-1 +b i )
f t ＝sigm(W xf x t +W hf h t-1 +b f )
o t ＝sigm(W xo x t +W ho h t-1 +W co c t-1 +b o )
wherein the operatorRepresenting the point multiplication operation, W and b represent the weight matrix and the bias vector, respectively, and sigma and tanh represent the sigmoid function and the hyperbolic tangent function, respectively.
10. The depth neural network-based multi-factor embedded personalized package recommendation method as claimed in claim 7, wherein: in step (2.11), loss of all samples in the batchThe calculation formula of (2) is as follows:
wherein, thereinFor the loss of the mth sample in the batch, M is the number of samples in each batch; according to the loss->The network parameters in the whole model are adjusted, and the updating formula is as follows:
wherein eta is the learning rate and theta is all the learnable parameters of the model.
11. The depth neural network-based multi-factor embedded personalized package recommendation method as claimed in claim 1, wherein: the package recommending stage specifically comprises the following steps:
the natural attribute codes after the pretreatment of the target user are spliced with the attribute codes of all packages in sequence, and the probability of handling each package by the target user is calculated by sending the natural attribute codes into a trained model;
and (3.2) selecting M packages with highest handling probability as a package recommendation list of the user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911134269.8A CN110955828B (en) | 2019-11-19 | 2019-11-19 | Multi-factor embedded personalized package recommendation method based on deep neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911134269.8A CN110955828B (en) | 2019-11-19 | 2019-11-19 | Multi-factor embedded personalized package recommendation method based on deep neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110955828A CN110955828A (en) | 2020-04-03 |
| CN110955828B true CN110955828B (en) | 2023-08-01 |
Family
ID=69977697
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911134269.8A Active CN110955828B (en) | 2019-11-19 | 2019-11-19 | Multi-factor embedded personalized package recommendation method based on deep neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110955828B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112487295B (en) * | 2020-12-04 | 2024-08-13 | 中国移动通信集团江苏有限公司 | 5G package pushing method and device, electronic equipment and computer storage medium |
| CN112836081B (en) * | 2021-03-01 | 2024-09-06 | 腾讯音乐娱乐科技（深圳）有限公司 | Neural network model training method, information recommendation method and storage medium |
| CN113139856B (en) * | 2021-05-14 | 2023-04-07 | 青岛聚看云科技有限公司 | Movie and television member package recommendation method and device |
| CN113434757A (en) * | 2021-06-23 | 2021-09-24 | 青岛聚看云科技有限公司 | Media asset package recommendation method and display device |
| CN116456323B (en) * | 2023-05-30 | 2023-09-12 | 南京邮电大学 | User package recommendation method and system based on user preference decoupling |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018023439A1 (en) * | 2016-08-02 | 2018-02-08 | 步晓芳 | Information prompt method performed during mobile phone bill recharging, and mobile phone |
| CN110196945A (en) * | 2019-05-27 | 2019-09-03 | 北京理工大学 | A kind of microblog users age prediction technique merged based on LSTM with LeNet |
| CN110287335A (en) * | 2019-06-17 | 2019-09-27 | 桂林电子科技大学 | The personalized recommending scenery spot method and device of knowledge based map and user's shot and long term preference |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10824941B2 (en) * | 2015-12-23 | 2020-11-03 | The Toronto-Dominion Bank | End-to-end deep collaborative filtering |
| US10552431B2 (en) * | 2017-09-29 | 2020-02-04 | Microsoft Technology Licensing, Llc | Personalizing search and/or recommendation results based on member activity models |
| CN107909427A (en) * | 2017-10-25 | 2018-04-13 | 浙江大学 | A kind of Recognition with Recurrent Neural Network method for lifting recommended models timing driving ability |
| CN109218034A (en) * | 2018-08-24 | 2019-01-15 | 曹春江 | Communication user set meal accurate recommendation method based on multi-angle neural network model |
| CN109389245A (en) * | 2018-09-06 | 2019-02-26 | 浙江鸿程计算机系统有限公司 | A kind of multifactor fusion school district school age population prediction technique based on deep neural network |
| CN109919675A (en) * | 2019-03-04 | 2019-06-21 | 深圳微品致远信息科技有限公司 | Communication user upshift prediction probability recognition methods neural network based and system |
| CN110097225A (en) * | 2019-05-05 | 2019-08-06 | 中国科学技术大学 | Collaborative forecasting method based on sound state depth characterization |
-
2019
- 2019-11-19 CN CN201911134269.8A patent/CN110955828B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018023439A1 (en) * | 2016-08-02 | 2018-02-08 | 步晓芳 | Information prompt method performed during mobile phone bill recharging, and mobile phone |
| CN110196945A (en) * | 2019-05-27 | 2019-09-03 | 北京理工大学 | A kind of microblog users age prediction technique merged based on LSTM with LeNet |
| CN110287335A (en) * | 2019-06-17 | 2019-09-27 | 桂林电子科技大学 | The personalized recommending scenery spot method and device of knowledge based map and user's shot and long term preference |
Non-Patent Citations (1)
| Title |
|---|
| Adaptive user modeling with long and short-term preferences for personalized recommendation;Zeping Yu;/International Joint Conference on Artificial Intelligence.International Joint Conferences on Artificial Intelligence Organization;4213-4219 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110955828A (en) | 2020-04-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110955828B (en) | Multi-factor embedded personalized package recommendation method based on deep neural network | |
| CN110751286B (en) | Training method and training system for neural network model | |
| CN109961080B (en) | Terminal identification method and device | |
| CN109857871B (en) | User relationship discovery method based on social network mass contextual data | |
| CN108205766A (en) | Information-pushing method, apparatus and system | |
| CN109933699A (en) | A kind of construction method and device of academic portrait model | |
| CN107862053A (en) | User's portrait building method, device and computing device based on customer relationship | |
| US20230108469A1 (en) | Systems and methods for generating dynamic conversational responses using cluster-level collaborative filtering matrices | |
| CN112995414B (en) | Behavior quality inspection method, device, equipment and storage medium based on voice call | |
| CN112633962A (en) | Service recommendation method and device, computer equipment and storage medium | |
| CN113011646A (en) | Data processing method and device and readable storage medium | |
| CN110704510A (en) | User portrait combined question recommendation method and system | |
| CN113420212A (en) | Deep feature learning-based recommendation method, device, equipment and storage medium | |
| CN110866637B (en) | Scoring prediction method, scoring prediction device, computer equipment and storage medium | |
| CN118134630A (en) | Credit risk level assessment method and device and electronic equipment | |
| CN113704637A (en) | Object recommendation method, device and storage medium based on artificial intelligence | |
| CN115249082A (en) | User interest prediction method, device, storage medium and electronic equipment | |
| CN110991711A (en) | Multi-factor perception terminal switching prediction method based on deep neural network | |
| US11568308B2 (en) | Correcting bias in supervised machine learning data | |
| CN115376668B (en) | Big data business analysis method and system applied to intelligent medical treatment | |
| CN111325578A (en) | Prediction model sample determination method, prediction model sample determination device, prediction model sample determination medium, and prediction model sample determination device | |
| CN110765352A (en) | User interest identification method and device | |
| CN115545834A (en) | Personalized service recommendation method based on graph neural network and metadata | |
| CN113626697A (en) | Anchor-LDA and convolutional neural network-based interest point recommendation method | |
| CN116800831B (en) | Service data pushing method, device, storage medium and processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |