CN111311324A - User-commodity preference prediction system and method based on stable neural collaborative filtering - Google Patents

User-commodity preference prediction system and method based on stable neural collaborative filtering Download PDF

Info

Publication number
CN111311324A
CN111311324A CN202010098124.3A CN202010098124A CN111311324A CN 111311324 A CN111311324 A CN 111311324A CN 202010098124 A CN202010098124 A CN 202010098124A CN 111311324 A CN111311324 A CN 111311324A
Authority
CN
China
Prior art keywords
user
commodity
model
representing
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010098124.3A
Other languages
Chinese (zh)
Other versions
CN111311324B (en
Inventor
王庆先
张枭
陈彪
马康康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN202010098124.3A priority Critical patent/CN111311324B/en
Publication of CN111311324A publication Critical patent/CN111311324A/en
Application granted granted Critical
Publication of CN111311324B publication Critical patent/CN111311324B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0201—Market modelling; Market analysis; Collecting market data
    • G06Q30/0202—Market predictions or forecasting for commercial activities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G06Q30/0631—Item recommendations

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • General Physics & Mathematics (AREA)
  • Strategic Management (AREA)
  • Development Economics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Business, Economics & Management (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Game Theory and Decision Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)

Abstract

The invention provides a user-commodity preference prediction system based on stable neural collaborative filtering, which comprises a data preprocessing module, a data storage module connected with the data preprocessing module, a parameter control module, a data output module, a prediction result generation module and a model training module, wherein the parameter control module, the data output module and the prediction result generation module are respectively connected with the data storage module, and the model training module is connected with the parameter control module and the model training module. Based on the system, the invention also provides a user-commodity preference prediction method based on stable neural collaborative filtering. The method aims to simulate data fluctuation when a user maliciously attacks by using noise, and analyze the internal statistical rules of known user-commodity scoring data by introducing a guide model for aided training, thereby providing a stable and accurate user-commodity scoring prediction result and providing personalized, safe and reliable financial product recommendation service for the user.

Description

User-commodity preference prediction system and method based on stable neural collaborative filtering
Technical Field
The invention relates to the technical field of computer data processing, in particular to a user-commodity preference prediction system and method based on stable neural collaborative filtering.
Background
Financial products on the market are various in types, and users can not find satisfactory financial products suitable for the users by submerging in various choices, so that customized recommendation aiming at personal conditions and preferences of the users is necessary, and the recommendation system in modern electronic commerce well solves the problem. In the financial product platform, a huge user-commodity scoring matrix is formed through the scoring of a user on commodities, and the favor degree of the user on the commodities is predicted by utilizing the grade. Because of the large number of users and the abundance of goods in the system, in general, the user-goods scoring matrix is a very sparse high-dimensional matrix because it is impossible for each user to score all goods one by one.
According to historical scoring of the e-commerce platform, the preference rule of the user on the commodity can be known and analyzed, and an effective user-commodity preference prediction model is established on the basis. And the real environment is simulated through the simulation environment of the commodity scoring by the user, so that an important scientific basis is provided for the recommendation strategy of the customized personal financial product. Currently, there are many prediction methods regarding user-commodity preferences. The matrix decomposition model is one of the most successful methods in the recommendation system, a new score is predicted by mapping users and commodities to a hidden feature space and then calculating potential interaction of the users and the commodities, and then due to the excellent capability of a deep learning algorithm in the aspect of representing the essence of data, the deep neural network-based collaborative filtering method has a good effect in the aspect of learning the hidden representation of the users and the commodities. However, due to the openness of the recommendation system, many studies in recent years show that the factorization-based method is vulnerable to malicious user attacks, and through the knowledge of the collaborative filtering algorithm and its parameters, for example, an attacker knows the architecture of the recommendation system or legitimate user data to easily attack the recommendation system, thereby reducing the performance and stability of the recommendation system. On the one hand, attackers generate malicious user data to reduce the performance of the system, and on the other hand, the attackers keep their own behaviors near normal users to avoid being discovered, which results in that the original recommendation algorithm can not process the malicious attacks from the users at all. Due to the particularity of the financial industry, high requirements are required on the stability and the prediction accuracy of the model. Therefore, the method has great disadvantages and hidden dangers when being practically applied to an electronic commerce system related to financial products.
Disclosure of Invention
In order to overcome the defects in the prior art, the user-commodity preference prediction system and method based on stable neural collaborative filtering provided by the invention can be realized according to the user-commodity interaction records: constructing a recommendation system which accords with the financial market environment and is still stable when the recommendation system faces attacks from malicious users; customized accurate financial product predictions and recommendations for different users.
In order to achieve the above purpose, the invention adopts the technical scheme that:
the scheme provides a user-commodity preference prediction system based on stable neural collaborative filtering, which comprises a data preprocessing module, a data storage module connected with the data preprocessing module, a parameter control module, a data output module, a prediction result generation module and a model training module, wherein the parameter control module, the data output module and the prediction result generation module are respectively connected with the data storage module, and the model training module is connected with the parameter control module and the model training module;
the data preprocessing module is used for acquiring user-commodity scoring data, preprocessing the user-commodity scoring data and inputting the preprocessed user-commodity scoring data into the data storage module;
the data storage module is used for storing the preprocessed user-commodity scoring data, the parameters of the user-commodity preference prediction model and the generated prediction scoring data;
the model training module is used for constructing and training a user-commodity preference prediction model based on stable neural collaborative filtering according to the preprocessed user-commodity scoring data;
the parameter control module is used for initializing parameters of the user-commodity preference prediction model and controlling the training process of the parameters and the user-commodity preference prediction model;
the prediction result generation module is used for generating unknown user-commodity prediction scoring data by utilizing the hidden feature vectors of the user and the commodity according to the user-commodity preference prediction model and storing the generated prediction scoring data into the data storage module;
and the data output module is used for outputting the hidden feature representation of the user and the commodity and the unknown user-commodity prediction scoring data.
Further, the model training module comprises a guidance model training unit, a guidance information generating unit and a neural network training unit;
the guide model training unit is used for training a non-negative matrix decomposition model which is used as a guide model and does not contain noise according to the preprocessed user-commodity scoring data;
the guidance information generating unit is used for generating guidance information according to the non-negative matrix factorization model;
the neural network training unit is used for constructing and training a user-commodity preference prediction model based on stable neural collaborative filtering according to the generated guide information, wherein:
the user-commodity preference prediction model comprises a vectorization layer, a noise layer, a multi-layer perceptron layer and a fully-connected output layer which are connected in sequence.
Still further, the parameter control module comprises an initialization unit and a parameter control unit;
the initialization unit is used for initializing parameters of the user-commodity preference prediction model;
and the parameter control unit is used for transmitting parameters to the prediction result generation module and controlling the training process of the user-commodity preference prediction model.
Still further, the prediction result generation module comprises a parameter receiving unit and a prediction result generation unit;
the parameter receiving unit is used for receiving the parameters transmitted by the parameter control module;
and the prediction result generation unit is used for predicting the unknown user-commodity scores by using the hidden feature vectors of the users and the commodities.
Based on the system, the invention also discloses a user-commodity preference prediction method based on stable neural collaborative filtering, which comprises the following steps:
s1, obtaining user-commodity scoring data;
s2, preprocessing the user-commodity scoring data and storing the preprocessed user-commodity scoring data;
s3, training a non-negative matrix decomposition model which is used as a guide model and does not contain noise according to the preprocessed user-commodity scoring data;
s4, generating guide information by using the non-negative matrix factorization model;
s5, constructing and training a user-commodity preference prediction model containing a noise layer and based on stable neural collaborative filtering according to the guide information;
s6, generating unknown user-commodity prediction scores by using the user-commodity preference prediction model according to the hidden feature vectors of the user and the commodities, and storing the user-commodity prediction scores;
and S7, outputting the implicit characteristic representation of the user and the commodity and the unknown user-commodity prediction score, thereby completing the prediction of the user-commodity preference.
Further, the step S3 includes the following steps:
s301, initializing parameters;
s302, constructing a target function according to the preprocessed user-commodity rating data;
s303, training the hidden feature matrixes of the user and the commodity by using a gradient descent method according to the target function, thereby finishing the training of a non-negative matrix decomposition model which is used as a guide model and does not contain noise, wherein the target function RSER(A)The expression of (a) is as follows:
Figure BDA0002385906350000051
Figure BDA0002385906350000052
Figure BDA0002385906350000053
wherein R is(A)Representing a set of user-to-commodity known rating data, R, in a user-to-commodity rating matrix Ru,iDenotes the user u's score, p, for item iuRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden features, lambda, corresponding to the ith commodity in the financial product hidden feature matrix QpAnd λQBoth represent regularization factors acting on hidden feature matrices P and Q of the user and the commodity, respectively, F represents a norm,
Figure BDA0002385906350000054
representing the predicted score, I, of user u on item I calculated by the modelu,UiRespectively representing a set of items interacted by the user u and a set of users interacted with the item i.
Still further, the expression of the guidance information h in step S4 is as follows:
h＝emb(pu,qi,θG)
wherein p isuRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden features, theta, corresponding to the ith commodity in the financial product hidden feature matrix QGRepresents the parameters of vectorization, emb (-) represents the vectorization operation.
Still further, the step S5 includes the following steps:
s501, initializing parameters, and constructing a user-commodity preference prediction model which comprises a noise layer and is based on stable neural collaborative filtering;
s502, sequentially inputting the IDs and the side information of the user and the commodity into a vectorization layer and a noise layer of a user-commodity preference prediction model to obtain the hidden feature vectors of the user and the commodity, wherein the hidden feature vectors of the user and the commodity are pi (p)u,qi) The expression of (a) is as follows:
Figure BDA0002385906350000061
wherein e is Gaussian noise with mean 0 and variance δ, N (-) represents Gaussian distribution, puRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden feature corresponding to the ith commodity in the financial product hidden feature matrix Q, emb ((-)) representing vectorization operation,
Figure BDA0002385906350000062
an id representing the user u and a location id,
Figure BDA0002385906350000063
which represents the characteristics of the user u,
Figure BDA0002385906350000064
an id representing the number of the item i,
Figure BDA0002385906350000065
a feature representing item i;
s503, with the guide information as a target, performing feature extraction training on the hidden feature vectors of the user and the commodity, wherein the feature extraction training is L (theta)G，θr) The expression of (a) is as follows:
Figure BDA0002385906350000066
wherein h represents generated guidance information, emb (phi) represents vectorization operation, sigma (phi) represents activation function, b represents bias parameter, and thetarMapping vector p when hidden feature space dimensions in matrix decomposition model and neural network are differentuRepresenting user hidden feature momentsImplicit feature, q, corresponding to the u-th user in array PiRepresenting the hidden features, theta, corresponding to the ith commodity in the financial product hidden feature matrix QGRepresenting vectorized parameters, ru,iThe score of the user u on the commodity i is represented, and theta represents parameters of the user and the hidden feature vector of the commodity;
s504, the hidden feature vectors of the user and the commodity after feature extraction training are sequentially input into a multi-layer perceptron and a full-connection output layer of the user-commodity preference prediction model, and a neural network is trained on an objective function by using a random gradient descent method, so that the training of the user-commodity preference prediction model which comprises a noise layer and is based on stable neural collaborative filtering is completed.
Still further, an unknown user-commodity prediction score f (p) is generated in the step S6u,qi) The expression of (a) is as follows:
f(pu,qi)＝φout(φX(...φ2(φ1(π(pu,qi)))...))
wherein phi isoutRepresenting a non-linear transformation of an output layer network in a multi-layer neural network, phiXRepresents the non-linear transformation of the X-th network in a multi-layer neural network, phi2Representing a non-linear transformation, phi, of a second-layer neural network in a multi-layer neural network1Representing a non-linear transformation, p, of a first layer of a multi-layer neural networkuRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiAnd representing the hidden feature corresponding to the ith commodity in the financial product hidden feature matrix Q.
The invention has the beneficial effects that:
(1) the invention provides a user-commodity preference prediction system and method based on stable neural collaborative filtering, aiming at simulating data fluctuation when a user maliciously attacks by utilizing noise, and analyzing the internal statistical rules of known user-commodity scoring data by introducing a guidance model for aided training, thereby providing a stable and accurate user-commodity scoring prediction result and providing personalized, safe and reliable financial product recommendation service for the user. The invention realizes the following through the design: constructing a recommendation system which accords with the financial market environment and is still stable when the recommendation system faces attacks from malicious users; customized accurate financial product prediction and recommendation for different users;
(2) the noise layer is added into the neural collaborative filtering model, so that the robustness of the model is improved, better stability is shown in the face of attack of malicious users, the noise-free guidance model is trained firstly, and then the generated guidance information is utilized to train a staged training process of a network containing noise, so that the model stability is improved, the prediction precision of the model is ensured, and the method can be widely applied to an electronic commerce platform for providing personalized service.
Drawings
FIG. 1 is a system block diagram of the present invention.
FIG. 2 is a flow chart of the method of the present invention.
Fig. 3 is a schematic diagram of success rate of malicious attacks on a system before and after application of the present invention.
FIG. 4 is a comparison graph of RMSE during data analysis before and after the application of the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
Examples
As shown in fig. 1, the present invention provides a user-commodity preference prediction system based on stable neural collaborative filtering, which includes a data preprocessing module, a data storage module connected to the data preprocessing module, a parameter control module, a data output module, and a prediction result generation module respectively connected to the data storage module, and a model training module connected to the parameter control module, wherein the prediction result generation module is respectively connected to the parameter control module and the model training module;
the data preprocessing module is used for acquiring user-commodity scoring data, preprocessing the user-commodity scoring data and inputting the preprocessed user-commodity scoring data into the data storage module;
the data storage module is used for storing the preprocessed user-commodity scoring data, the parameters of the user-commodity preference prediction model and the generated prediction scoring data;
the model training module is used for constructing and training a user-commodity preference prediction model based on stable neural collaborative filtering according to the preprocessed user-commodity scoring data; the model training module comprises a guidance model training unit, a guidance information generating unit and a neural network training unit;
the guide model training unit is used for training a non-negative matrix decomposition model which is used as a guide model and does not contain noise according to the preprocessed user-commodity scoring data;
the guidance information generating unit is used for generating guidance information according to the non-negative matrix factorization model;
the neural network training unit is used for constructing and training a user-commodity preference prediction model based on stable neural collaborative filtering according to the generated guide information, wherein:
the user-commodity preference prediction model comprises a vectorization layer, a noise layer, a multi-layer sensor layer and a full-connection output layer which are sequentially connected;
the parameter control module is used for initializing parameters of the user-commodity preference prediction model and controlling the training process of the parameters and the user-commodity preference prediction model; the parameter control module comprises an initialization unit and a parameter control unit;
an initialization unit for initializing parameters of a user-commodity preference prediction model;
the parameter control unit is used for transmitting parameters to the prediction result generation module and controlling the training process of the user-commodity preference prediction model;
the prediction result generation module is used for generating unknown user-commodity prediction scoring data by using the hidden feature vectors of the user and the commodity according to the user-commodity preference prediction model and storing the generated user-commodity prediction scoring data into the data storage module; the prediction result generation module comprises a parameter receiving unit and a prediction result generation unit;
the parameter receiving unit is used for receiving the parameters transmitted by the parameter control module;
the prediction result generation unit is used for predicting the unknown user-commodity scores by using the hidden feature vectors of the users and the commodities;
and the data output module is used for outputting the hidden feature representation of the user and the commodity and the unknown user-commodity prediction scoring data.
As shown in fig. 2, based on the above system, the present invention also discloses a user-commodity preference prediction method based on stable neural collaborative filtering, which includes the following steps:
s1, obtaining user-commodity scoring data;
in this embodiment, the user-commodity scoring data is scoring data of a commodity after the commodity is actually selected by a user on a financial product platform.
S2, preprocessing the user-commodity scoring data and storing the preprocessed user-commodity scoring data;
in the embodiment, an array is used for representing the scoring relationship between the user and the commodity, a row where an element in the matrix is located represents the user number, a column where the element is located represents the commodity number, a user set is represented as U, a commodity set is represented as I, a matrix of | U | rows and | I | columns is established as a user-commodity scoring matrix R according to scoring data of the user on the commodity, for each matrix element in the matrix, the row where the element is located represents the user number, and the column where the element is located represents the commodity number, and the scoring matrix R is stored in the data storage module.
S3, training a non-negative matrix decomposition model which is used as a guide model and does not contain noise according to the preprocessed user-commodity scoring data, wherein the realization method comprises the following steps:
s301, initializing parameters;
in this embodiment, model parameters are initializedInitializing hidden feature matrixes P and Q and a hidden feature space d dimension of a user and a commodity, wherein P is a matrix of | U | rows and d columns, each row vector in P corresponds to a user and is a hidden feature vector of the user, Q is a matrix of | I | rows and d columns, each row vector in Q corresponds to a commodity and is a hidden feature vector of the commodity, the hidden feature vectors are respectively initialized by using random smaller positive numbers, and d is the dimension of the hidden feature space of the user and the hidden feature space of the commodity and is initialized to be a positive integer; initializing a regularization factor λPAnd λQRespectively acting on hidden feature matrixes P and Q of users and commodities, and initializing the hidden feature matrixes P and Q into smaller positive numbers; initializing a maximum iteration round number T as a variable for controlling the upper limit of the training times to be a larger positive integer; the initial convergence termination threshold s is a parameter for determining whether training is terminated, and is initialized to a very small positive number.
S302, constructing a target function according to the preprocessed user-commodity rating data;
in this embodiment, a reference model without noise is trained, and guidance parameters are provided for subsequent neural network training. The formula is as follows:
Figure BDA0002385906350000101
wherein R is(A)Representing a set of user-to-commodity known rating data, R, in a user-to-commodity rating matrix Ru,iDenotes the user u's score, p, for item iuRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden features, lambda, corresponding to the ith commodity in the financial product hidden feature matrix QpAnd λQBoth represent regularization factors acting on hidden feature matrices P and Q of the user and the commodity, respectively, F represents a norm,
Figure BDA0002385906350000111
representing the predicted score, I, of user u on item I calculated by the modelu,UiRespectively representing a set of commodities interacted by a user u and a set of users interacted with the commodities i;
and updating the formula according to the known grading data during model training as follows:
Figure BDA0002385906350000112
Figure BDA0002385906350000113
wherein p isuRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden features, theta, corresponding to the ith commodity in the financial product hidden feature matrix QGRepresenting vectorized parameters, ru,iDenotes the user u's score, λ, for item ipAnd λQBoth represent the regularization factors acting on the implicit feature matrices P and Q for the user and the good, respectively.
S303, training the hidden feature matrixes of the user and the commodity by using a gradient descent method according to the target function, thereby finishing the training of a non-negative matrix decomposition model which is used as a guide model and does not contain noise;
s4, generating guide information by using the non-negative matrix factorization model;
in this embodiment, a common matrix decomposition model cannot maintain stability in the face of malicious attacks by users, which results in accuracy degradation. A common attack is to reduce the prediction accuracy of the recommendation algorithm by perturbing the user and commodity feature matrices:
Figure BDA0002385906350000114
in order to prevent malicious attacks and improve the stability of the algorithm in the face of disturbance in the aspect of preventing the malicious attacks, the noise layer is added behind the vectorization layer of the user and the commodity to simulate the disturbance of the characteristic vector in the malicious attacks, and then the final stable model is obtained by optimizing the network containing the noise. Meanwhile, because noise is added, the extraction of the hidden features of the model can be influenced, so that a matrix decomposition model trained in advance is used as a guide, the hidden features trained in a noise-free environment are used for assisting training, and guide information is generated:
h＝emb(pu,qi,θG)
wherein p isuRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden features, theta, corresponding to the ith commodity in the financial product hidden feature matrix QGRepresents the parameters of vectorization, emb (-) represents the vectorization operation.
S5, according to the guiding information, constructing and training a user-commodity preference prediction model containing a noise layer and based on stable neural collaborative filtering, wherein the implementation method comprises the following steps:
s501, initializing parameters, and constructing a user-commodity preference prediction model which comprises a noise layer and is based on stable neural collaborative filtering;
in this embodiment, the parameters are initialized and a neural collaborative filtering model with a noise layer added is constructed, the model includes a vectorization layer, a noise layer, a multi-layer perceptron layer and a full-connection output layer, and a staged training mode is adopted. The initialization parameters include: initializing vectorization layer sizes of users and commodities, wherein each vector represents the hidden features of the users or the commodities, corresponds to the hidden feature matrix, is a hidden feature space dimension, and is initialized to be a positive integer; initializing a model optimization method optim, a method used in a back propagation optimization network, and initializing into Adam; initializing an activation function of the multilayer perceptron, and initializing to ReLu by default; initializing a maximum iteration round number T as a variable for controlling the upper limit of the training times to be a larger positive integer; initializing dropout, randomly losing the number of network units in training, and initializing to be floating point numbers smaller than 1; and initializing the error accuracy tol of the training in the training process.
S502, sequentially inputting the IDs and the side information of the user and the commodity into a vectorization layer and a noise layer of a user-commodity preference prediction model to obtain the hidden feature vectors of the user and the commodity, wherein the hidden feature vectors of the user and the commodity are pi (p)u,qi) The expression of (a) is as follows:
Figure BDA0002385906350000131
wherein e is Gaussian noise with mean 0 and variance δ, N (-) represents Gaussian distribution, puRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden feature corresponding to the ith commodity in the financial product hidden feature matrix Q, emb ((-)) representing vectorization operation,
Figure BDA0002385906350000132
an id representing the user u and a location id,
Figure BDA0002385906350000133
which represents the characteristics of the user u,
Figure BDA0002385906350000134
an id representing the number of the item i,
Figure BDA0002385906350000135
a feature representing item i;
s503, with the guide information as the target, carrying out feature extraction training on the hidden feature vectors of the user and the commodity, and carrying out feature extraction training L (theta)G，θr) The expression of (a) is as follows:
Figure BDA0002385906350000136
wherein h represents generated guidance information, emb (phi) represents vectorization operation, sigma (phi) represents activation function, b represents bias parameter, and thetarMapping vector p when hidden feature space dimensions in matrix decomposition model and neural network are differentuRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden features, theta, corresponding to the ith commodity in the financial product hidden feature matrix QGRepresenting vectorized parameters, ru,iDenotes the user u's score, λ, for item ipAnd λQImplicit feature matrices with regularization factors acting on users and commodities respectivelyP and Q, theta represent parameters of hidden feature vectors of users and commodities;
s504, the hidden feature vectors of the user and the commodity after feature extraction training are sequentially input into a multilayer perceptron and a full-connection output layer of the user-commodity preference prediction model:
f(pu,qi)＝φout(φX(...φ2(φ1(π(pu,qi)))...))
wherein phi isoutRepresenting a non-linear transformation of an output layer network in a multi-layer neural network, phiXRepresents the non-linear transformation of the X-th network in a multi-layer neural network, phi2Representing a non-linear transformation, phi, of a second-layer neural network in a multi-layer neural network1Representing a non-linear transformation, p, of a first layer of a multi-layer neural networkuRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden feature corresponding to the ith commodity in the financial product hidden feature matrix Q;
and training the neural network on the target function by using a random gradient descent method:
Figure BDA0002385906350000141
and obtaining a final user-commodity preference prediction model based on the stable neural collaborative filtering, and storing corresponding parameters into a data storage module, thereby completing the training of the user-commodity preference prediction model containing a noise layer and based on the stable neural collaborative filtering.
S6, generating unknown user-commodity prediction scores by using a user-commodity preference prediction model according to the hidden feature vectors of the users and the commodities, and storing the user-commodity prediction scores;
in this embodiment, the parameter receiving unit receives the parameters transmitted by the parameter control module, and in combination with the model constructed and trained by the model training module, the prediction result generating module predicts the unknown scores by using the implicit feature vectors of the user and the commodity:
f(pu,qi)＝φout(φX(...φ2(φ1(π(pu,qi)))...))
the generated prediction result is stored in the data storage module of the device.
And S7, outputting the implicit characteristic representation of the user and the commodity and the unknown user-commodity scoring prediction data, thereby completing the prediction of the user-commodity preference.
In this embodiment, fig. 3 is a schematic diagram of attack success rate when a previous system and a later system applying the present invention face malicious attacks, and fig. 4 is a schematic diagram of prediction accuracy of the previous system and the later system applying the present invention. First, it is apparent from fig. 3 that, after the prediction system and method of the present invention are applied, the success rate of malicious attacks on the recommendation system is significantly reduced, and the system exhibits higher stability in the face of attacks, which is far better than the case of not applying the prediction system and method of the present invention. Specifically, as shown by the data in fig. 3, the y-axis is the success rate of malicious attack, and the x-axis is the comparison between before and after applying the present invention. Before the prediction system and the prediction method are not used, the success rate of malicious attack on the system is up to 0.85, after the prediction system and the prediction method are used, the success rate of attack is reduced to 0.21, and data analysis can clearly see that the success rate of attacking the model which does not use the prediction system and the prediction method is 4 times that of the model which uses the prediction system and the prediction method. Namely, after the prediction system and the method are applied, the safety and the stability of the recommendation system are considered, the prediction system and the method are improved by more than 4 times compared with the unused condition, and the robustness of the system is greatly improved. Meanwhile, as can be seen from the comparison graph of the system prediction accuracy in fig. 4, the y-axis is the RMSE value (RMSE is a standard commonly used in the field of recommendation algorithms to measure the prediction accuracy, and the smaller the RMSE, the higher the accuracy, the more accurate the model prediction), and the x-axis is the comparison between the previous and subsequent cases in which the present invention is applied. The RMSE of the model before the use of the invention is 0.855, and after the use of the invention, the RMSE value of the model is 0.841, and the data analysis from the front and back is performed, because the smaller the RMSE, the higher the prediction accuracy of the model is, that is, after the prediction system and method of the invention are applied, not only the robustness of the model is greatly enhanced (conclusion in fig. 3), but also the accuracy of the model prediction is slightly improved compared with that without the use of the invention. In practical application, the invention can better provide safe and reliable commodity recommendation meeting the individual requirements of the user for the user.
The invention adds a noise layer in the neural collaborative filtering model, increases the robustness of the model, shows better stability in the face of attack of malicious users, improves the stability of the model and ensures the prediction precision of the model at the same time by training a noiseless guide model and then training a staged training process of a network containing noise by utilizing the generated guide information, and can be widely used for an electronic commerce platform for providing personalized service, provides a stable and accurate user-commodity scoring prediction result, provides personalized, safe and reliable financial product recommendation service for the user, and realizes that: constructing a recommendation system which accords with the financial market environment and is still stable when the recommendation system faces attacks from malicious users; customized accurate financial product predictions and recommendations for different users.

Claims (9)

1. The user-commodity preference prediction system based on stable neural collaborative filtering is characterized by comprising a data preprocessing module, a data storage module connected with the data preprocessing module, a parameter control module, a data output module and a prediction result generation module which are respectively connected with the data storage module, and a model training module connected with the parameter control module, wherein the prediction result generation module is respectively connected with the parameter control module and the model training module;
the data preprocessing module is used for acquiring user-commodity scoring data, preprocessing the user-commodity scoring data and inputting the preprocessed user-commodity scoring data into the data storage module;
the data storage module is used for storing the preprocessed user-commodity scoring data, the parameters of the user-commodity preference prediction model and the generated prediction scoring data;
the model training module is used for constructing and training a user-commodity preference prediction model based on stable neural collaborative filtering according to the preprocessed user-commodity scoring data;
the parameter control module is used for initializing parameters of the user-commodity preference prediction model and controlling the training process of the parameters and the user-commodity preference prediction model;
the prediction result generation module is used for generating unknown user-commodity prediction scoring data by utilizing the hidden feature vectors of the user and the commodity according to the user-commodity preference prediction model and storing the generated prediction scoring data into the data storage module;
and the data output module is used for outputting the hidden feature representation of the user and the commodity and the unknown user-commodity prediction scoring data.
2. The system of claim 1, wherein the model training module comprises a guidance model training unit, a guidance information generation unit, and a neural network training unit;
the guide model training unit is used for training a non-negative matrix decomposition model which is used as a guide model and does not contain noise according to the preprocessed user-commodity scoring data;
the guidance information generating unit is used for generating guidance information according to the non-negative matrix factorization model;
the neural network training unit is used for constructing and training a user-commodity preference prediction model based on stable neural collaborative filtering according to the generated guide information, wherein:
the user-commodity preference prediction model comprises a vectorization layer, a noise layer, a multi-layer perceptron layer and a fully-connected output layer which are connected in sequence.
3. The stable neural collaborative filtering based user-commodity preference prediction system according to claim 1, wherein the parameter control module comprises an initialization unit and a parameter control unit;
the initialization unit is used for initializing parameters of the user-commodity preference prediction model;
and the parameter control unit is used for transmitting parameters to the prediction result generation module and controlling the training process of the user-commodity preference prediction model.
4. The system of claim 1, wherein the prediction result generation module comprises a parameter receiving unit and a prediction result generation unit;
the parameter receiving unit is used for receiving the parameters transmitted by the parameter control module;
and the prediction result generation unit is used for predicting the unknown user-commodity scores by using the hidden feature vectors of the users and the commodities.
5. The user-commodity preference prediction method based on the stable neural collaborative filtering is characterized by comprising the following steps of:
s1, obtaining user-commodity scoring data;
s2, preprocessing the user-commodity scoring data and storing the preprocessed user-commodity scoring data;
s3, training a non-negative matrix decomposition model which is used as a guide model and does not contain noise according to the preprocessed user-commodity scoring data;
s4, generating guide information by using the non-negative matrix factorization model;
s5, constructing and training a user-commodity preference prediction model containing a noise layer and based on stable neural collaborative filtering according to the guide information;
s6, generating unknown user-commodity prediction scores by using the user-commodity preference prediction model according to the hidden feature vectors of the user and the commodities, and storing the user-commodity prediction scores;
and S7, outputting the implicit characteristic representation of the user and the commodity and the unknown user-commodity prediction score, thereby completing the prediction of the user-commodity preference.
6. The stable neural collaborative filtering-based user-commodity preference prediction method according to claim 5, wherein the step S3 includes the steps of:
s301, initializing parameters;
s302, constructing a target function according to the preprocessed user-commodity rating data;
s303, training the hidden feature matrixes of the user and the commodity by using a gradient descent method according to the target function, thereby finishing the training of a non-negative matrix decomposition model which is used as a guide model and does not contain noise, wherein the target function RSER(A)The expression of (a) is as follows:
Figure FDA0002385906340000031
Figure FDA0002385906340000032
Figure FDA0002385906340000033
wherein R is(A)Representing a set of user-to-commodity known rating data, R, in a user-to-commodity rating matrix Ru,iDenotes the user u's score, p, for item iuRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden features, lambda, corresponding to the ith commodity in the financial product hidden feature matrix QpAnd λQBoth represent regularization factors acting on hidden feature matrices P and Q of the user and the commodity, respectively, F represents a norm,
Figure FDA0002385906340000041
representing the predicted score, I, of user u on item I calculated by the modelu,UiRespectively representing a set of items interacted by the user u and a set of users interacted with the item i.
7. The method for predicting user-commodity preference based on stable neural collaborative filtering according to claim 5, wherein the expression of the guidance information h in the step S4 is as follows:
h＝emb(pu,qi,θG)
wherein p isuRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden features, theta, corresponding to the ith commodity in the financial product hidden feature matrix QGRepresents the parameters of vectorization, emb (-) represents the vectorization operation.
8. The stable neural collaborative filtering based user-commodity preference prediction method according to claim 5, wherein the step S5 includes the steps of:
s501, initializing parameters, and constructing a user-commodity preference prediction model which comprises a noise layer and is based on stable neural collaborative filtering;
s502, sequentially inputting the IDs and the side information of the user and the commodity into a vectorization layer and a noise layer of a user-commodity preference prediction model to obtain the hidden feature vectors of the user and the commodity, wherein the hidden feature vectors of the user and the commodity are pi (p)u,qi) The expression of (a) is as follows:
Figure FDA0002385906340000042
wherein e is Gaussian noise with mean 0 and variance δ, N (-) represents Gaussian distribution, puRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden feature corresponding to the ith commodity in the financial product hidden feature matrix Q, emb ((-)) representing vectorization operation,
Figure FDA0002385906340000043
an id representing the user u and a location id,
Figure FDA0002385906340000044
which represents the characteristics of the user u,
Figure FDA0002385906340000045
an id representing the number of the item i,
Figure FDA0002385906340000046
a feature representing item i;
s503, with the guide information as a target, performing feature extraction training on the hidden feature vectors of the user and the commodity, wherein the feature extraction training is L (theta)G，θr) The expression of (a) is as follows:
Figure FDA0002385906340000051
wherein h represents generated guidance information, emb (phi) represents vectorization operation, sigma (phi) represents activation function, b represents bias parameter, and thetarMapping vector p when hidden feature space dimensions in matrix decomposition model and neural network are differentuRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiRepresenting the hidden features, theta, corresponding to the ith commodity in the financial product hidden feature matrix QGRepresenting vectorized parameters, ru,iThe score of the user u on the commodity i is represented, and theta represents parameters of the user and the hidden feature vector of the commodity;
s504, the hidden feature vectors of the user and the commodity after feature extraction training are sequentially input into a multi-layer perceptron and a full-connection output layer of the user-commodity preference prediction model, and a neural network is trained on an objective function by using a random gradient descent method, so that the training of the user-commodity preference prediction model which comprises a noise layer and is based on stable neural collaborative filtering is completed.
9. The method for predicting user-commodity preference based on stable neural collaborative filtering according to claim 5, wherein the unknown user-commodity prediction score f (p) is generated in step S6u,qi) The expression of (a) is as follows:
f(pu,qi)＝φout(φX(...φ2(φ1(π(pu,qi)))...))
wherein phi isoutRepresenting a non-linear transformation of an output layer network in a multi-layer neural network, phiXRepresents the non-linear transformation of the X-th network in a multi-layer neural network, phi2Representing a non-linear transformation, phi, of a second-layer neural network in a multi-layer neural network1Representing a non-linear transformation, p, of a first layer of a multi-layer neural networkuRepresenting the hidden features corresponding to the u-th user in the user hidden feature matrix P, qiAnd representing the hidden feature corresponding to the ith commodity in the financial product hidden feature matrix Q.
CN202010098124.3A 2020-02-18 2020-02-18 User-commodity preference prediction system and method based on stable neural collaborative filtering Active CN111311324B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010098124.3A CN111311324B (en) 2020-02-18 2020-02-18 User-commodity preference prediction system and method based on stable neural collaborative filtering

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010098124.3A CN111311324B (en) 2020-02-18 2020-02-18 User-commodity preference prediction system and method based on stable neural collaborative filtering

Publications (2)

Publication Number Publication Date
CN111311324A true CN111311324A (en) 2020-06-19
CN111311324B CN111311324B (en) 2022-05-20

Family

ID=71158272

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010098124.3A Active CN111311324B (en) 2020-02-18 2020-02-18 User-commodity preference prediction system and method based on stable neural collaborative filtering

Country Status (1)

Country Link
CN (1) CN111311324B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112541639A (en) * 2020-12-22 2021-03-23 宜宾电子科技大学研究院 Recommendation system scoring prediction method based on graph neural network and attention mechanism
CN112862008A (en) * 2021-03-29 2021-05-28 中信银行股份有限公司 Training method of preference prediction model and prediction method of user preference
CN113112328A (en) * 2021-04-13 2021-07-13 华南农业大学 Commodity recommendation method based on matrix transformation and decomposition of point-cut set graph segmentation

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130226839A1 (en) * 2012-02-27 2013-08-29 Xerox Corporation Robust bayesian matrix factorization and recommender systems using same
CN108874960A (en) * 2018-06-06 2018-11-23 电子科技大学 Curriculum video proposed algorithm based on noise reduction self-encoding encoder mixed model in a kind of on-line study
CN108921670A (en) * 2018-07-04 2018-11-30 重庆大学 A kind of potential interest of fusion user, the Drug trading recommended method of space-time data and classification popularity
CN109543109A (en) * 2018-11-27 2019-03-29 山东建筑大学 A kind of proposed algorithm of time of fusion window setting technique and score in predicting model
CN109740655A (en) * 2018-12-26 2019-05-10 西安电子科技大学 Article score in predicting method based on matrix decomposition and neural collaborative filtering
CN109785062A (en) * 2019-01-10 2019-05-21 电子科技大学 A kind of hybrid neural networks recommender system based on collaborative filtering model
CN110390561A (en) * 2019-07-04 2019-10-29 四川金赞科技有限公司 User-financial product of stochastic gradient descent is accelerated to select tendency ultra rapid predictions method and apparatus based on momentum
CN110413878A (en) * 2019-07-04 2019-11-05 四川金赞科技有限公司 User based on adaptive elastomeric network-commodity preference prediction meanss and method

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130226839A1 (en) * 2012-02-27 2013-08-29 Xerox Corporation Robust bayesian matrix factorization and recommender systems using same
CN108874960A (en) * 2018-06-06 2018-11-23 电子科技大学 Curriculum video proposed algorithm based on noise reduction self-encoding encoder mixed model in a kind of on-line study
CN108921670A (en) * 2018-07-04 2018-11-30 重庆大学 A kind of potential interest of fusion user, the Drug trading recommended method of space-time data and classification popularity
CN109543109A (en) * 2018-11-27 2019-03-29 山东建筑大学 A kind of proposed algorithm of time of fusion window setting technique and score in predicting model
CN109740655A (en) * 2018-12-26 2019-05-10 西安电子科技大学 Article score in predicting method based on matrix decomposition and neural collaborative filtering
CN109785062A (en) * 2019-01-10 2019-05-21 电子科技大学 A kind of hybrid neural networks recommender system based on collaborative filtering model
CN110390561A (en) * 2019-07-04 2019-10-29 四川金赞科技有限公司 User-financial product of stochastic gradient descent is accelerated to select tendency ultra rapid predictions method and apparatus based on momentum
CN110413878A (en) * 2019-07-04 2019-11-05 四川金赞科技有限公司 User based on adaptive elastomeric network-commodity preference prediction meanss and method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
YALI DU ET AL.: "Enhancing the Robustness of Neural Collaborative Filtering Systems Under Malicious Attacks", 《 IEEE TRANSACTIONS ON MULTIMEDIA》 *
伊华伟: "基于模糊核聚类和支持向量机的鲁棒协同推荐算法", 《电子与信息学报》 *
蒋伟: "推荐系统若干关键技术研究", 《中国优秀博硕士学位论文全文数据库(博士)信息科技辑》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112541639A (en) * 2020-12-22 2021-03-23 宜宾电子科技大学研究院 Recommendation system scoring prediction method based on graph neural network and attention mechanism
CN112862008A (en) * 2021-03-29 2021-05-28 中信银行股份有限公司 Training method of preference prediction model and prediction method of user preference
CN113112328A (en) * 2021-04-13 2021-07-13 华南农业大学 Commodity recommendation method based on matrix transformation and decomposition of point-cut set graph segmentation

Also Published As

Publication number Publication date
CN111311324B (en) 2022-05-20

Similar Documents

Publication Publication Date Title
CN111311324B (en) User-commodity preference prediction system and method based on stable neural collaborative filtering
CN110390561B (en) User-financial product selection tendency high-speed prediction method and device based on momentum acceleration random gradient decline
CN111753881B (en) Concept sensitivity-based quantitative recognition defending method against attacks
CN113408743A (en) Federal model generation method and device, electronic equipment and storage medium
CN111414749B (en) Social text dependency syntactic analysis system based on deep neural network
CN113449783B (en) Countermeasure sample generation method, system, computer device and storage medium
US20220237465A1 (en) Performing inference and signal-to-noise ratio based pruning to train sparse neural network architectures
CN113033822A (en) Antagonistic attack and defense method and system based on prediction correction and random step length optimization
CN112085050A (en) Antagonistic attack and defense method and system based on PID controller
CN112434213A (en) Network model training method, information pushing method and related device
Bu et al. Differentially private optimization on large model at small cost
CN111178986B (en) User-commodity preference prediction method and system
CN110837603A (en) Integrated recommendation method based on differential privacy protection
CN114330580A (en) Robust knowledge distillation method based on ambiguity-oriented mutual label updating
CN112926661A (en) Method for enhancing image classification robustness
CN113935496A (en) Robustness improvement defense method for integrated model
Lin et al. Differential privacy protection over deep learning: An investigation of its impacted factors
CN115834251B (en) Hypergraph-transform-based threat hunting model building method
CN112214668B (en) Personalized financial service recommendation device and method based on big data
CN115510986A (en) Countermeasure sample generation method based on AdvGAN
CN113935913A (en) Black box image confrontation sample generation method with visual perception concealment
CN114187966A (en) Single-cell RNA sequence missing value filling method based on generation countermeasure network
CN111179070A (en) Loan risk timeliness prediction system and method based on LSTM
CN117540791B (en) Method and device for countermeasure training
CN117454187B (en) Integrated model training method based on frequency domain limiting target attack

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant