CN115033780A - Privacy protection cross-domain recommendation system based on federal learning - Google Patents
Privacy protection cross-domain recommendation system based on federal learning Download PDFInfo
- Publication number
- CN115033780A CN115033780A CN202210484040.2A CN202210484040A CN115033780A CN 115033780 A CN115033780 A CN 115033780A CN 202210484040 A CN202210484040 A CN 202210484040A CN 115033780 A CN115033780 A CN 115033780A
- Authority
- CN
- China
- Prior art keywords
- domain
- user
- global
- local
- embedding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 claims abstract description 35
- 230000002776 aggregation Effects 0.000 claims abstract description 26
- 238000004220 aggregation Methods 0.000 claims abstract description 26
- 230000003993 interaction Effects 0.000 claims abstract description 25
- 230000008569 process Effects 0.000 claims abstract description 22
- 230000007246 mechanism Effects 0.000 claims abstract description 17
- 238000004891 communication Methods 0.000 claims abstract description 16
- 230000000737 periodic effect Effects 0.000 claims abstract description 11
- 238000013508 migration Methods 0.000 claims abstract description 10
- 230000005012 migration Effects 0.000 claims abstract description 10
- 230000006870 function Effects 0.000 claims description 15
- 230000005540 biological transmission Effects 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 claims description 6
- 230000014759 maintenance of location Effects 0.000 claims description 6
- 238000012546 transfer Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 3
- 230000006399 behavior Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000002452 interceptive effect Effects 0.000 claims description 3
- 238000012423 maintenance Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Bioethics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Data Mining & Analysis (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a privacy protection cross-field recommendation system based on federal learning, which comprises the following steps: s1: private updating in a single domain, namely fusing user global and local preferences through a graph migration module, and locally updating the global and local preferences by a gradient descent algorithm based on data in the domain; s2: after private updating, collaborative learning of global user preferences based on multi-domain data is performed by adopting a federal updating process, and the global preferences are adapted to heterogeneous domain data through personalized aggregation; s3: the use of a periodic synchronization mechanism reduces communication costs. The privacy protection cross-domain recommendation system based on federal learning provided by the application protects user privacy information from multiple layers. In the invention, the original interaction data of each domain is locally stored in the private space of the domain and is not uploaded to other domains, so that the risk of privacy disclosure can be effectively reduced.
Description
Technical Field
The invention relates to the field of artificial intelligence natural language processing and the field of recommendation systems, in particular to a privacy protection cross-field recommendation system based on federal learning.
Background
In modern recommendation systems, it has become a technical trend to develop multi-domain recommendation services to meet different user requirements. In order to improve multi-domain services of information systems, cross-domain recommendation is increasingly concerned by research and industry, and the purpose of the problem is to utilize useful information in other domains to improve the recommendation quality in target domains. A typical cross-domain recommendation method is to establish a connection between different domains mainly by overlapping users/items and to transfer useful information between these domains. In this study we are concerned with collaborative cross-domain recommendations without auxiliary interaction data, which tend to apply classical machine learning techniques, either directly share or indirectly map user/item embedding, or capture cross-domain common patterns.
Existing cross-domain recommendation methods, while effective, typically rely on a strong assumption that all or part of the user item interaction data can be accessed between different domains. However, this assumption may not be realistic in practice due to business competition and privacy issues. For example, application data from different domains often belong to different companies or departments, and these data cannot be directly shared. Furthermore, recent data protection legislation, such as GDPR and the like, places severe restrictions on the storage and sharing of privacy-sensitive user data. The above privacy and security issues severely limit the practical application of existing cross-domain recommendation methods. Recently, some studies propose cross-domain recommendation models that consider privacy, but these approaches either ignore the heterogeneity of different domain data or fail to achieve consistent improvements for all users. Aiming at the defects of the existing cross-domain recommendation method, the invention provides a privacy protection cross-domain recommendation system based on federal learning.
The information disclosed in this background section is only for enhancement of understanding of the general background of the invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person skilled in the art.
Disclosure of Invention
The invention aims to provide a privacy protection cross-domain recommendation system based on federal learning. Specifically, for each domain, a domain-specific user-item interaction graph (user-entry interaction graph) is constructed according to user interaction data of the domain, global and local user nodes are set to model global and local user preferences, and edges connecting the global and local user nodes are added. Based on the user-project diagram, the invention designs a federal cross-domain recommendation model. In order to learn cross-domain recommendation knowledge in a privacy-protecting manner, each training iteration of the cross-domain recommendation model provided by the invention is composed of a private updating process in a local domain and a federal updating process across multiple domains. In the private updating process, a Graph transfer module (Graph transfer module) is designed for each domain to perform bidirectional message exchange and propagation, so that the global preference and the local preference of a user are fused. Then, during the federal update, each domain applies Local Differential Privacy (LDP) techniques to the learned global user preferences and shares them to other domains. Meanwhile, each domain receives global user preference from other domains, and then the global user preference is updated locally through the personalized aggregation strategy, so that the self-adaption of the user preference in a specific domain is realized. Therefore, the cross-domain recommendation model provided by the invention can effectively approach a multi-domain training process, and the process directly shares local interactive data in a privacy protection mode. In addition, the invention provides a periodic synchronization mechanism to reduce communication cost brought by learning of cross-domain global preference.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a privacy protection cross-field recommendation system based on federal learning, which comprises the following steps:
s1: private updating in a single domain, namely fusing user global and local preferences through a graph migration module, and locally updating the global and local preferences by a gradient descent algorithm based on data in the domain;
s2: after private updating, collaborative learning of global user preferences based on multi-domain data is performed by adopting a federal updating process, and the global preferences are adapted to heterogeneous domain data through personalized aggregation;
s3: the use of a periodic synchronization mechanism reduces communication costs.
As a further technical solution, the graph migration module in step S1 specifically is:
for each domain, constructing a domain-specific user-item interaction graph (as shown in FIG. 3) according to the user interaction data of the domain, setting global and local user nodes to model global and local user preferences, and adding edges connecting the global and local user nodes; wherein, the global user node and embedding (embedding) of the user u
Associated, original user node of user u with embedded e u Is associated with e u And
all vectors are vectors with dimension m in real number space; the graph transmission module can be abstracted into L-layer conversion; at the l-th level, a bi-directional embedded transport is first applied to exchange messages between local and global user preferences, which is calculated in the following way:
wherein,
f T (. beta.) is the transfer function, beta 1 And beta 2 Is at [0,1 ]]An over-parameter controlling the retention rate in the transmission in the range,
represents the neighbor set of user u in the diagram;
and
respectively representing a local and global representation of user u before transmission, wherein the local user representation
Before being sent to the l +1 layer, the message is further updated in the message propagation, and the specific operation of the message propagation is as follows:
wherein,
representing the neighbor set of item i in the graph on the user item interaction graph, and item representation
By embedding e in learning i ∈R m To initialize; after L-layer transformation in the graph transport module, we concatenate the representations generated by all L layers, resulting in the final user representation and item representation as follows:
wherein Concat () represents a join operation; final user representation h u Global and local user preferences have been encoded, as well as high-level information on the user project graph.
As a further technical solution, the step S1 of locally updating global and local preferences by using a gradient descent algorithm based on intra-domain data specifically includes:
given user and item representations, an inner product operation is adopted to generate a score so as to predict the interaction possibility of the user u and the item i from the d domain, and the specific calculation mode is as follows:
wherein
Predicted scores for user u and item i; then updating local and global user embedding based on single-domain interaction data by adopting Bayes personalized sorting loss, wherein the definition is as follows:
where σ () is the Sigmoid function, λ controls the strength of L2 regularization, Θ d Is a model parameter for domain d; a training sample pair is constructed by negative sampling,
representing a set of paired training data; in this way, the model learns locally for each domain local user embedding and domain specific item embedding based on the data within the domain.
As a further technical solution, after the private update in step S2, the collaborative learning of the global user preference based on the multi-domain data by using the federal update process specifically includes:
in the private update process, each domain d is a global user
Maintenance insert
The global user of (2), the user
Local updating is carried out by using data in the domain; to characterize more comprehensive user preferences, it is necessary to learn cross-domain knowledge to enhance local user preferences; to this end, decentralized federated learning is employed to cooperatively update global user embedding based on data of multiple domains; these global user embeddings will be multipleSharing among the individual domains; however, these users embed private information that contains the user's behavior and cannot be shared directly outside the domain for privacy reasons; applying a local differential privacy technique to global user embedding prior to shared global user embedding; in particular, for each global user
Addition intensity of lambda LDP To obtain cryptographically protected embedding
Then, each domain d sends a protected embedding
To other domains while accepting shared embedding from other domains
As a further technical solution, the adapting, in step S2, the global preferences to the heterogeneous domain data through personalized aggregation specifically includes:
in a cross-domain recommendation scene, in order to consider the domain adaptability of cross-domain knowledge utilization, a personalized preference aggregation strategy is adopted to generate a domain-specific global user embedding for each domain; specifically, personalized preference aggregation is performed in each domain based on an attention mechanism; when domain name d accepts shared embedding from other domains
The personalized aggregation layer adopts a learnable transformation matrix W ∈ R 2m And calculating the attention coefficient in a specific calculation mode of:
wherein g (-) is LeakyReLU activation function, and softmax function is adopted to couple attention coefficientsCarrying out normalization; attention coefficient alpha d,d′ Representing the importance of the domain knowledge to the domain d; these attention coefficients are used as weights for personalized federated aggregation to generate global embedding of user u in d-domain, in the following way:
wherein, beta F Is at [0,1 ]]The over-parameters of the retention rate are controlled within the range, and the second item of the equation is self-adaptive and combines knowledge from different fields; for the user
Generated global user inlining
Private updates will be made by the graph transmission module in the next round of training.
As a further technical solution, step S3 specifically includes:
when each domain receives a protected global user embedding shared by other domains, applying personalized preference aggregation to generate a domain-specific global user embedding; then, given that these global user embeddings and local user embeddings are maintained locally, the private update process is first performed a number of times to optimize the local BPR penalty function, and then a global update is invoked.
By adopting the technical scheme, the invention has the following beneficial effects:
compared with the existing cross-domain recommendation system, the privacy protection cross-domain recommendation system based on the federal learning provided by the invention protects the user privacy information from multiple layers. Firstly, in the method, the original interaction data of each domain is locally stored in the private space of the domain and is not uploaded to other domains, so that the risk of privacy disclosure can be effectively reduced. Secondly, according to data processing inequality, the only data of inter-domain communication is user global embedding, and the private information contained in the data is much less than the original interactive data; furthermore, the user global embedding encodes the user's global preferences across domains, rather than domain-specific preferences, making upload global embedding more secure. Third, user global embedding is updated based on a set of user interactions rather than a single user interaction, which makes it difficult to recover a particular interaction history. Fourthly, a Local Differential Privacy (LDP) technology is applied to user global embedding, zero-mean Laplacian noise is added to the user global embedding, and the difficulty of deducing privacy information is increased. Therefore, the method provided by the invention can be used for modeling the global preference of the user for recommendation on the premise of not revealing the privacy of the user, and is safer compared with the existing cross-domain recommendation system.
Compared with the existing recommendation system considering privacy, the method provided by the invention learns uniform cross-domain information for all the fields, and the information in different fields is personalized and aggregated by means of an attention mechanism, so that each field can be subjected to field-specific adaptation according to the received global user preference. This domain-specific adaptation is necessary due to the heterogeneity of data across domains in the real world. In addition, the periodic synchronization mechanism provided by the invention reduces the communication cost, and the mechanism executes a cross-domain federal update process after a fixed number of private updates. Compared with the existing cross-domain recommendation system considering privacy, the method provided by the invention is lower in communication cost.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is an overall architecture diagram of the federated learning-based privacy preserving cross-domain recommendation system of the present invention;
FIG. 2 is a flow chart of the algorithm in an update cycle within a single domain according to the present invention;
FIG. 3 is a domain-specific user-item interaction diagram constructed by the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The following describes in detail embodiments of the present invention with reference to the drawings. It should be understood that the detailed description and specific examples, while indicating the present invention, are given by way of illustration and explanation only, not limitation.
With reference to fig. 1-2, the present embodiment provides a privacy protection cross-domain recommendation system based on federal learning, including the following steps:
s1: private updating in a single domain, namely fusing user global and local preferences through a graph migration module, and locally updating the global and local preferences by a gradient descent algorithm based on data in the domain; the main purpose of the private update process is to locally fuse the global and local preferences of the user and update them based on the intra-domain data to capture domain-specific knowledge. Specifically, the invention provides a graph migration module to fuse user global and local preferences and locally update the global and local preferences with a gradient descent algorithm based on intra-domain data.
S2: after private updating, collaborative learning of global user preferences based on multi-domain data is performed by adopting a federal updating process, and the global preferences are adapted to heterogeneous domain data through personalized aggregation;
s3: the use of a periodic synchronization mechanism reduces communication costs.
As a further technical solution, the graph migration module in step S1 specifically is:
for each domain, a domain-specific user-item interaction graph (as shown in FIG. 3) is constructed based on the domain's user interaction data, global and local user nodes are set to model global and local user preferences, and links are addedEdges of local and local user nodes; wherein, the global user node and embedding (embedding) of the user u
Associated, original user node of user u with embedded e u Is associated with e u And
all vectors are vectors with dimension m in real number space; the graph transport module can be abstracted as an L-layer conversion; at the l-th level, a bi-directional embedded transport is first applied to exchange messages between local and global user preferences, which is calculated in the following way:
wherein,
f T (. beta.) is the transfer function, beta 1 And beta 2 Is at [0,1 ]]Model (A) of
A hyper-parameter that controls the retention rate in transmission within the enclosure,
represents the neighbor set of user u in the diagram;
and
respectively representing the user u's own book before transmissionGround and global representations, where local user represents
Before being sent to the l +1 layer, the message is further updated in the message propagation, and the specific operation of the message propagation is as follows:
wherein,
representing the neighbor set of item i in the graph on the user item interaction graph, and item representation
By embedding e in learning i ∈R m To initialize; after L-layer transformation in the graph transport module, we concatenate the representations generated by all L layers, resulting in the final user representation and item representation as follows:
wherein Concat () represents a join operation; final user representation h u Global and local user preferences have been encoded, as well as high-level information on the user project graph.
As a further technical solution, the step S1 of locally updating global and local preferences by using a gradient descent algorithm based on intra-domain data specifically includes:
given user and item representations, an inner product operation is adopted to generate a score so as to predict the interaction possibility of the user u and the item i from the d domain, and the specific calculation mode is as follows:
wherein
Predicted scores for user u and item i; then adopts BayesThe si-personalized ranking penalty, updating local and global user embedding based on single domain interaction data, is defined as:
where σ () is the Sigmoid function, λ controls the strength of L2 regularization, Θ d Is a model parameter for domain d; a training sample pair is constructed by negative sampling,
representing a set of paired training data; in this way, the model learns locally for each domain local user embedding and domain specific item embedding based on the data within the domain.
As a further technical solution, after the private update in step S2, the collaborative learning of the global user preference based on the multi-domain data by using the federal update process specifically includes:
in the private update process, each domain d is a global user
Maintenance insert
The global user of (2), the user
Local updating is carried out by using data in the domain; to characterize more comprehensive user preferences, it is necessary to learn cross-domain knowledge to enhance local user preferences; to this end, decentralized federated learning is employed to cooperatively update global user embedding based on data of multiple domains; these global user embeddings will be shared among multiple domains; however, these users embed private information that contains the user's behavior and cannot be shared directly outside the domain for privacy reasons; applying a local differential privacy technique to global user embedding prior to shared global user embedding; in particular, for each global user
The addition strength is lambda LDP To obtain cryptographically protected embedding
Then, each domain d sends a protected embedding
To other domains while accepting shared embedding from other domains
As a further technical solution, the adapting, in step S2, the global preferences to the heterogeneous domain data through personalized aggregation specifically includes:
in a cross-domain recommendation scene, in order to consider the domain adaptability of cross-domain knowledge utilization, a personalized preference aggregation strategy is adopted to generate a domain-specific global user embedding for each domain; specifically, personalized preference aggregation is performed in each domain based on an attention mechanism; when domain name d accepts shared embedding from other domains
The personalized aggregation layer adopts a learnable transformation matrix W ∈ R 2m And calculating the attention coefficient in a specific calculation mode of:
in the formula, g (-) is an LeakyReLU activation function, and the attention coefficient is normalized by adopting a softmax function; attention coefficient α d,d′ Representing the importance of the domain knowledge to the domain d; these attention coefficients are used as weights for personalized federated aggregation to generate global embedding of user u in d-domain, in the following way:
wherein beta is F Is at [0,1 ]]Controlling the over-parameter of the retention rate in the range, and combining the second term of the equation with knowledge from different fields in a self-adaptive manner; for the user
Generated global user inlining
Private updates will be made by the graph transmission module in the next round of training.
As a further technical solution, step S3 specifically includes:
in order to reduce the communication cost, the invention additionally provides a periodic synchronization mechanism to reduce the communication cost. And (4) periodic synchronization. In decentralized federated learning, communication bandwidth is a major bottleneck, because clients attempt to pass their local update information to other clients, and for this reason the present invention proposes a periodic synchronization mechanism to reduce communication costs, which performs a cross-domain federated update process after a fixed number of private updates. In particular, when each domain receives a protected global user embedding shared by other domains, personalized preference aggregation is applied to generate domain-specific global user embedding. Then, given that these global user embeddings and local user embeddings are maintained locally, the private update process is first performed T times to optimize the local BPR penalty function, and then a global update is invoked.
In summary, the present invention at least includes the following inventions:
1. a graph migration module: on the basis of the user project interaction graph with the global user nodes and the local user nodes, the graph migration module expands the message transmission scheme of the traditional graph neural network, and simultaneously considers the following steps: (1) message exchange between global and local user preferences; (2) message propagation on user-project graphs.
2. Personalized aggregation module of global user preferences: to account for domain adaptability across domain knowledge utilization, a personalized aggregation module performs personalized preference aggregation in each domain based on an attention mechanism, generating domain-specific global user embedding for each domain.
3. A periodic synchronization mechanism: in order to reduce the communication cost, the invention provides a periodic synchronization mechanism to reduce the communication cost, and the mechanism executes a cross-domain federal update process after a fixed number of private updates, thereby reducing the communication frequency and further reducing the communication cost.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and these modifications or substitutions do not depart from the spirit of the corresponding technical solutions of the embodiments of the present invention.
Claims (6)
1. A privacy protection cross-domain recommendation system based on federal learning is characterized by comprising the following steps:
s1: private updating in a single domain, namely fusing user global and local preferences through a graph migration module, and locally updating the global and local preferences by a gradient descent algorithm based on data in the domain;
s2: after private updating, collaborative learning of global user preferences based on multi-domain data is performed by adopting a federal updating process, and the global preferences are adapted to heterogeneous domain data through personalized aggregation;
s3: the use of a periodic synchronization mechanism reduces communication costs.
2. The federated learning-based privacy protection cross-domain recommendation system according to claim 1, wherein the graph migration module in step S1 is specifically:
for each domain, according to the fieldDomain-specific user-project interaction graphs are constructed, global and local user nodes are set to model global and local user preferences, and edges connecting the global and local user nodes are added; wherein, the global user node and the embedding of the user u
Associated, original user node of user u with embedded e u Is associated with e u And
all vectors are vectors with dimension m in real number space; the graph transmission module can be abstracted into L-layer conversion; at the l-th level, a bi-directional embedded transport is first applied to exchange messages between local and global user preferences, which is calculated in the following way:
wherein,
f T (. beta.) is the transfer function, beta 1 And beta 2 Is at [0,1 ]]Model (A) of
A hyper-parameter that controls the retention rate in transmission within the enclosure,
represents the neighbor set of user u in the diagram;
and
respectively representing a local and global representation of user u before transmission, wherein the local user representation
Before being sent to the l +1 layer, the message is further updated in the message propagation, and the specific operation of the message propagation is as follows:
wherein,
representing the neighbor set of the item i in the graph on the user item interaction graph, the item represents
By embedding e in learning i ∈R m To initialize; after L-layer transformation in the graph transport module, we concatenate the representations generated by all L layers, resulting in the final user representation and item representation as follows:
wherein Concat () represents a join operation; final user representation h u Global and local user preferences have been encoded, as well as high-level information on the user project graph.
3. The federated learning-based privacy-preserving cross-domain recommendation system according to claim 2, wherein the local updating of global and local preferences with a gradient descent algorithm based on intra-domain data in step S1 is specifically:
given user and item representations, an inner product operation is adopted to generate a score so as to predict the interaction possibility of the user u and the item i from the d domain, and the specific calculation mode is as follows:
wherein
Predicted scores for user u and item i; then, updating local and global user embedding based on single-domain interactive data by adopting Bayes personalized sorting loss, wherein the updating is defined as follows:
where σ (-) is the Sigmoid function, λ controls the strength of L2 regularization, Θ d Is a model parameter for domain d; a training sample pair is constructed by negative sampling,
representing a set of paired training data; in this way, the model learns locally per-domain local user embedding and domain-specific item embedding based on intra-domain data.
4. The privacy-preserving cross-domain recommendation system based on federated learning according to claim 1, wherein after the private update in step S2, the collaborative learning of the multi-domain data-based global user preferences using the federated update process is specifically:
in the private update process, each domain d is a global user
Maintenance insert
The global user of (2), the user
Local updating is carried out by using data in the domain; to characterize more comprehensive user preferences, it is necessary to learn cross-domain knowledge to enhance local user preferences; to this end, decentralized federated learning is employed to cooperatively update global user embedding based on data of multiple domains; these global user inlays will be shared among multiple domains; however, these users embed private information that contains the user's behavior and cannot be shared directly outside the domain for privacy reasons; applying a local differential privacy technique to global user embedding prior to shared global user embedding; in particular, for each global user
Addition intensity of lambda 3DP To obtain cryptographically protected embedding
Then, each domain d sends a protected embedding
To other domains while accepting shared embedding from other domains
5. The system according to claim 1, wherein the step S2 of adapting the global preferences to the heterogeneous domain data through personalized aggregation is specifically:
in a cross-domain recommendation scene, in order to consider the domain adaptability of cross-domain knowledge utilization, a personalized preference aggregation strategy is adopted to generate domain-specific global user embedding for each domain; specifically, personalized preference aggregation is performed in each domain based on an attention mechanism; when domain name d accepts shared embedding from other domains
The personalized aggregation layer adopts a learnable transformation matrix W ∈ R 2m And calculating the attention coefficient in a specific calculation mode of:
in the formula, g (-) is an LeakyReLU activation function, and the attention coefficient is normalized by adopting a softmax function; attention coefficient α d,d′ Representing the importance of the domain knowledge to the domain d; these attention coefficients are used as weights for personalized federated aggregation to generate global embedding of user u in d-domain, in the following way:
wherein, beta 3 Is at [0,1 ]]Controlling the over-parameter of the retention rate in the range, and combining the second term of the equation with knowledge from different fields in a self-adaptive manner; for the user
Generated global user inlining
Private updates will be made by the graph transmission module in the next round of training.
6. The federated learning-based privacy protection cross-domain recommendation system according to claim 1, wherein step S3 specifically is:
when each domain receives a protected global user embedding shared by other domains, applying personalized preference aggregation to generate a domain-specific global user embedding; then, given that these global user embeddings and local user embeddings are maintained locally, the private update process is first performed M times to optimize the local BPR penalty function, and then a global update is invoked.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210484040.2A CN115033780A (en) | 2022-05-06 | 2022-05-06 | Privacy protection cross-domain recommendation system based on federal learning |
| CN202211119135.0A CN115510316A (en) | 2022-05-06 | 2022-09-13 | Privacy protection cross-domain recommendation system based on federal representation learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210484040.2A CN115033780A (en) | 2022-05-06 | 2022-05-06 | Privacy protection cross-domain recommendation system based on federal learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115033780A true CN115033780A (en) | 2022-09-09 |
Family
ID=83119278
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210484040.2A Pending CN115033780A (en) | 2022-05-06 | 2022-05-06 | Privacy protection cross-domain recommendation system based on federal learning |
| CN202211119135.0A Pending CN115510316A (en) | 2022-05-06 | 2022-09-13 | Privacy protection cross-domain recommendation system based on federal representation learning |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211119135.0A Pending CN115510316A (en) | 2022-05-06 | 2022-09-13 | Privacy protection cross-domain recommendation system based on federal representation learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN115033780A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115987694A (en) * | 2023-03-20 | 2023-04-18 | 杭州海康威视数字技术股份有限公司 | Equipment privacy protection method, system and device based on multi-domain federation |
| CN116361561A (en) * | 2023-05-30 | 2023-06-30 | 安徽省模式识别信息技术有限公司 | Distributed cross-border service recommendation method and system based on variational reasoning |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116016212B (en) * | 2022-12-26 | 2024-06-04 | 电子科技大学 | Decentralised federation learning method and device for bandwidth perception |
-
2022
- 2022-05-06 CN CN202210484040.2A patent/CN115033780A/en active Pending
- 2022-09-13 CN CN202211119135.0A patent/CN115510316A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115987694A (en) * | 2023-03-20 | 2023-04-18 | 杭州海康威视数字技术股份有限公司 | Equipment privacy protection method, system and device based on multi-domain federation |
| CN115987694B (en) * | 2023-03-20 | 2023-06-27 | 杭州海康威视数字技术股份有限公司 | Multi-domain federation-based device privacy protection method, system and device |
| CN116361561A (en) * | 2023-05-30 | 2023-06-30 | 安徽省模式识别信息技术有限公司 | Distributed cross-border service recommendation method and system based on variational reasoning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115510316A (en) | 2022-12-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115033780A (en) | Privacy protection cross-domain recommendation system based on federal learning | |
| CN113905391B (en) | Integrated learning network traffic prediction method, system, equipment, terminal and medium | |
| Györgyi | Inference of a rule by a neural network with thermal noise | |
| CN112256980A (en) | Dynamic graph attention network-based multi-relation collaborative filtering recommendation | |
| CN109033107A (en) | Image search method and device, computer equipment and storage medium | |
| EP4320556A1 (en) | Privacy-aware pruning in machine learning | |
| CN111813539A (en) | Edge computing resource allocation method based on priority and cooperation | |
| CN114896899A (en) | Multi-agent distributed decision method and system based on information interaction | |
| CN114358250A (en) | Data processing method, data processing apparatus, computer device, medium, and program product | |
| Dang et al. | Combined relay selection enabled by supervised machine learning | |
| CN108921281A (en) | A kind of field adaptation method based on depth network and countermeasure techniques | |
| Ahmed et al. | 5G-empowered drone networks in federated and deep reinforcement learning environments | |
| Huang et al. | Collective reinforcement learning based resource allocation for digital twin service in 6G networks | |
| WO2024139666A1 (en) | Training method and apparatus for dual-target domain recommendation model | |
| CN116226540B (en) | End-to-end federation personalized recommendation method and system based on user interest domain | |
| CN116367190A (en) | Digital twin function virtualization method for 6G mobile network | |
| CN113645702B (en) | Internet of things system supporting block chain and optimized by strategy gradient technology | |
| CN113572647B (en) | Block chain-edge calculation combined system based on reinforcement learning | |
| CN115168722A (en) | Content interaction prediction method and related equipment | |
| CN115204374A (en) | System, method and device for federated learning and prediction based on segmented neural networks | |
| CN115238282A (en) | Distributed network architecture and method for realizing ABAC attribute mining based on edge calculation under distributed network architecture | |
| CN114723012A (en) | Computing method and device based on distributed training system | |
| Tran et al. | Privacy-preserving learning models for communication: A tutorial on advanced split learning | |
| CN112926738A (en) | Method, equipment and storage medium for designing neural network adaptive output layer | |
| Gao et al. | Wide & ResNet: An improved network for CTR prediction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20220909 |
|
| WD01 | Invention patent application deemed withdrawn after publication |