CN115860053A - Label recommendation method and system based on parameter anti-attack metric learning - Google Patents

Label recommendation method and system based on parameter anti-attack metric learning Download PDF

Info

Publication number
CN115860053A
CN115860053A CN202211603826.8A CN202211603826A CN115860053A CN 115860053 A CN115860053 A CN 115860053A CN 202211603826 A CN202211603826 A CN 202211603826A CN 115860053 A CN115860053 A CN 115860053A
Authority
CN
China
Prior art keywords
user
item
label
representation
tag
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211603826.8A
Other languages
Chinese (zh)
Inventor
费正顺
王经龙
周昊天
辛凯
李雅玄
章锦
陈贵
刘康玲
项新建
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Lover Health Science and Technology Development Co Ltd
Original Assignee
Zhejiang Lover Health Science and Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Lover Health Science and Technology Development Co Ltd filed Critical Zhejiang Lover Health Science and Technology Development Co Ltd
Priority to CN202211603826.8A priority Critical patent/CN115860053A/en
Publication of CN115860053A publication Critical patent/CN115860053A/en
Pending legal-status Critical Current

Links

Images

Abstract

The invention discloses a label recommendation method and a system based on parameter anti-attack metric learning, wherein the method comprises the following steps: acquiring user, item and label identification; converting the user, item and label identifications into vectors to generate user, item and label representations; generating user, item and label confrontation disturbance; generating a confrontation user, a confrontation item, and a confrontation tag representation; representing the confrontation user, the confrontation item and the confrontation label to generate a confrontation potential relation vector of a user-label and an item-label and generate a potential relation vector of the user-label and the item-label; modeling distance measures from the user-tag and item-tag potential relationship vectors to user, item, and tag representations; modeling countermeasure distance metrics by representing the user-label and item-label countermeasure potential relationship vectors with countermeasure users, countermeasure items and countermeasure labels, and returning to a label recommendation list; jointly training a distance metric and an confrontation distance metric; updating parameters and countering disturbances.

Description

Label recommendation method and system based on parameter anti-attack metric learning
Technical Field
The invention belongs to the technical field of information retrieval, and particularly relates to a label recommendation method and system based on parameter counterattack metric learning.
Background
In the internet era containing a lot of information, tags are important tools for information retrieval, which can help users to classify and retrieve related resources, and large-scale applications at home and abroad such as LastFm, movieLens, naobao, tremble, kyoto, etc. use keywords to annotate songs, videos, books, products and other network resources, so-called tags. In addition to annotating items, tags are also useful for solving practical problems such as image recommendation tasks, interest discovery, and content search. As the availability of tags in various fields is increased, tag recommendation technology has become a key technology to help users more conveniently retrieve their desired network resources. The content management and the internet resource search are more and more important, the proper label is set, the effective classification can be made for the internet resource, and the experience of a user on the information index service is improved.
Traditional tag recommendation methods primarily recommend based on historical interaction data of users, items, and tags, such methods primarily focus on implicit feedback, as implicit interactions can generally predict the interests of users in greater numbers and at a lower cost. The main recommendation method depends on a tensor decomposition technology, however, the tensor decomposition technology cannot model distance measurement between data because inner products are used to violate triangle inequality, and the performance of tag recommendation is often affected by data sparseness and cold start. In addition, the graph-based approach may model the tag recommendation system as a three-part graph, pass messages along edges, and learn to summarize a representation of neighborhood information for each node. The nodes of the graph may be users, documents, or tags. The weight of one node can be propagated to other nodes through co-occurring connections, however, the entire algorithm must run online and the entire graph must be traversed for each user-document pair queried, and therefore the model is very time consuming. The existing method lacks attention to the data similarity problem and is not beneficial to mining the implicit information of the data. Meanwhile, data samples are often subjected to noise interference, so that the difference between test data and training data is large, and the model is often difficult to obtain good generalization capability.
Therefore, how to obtain similarity relationships among users, items and tags to build a reliable tag recommendation model is the focus of the present invention. Furthermore, there is no method in the art for combining fitness metric learning and counterlearning in a tag recommendation system, and therefore, the present invention explores the behavior of counterlearning acting on metric learning.
Disclosure of Invention
Based on the defects of the existing label recommendation technology, the invention aims to provide a label recommendation method and a system based on parameter attack resistance metric learning, which can effectively improve the flexibility and modeling capability of metric learning and capture the implicit semantic relation among different data. By adding corresponding confrontation disturbance to the model parameters in metric learning, the model is confronted and defended, the generalization capability of the model is improved, and the quality of label recommendation is improved.
In order to achieve the purpose, the invention adopts the following technical scheme:
a label recommendation method based on parameter counterattack metric learning comprises the following steps:
(1) Acquiring historical access record information corresponding to a user identifier, and acquiring the user identifier, an item identifier and a tag identifier according to the historical access record information;
(2) Converting the user identifier, the item identifier and the label identifier into low-dimensional dense vector representation by using One-Hot coding to generate user representation, item representation and label representation; generating user countermeasure disturbance, project countermeasure disturbance and label countermeasure disturbance according to the user representation, the project representation and the label representation;
(3) Directly adding the user countermeasure disturbance, the item countermeasure disturbance and the tag countermeasure disturbance into the user representation, the item representation and the tag representation to generate a countermeasure user representation, a countermeasure item representation and a countermeasure tag representation;
(4) Generating a user-label and item-label potential relationship vector with the confrontation by using an attention mechanism to the confrontation user representation, the confrontation item representation and the confrontation label representation, and generating the user-label and item-label potential relationship vector according to the user representation, the item representation and the label representation;
(5) Modeling user-label and item-label distance measures using Euclidean distances to represent the user, item and label representations, and the latent relationship vectors;
(6) Modeling the confrontation user representation, the confrontation item representation and the confrontation label representation, and the confrontation potential relationship vector by using Euclidean distance to measure the confrontation distance of the user-label and the item-label, and returning a recommendation list of the top K labels which are most interesting to the user;
(7) Performing joint training on the distance metric and the confrontation distance metric by utilizing triple loss to solve a maximum and minimum optimization problem; the original model parameters are minimized while maximizing the counterdisturbance, which is updated by maximization, and the original model parameters are updated by minimization.
(8) And updating parameters and updating the countermeasure disturbance by using random gradient descent.
Preferably, in step (2), the three unique hot coded vectors of the user identifier, the item identifier and the tag identifier are respectively associated with the embedded matrices U, V, T U And T V Multiplication results in:
u p ＝U.onehot(u),v q ＝V.onehot(v)
Figure BDA0003996528740000031
wherein onehot (-) represents a one-hot encoding operation,
Figure BDA0003996528740000032
and &>
Figure BDA0003996528740000033
Figure BDA0003996528740000034
Representing a number domain, wherein the number of users, the number of items and the number of labels are respectively represented by | U |, | V |, and | T |, and d is the dimension of the potential feature; user is denoted as u p Item is denoted by v q The label of a particular user is indicated as ≥>
Figure BDA0003996528740000035
The tag of a particular item is denoted as ≥>
Figure BDA0003996528740000036
Preferably, in step (3), the user is injected with the user opposition disturbance on the user representation, the item representation and the label representation
Figure BDA0003996528740000037
Item fighting disturbance pick>
Figure BDA0003996528740000038
The tag of a particular user resists a disturbance>
Figure BDA0003996528740000039
The tag of a particular item resists a perturbation>
Figure BDA00039965287400000310
Figure BDA00039965287400000311
The size limits of the user versus disturbance, the item versus disturbance, the tag versus disturbance for a particular user, and the tag versus disturbance for a particular item are:
Figure BDA00039965287400000312
wherein
Figure BDA00039965287400000313
And &>
Figure BDA00039965287400000314
| | represents L 2 Norm, ε is the magnitude of the control countermeasure disturbance.
Preferably, in step (4), the confrontation user representation, the confrontation item representation and the confrontation label representation are used for generating a user-label potential relation vector and an item-label potential relation vector with confrontation, and the item representation and the label representation are used for generating the user-label and item-label potential relation vector according to the user representation; user-tag
Figure BDA0003996528740000041
And item-tag->
Figure BDA0003996528740000042
Against the potential relationship vector and the user-tag->
Figure BDA0003996528740000043
And item-tag
Figure BDA0003996528740000044
The potential relationship of (c) can be calculated as:
Figure BDA0003996528740000045
Figure BDA0003996528740000046
wherein the content of the first and second substances,
Figure BDA0003996528740000047
represents a user-tag antagonistic attention vector, < '> based on the user's preference>
Figure BDA0003996528740000048
Represents an item-tag antagonistic attention vector, <' > based on the status of the item>
Figure BDA0003996528740000049
The elements representing the countering attention vector in the user-tag,
Figure BDA00039965287400000426
elements representing the anti-attention vector in the item-tag, w i ＝χ T a i Indicating the user-tag attention vector, μ i ＝ζ T b i Represents an item-tag attention vector, <' > based on the number of items in the list>
Figure BDA00039965287400000410
An element representing an attention vector in the user-tag, is @>
Figure BDA00039965287400000411
Elements representing attention vectors in item-tags; the hadamard points are used to learn the joint embedding ≥ of user-tag>
Figure BDA00039965287400000412
And item-tag combination embedding>
Figure BDA00039965287400000413
And joint embedding with countervailing user-tag +>
Figure BDA00039965287400000414
Figure BDA00039965287400000415
And item-tag joint embedding
Figure BDA00039965287400000416
Preferably, in the step (5), the similarity score between the user representation, the item representation and the tag representation is calculated by using the euclidean distance, and the calculation formula is as follows:
Figure BDA00039965287400000417
wherein the content of the first and second substances,
Figure BDA00039965287400000418
represents the original model parameter, <' > is selected>
Figure BDA00039965287400000419
And &>
Figure BDA00039965287400000420
Generating a user-label potential relation and an item-label potential relation for the user; a higher similarity score +>
Figure BDA00039965287400000421
Meaning that the probability that user u will annotate item v with tag t is higher.
Preferably, in the step (6), the similarity score between the confrontation user representation, the confrontation item representation and the confrontation label representation is calculated by using the euclidean distance as follows:
Figure BDA00039965287400000422
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA00039965287400000423
represents fighting a disturbance>
Figure BDA00039965287400000424
And &>
Figure BDA00039965287400000425
Adding a user-label potential relation and an item-label potential relation generated after resisting disturbance; using the formula based on the obtained similarity score: />
Figure BDA0003996528740000051
And returning an ordered tag recommendation list, wherein K represents the length of the tag recommendation list.
Preferably, in step (7), the distance metric and the robust distance metric are jointly trained by using triple loss, the robust disturbance Δ is updated through maximization, and the parameter θ is updated through minimization;
Figure BDA0003996528740000052
where D is an example of training, α represents the robust regularizer,
Figure BDA0003996528740000053
for a constant obtained in the training process>
Figure BDA0003996528740000054
Representing the current parameters of the model and,
Figure BDA0003996528740000055
Figure BDA0003996528740000056
t' denotes the label not observed, m denotes a fixed margin, max (0, x), also called Relu function, opposes the component aL MLT (θ+Δ adv ) Which is considered as a term of the counterregularization of the model.
The training process is regarded as a maximum and minimum optimization problem:
Figure BDA0003996528740000057
wherein the content of the first and second substances,
Figure BDA0003996528740000058
and &>
Figure BDA0003996528740000059
Preferably, the counterdisturbance is updated by maximization; given a training instance (u, v, t, t'), the counterdisturbance is
Figure BDA0003996528740000061
Comprises the following steps:
Figure BDA0003996528740000062
wherein the content of the first and second substances,
Figure BDA0003996528740000063
a constant representing the current model parameter when->
Figure BDA0003996528740000064
When the gradient of the opposition perturbation Δ is equal to 0; otherwise, to maximize Δ exactly, the disturbance L is countered adv (Δ) is approximated as a linear function, the gradient of Δ being:
Figure BDA0003996528740000065
/>
wherein, under the constraint that | | | | Δ | ≦ ε, Δ | | | adv The optimal solution of (a) is:
Figure BDA0003996528740000066
the update of the parameter θ is to solve the local objective function of the minimization of the training instance (u, v, t, t'):
Figure BDA0003996528740000067
for the parameters involved
Figure BDA0003996528740000068
Their derivatives are expressed as:
Figure BDA0003996528740000069
preferably, in step (8), the parameter update and the disturbance rejection update are performed by using a stochastic gradient descent method, and the calculation is as follows:
Figure BDA00039965287400000610
Figure BDA00039965287400000611
wherein, eta represents learning rate, original parameter
Figure BDA00039965287400000612
Antagonizes a perturbation>
Figure BDA00039965287400000613
Figure BDA00039965287400000614
The optimum is found by solving a maximum and minimum optimization problem.
The invention also discloses a label recommendation system based on the method, which comprises the following modules:
an identification acquisition module: acquiring historical access record information corresponding to a user identifier, and acquiring the user identifier, an item identifier and a tag identifier according to the historical access record information;
the representing and resisting disturbance generating module: converting the user identifier, the item identifier and the label identifier into low-dimensional dense vector representations by using One-Hot coding, and respectively generating a user representation, an item representation and a label representation; respectively generating user countermeasure disturbance, project countermeasure disturbance and label countermeasure disturbance according to the user representation, the project representation and the label representation;
the confrontation representation generation module: adding the user countermeasure disturbance, the item countermeasure disturbance and the tag countermeasure disturbance to a user representation, an item representation and a tag representation respectively to generate an countermeasure user representation, an countermeasure item representation and an countermeasure tag representation respectively;
a relationship vector generation module: generating a confrontation user representation, a confrontation item representation and a confrontation label representation generated by a confrontation representation generation module to form confrontation potential relation vectors of a user-label and an item-label by using an attention mechanism, and generating the potential relation vectors of the user-label and the item-label according to the user representation, the item representation and the label representation;
a distance metric modeling module: modeling user-label and item-label distance measures using Euclidean distances to the user representation, item representation and label representation, and user-label and item-label potential relationship vectors;
the confrontation distance metric modeling module: modeling the confrontation user representation, the confrontation item representation and the confrontation label representation, and the confrontation potential relation vectors of the user-label and the item-label by using Euclidean distance, and returning a top K label recommendation list which is most interested by the user;
a training module: performing joint training on the distance metric and the confrontation distance metric by utilizing triple loss to solve the maximum and minimum optimization problem; minimizing the original model parameters while maximizing the disturbance rejection, wherein the disturbance rejection is updated through maximization, and the original model parameters are updated through minimization;
an update module: and updating parameters and updating the anti-disturbance by using random gradient descent.
Compared with the prior art, the invention has the following technical effects:
(1) In order to relieve the geometric inflexibility of metric learning, a relation structure of entity information hiding is mined, a generation process of a potential relation vector is defined, the potential relations of a user-label and a project-label are integrated into the modeling of the metric learning, the flexibility and the modeling capability of the metric learning are improved, and the semantic relation implicit among different data can be effectively captured. As a metric learning model, the invention utilizes Euclidean distance as a distance metric to calculate similarity scores among potential relationship vectors, user preferences, item features and label information.
(2) The method adds corresponding countermeasure disturbance to the model parameters in the metric learning so that the model performs countermeasure defense, reduces the difference between the data in the test stage and the data in the training stage, and obtains good generalization capability.
Drawings
Fig. 1 is a schematic diagram of a learning process of a tag recommendation method based on parameter counterattack metric learning according to an embodiment of the present invention;
fig. 2 is an overall structural diagram of a tag recommendation system based on parameter counterattack metric learning according to an embodiment of the present invention.
Fig. 3 is a block diagram of a tag recommendation system based on parameter countering attack metric learning according to an embodiment of the present invention.
Detailed Description
The technical solution of the present invention is further explained by the following specific examples.
The invention discloses a learning label recommendation method based on parameter anti-attack metrics, which comprises the steps of firstly, creating potential characteristic vectors of users, items and labels, directly injecting anti-disturbance on original parameters, then generating potential vectors of user-labels and item-labels by using an attention mechanism, using Hadamard products to learn the joint embedding of the user-labels and the item-labels, learning specific types of attention vectors based on key value pairs of the user-labels and the item-labels, and converting the attention vectors of the user-labels and the item-labels into different attention scores by adopting a Softmax function. Then, the potential relation between the user-label and the item-label is adaptively learned according to different attention scores of the user-label and the item-label, and a Euclidean distance is utilized to calculate a potential relation vector and a similarity score between different entity information by modeling distance measurement of the user-label and the item-label so as to form a prediction. An objective function combining metric learning and countermeasure learning is constructed to minimize the original model parameters while maximizing the countermeasure disturbance. And finally, updating the parameters by using a random gradient descent method.
Specifically, the method of the preferred embodiment of the present invention adopts the following technical scheme steps:
(1) When a user logs in the system, obtaining historical access record information corresponding to the user identification, and obtaining the user identification, the item identification and the label identification according to the historical access record information.
(2) And converting the original input into a binary sparse vector by utilizing one-hot coding according to the user identification, the item identification and the label identification. These vectors are then projected onto a low-dimensional dense vector, generating an embedded representation of the user, item, and tag. The three heat vectors are respectively associated with the embedded matrices U, V, T U And T V The process of multiplication, embedding, is represented as:
u p ＝U.onehot(u),v q ＝V.onehot(v)
Figure BDA0003996528740000091
wherein onehot (-) represents a one-hot encoding operation,
Figure BDA0003996528740000092
and &>
Figure BDA0003996528740000093
Figure BDA0003996528740000094
Representing a number domain, wherein | U |, | V |, | T | respectively represent the number of users, the number of items and the number of labels, and d is the dimension of the potential feature. User's feature representation u p Potential representation of an item v q User-specific tag feature representation
Figure BDA0003996528740000095
The tag characteristic representation of a particular item->
Figure BDA0003996528740000096
(3) An antagonistic perturbation Δ is injected on the embedded vector to attack the original parameters. The confrontation disturbance delta can be respectively expressed as user confrontation representation
Figure BDA0003996528740000097
Item confrontation representation->
Figure BDA0003996528740000098
Tag confrontation representation for a particular user>
Figure BDA0003996528740000099
Tag antagonism for a particular item indicates >>
Figure BDA00039965287400000910
The size of the disturbance is limited to:
Figure BDA00039965287400000911
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA00039965287400000912
and &>
Figure BDA00039965287400000913
| | The | represents L 2 Norm, epsilon is the size of a control disturbance vector, so that the original sample data of the anti-data attack can be automatically generated;
(4) In the attention mechanism layer, attention mechanism is used to generate user-tags
Figure BDA00039965287400000914
And the item-tag potential vector +>
Figure BDA00039965287400000915
And against the user-tag->
Figure BDA00039965287400000916
And a potential vector that antagonizes the item-tag->
Figure BDA00039965287400000917
They are calculated as:
Figure BDA00039965287400000918
Figure BDA00039965287400000919
wherein the content of the first and second substances,
Figure BDA00039965287400000920
represents a user-tag antagonistic attention vector, < '> based on the user's preference>
Figure BDA00039965287400000921
Represents an item-tag antagonistic attention vector, <' > based on the status of the item>
Figure BDA00039965287400000922
The elements representing the countering attention vector in the user-tag,
Figure BDA00039965287400000923
an element representing an anti-attention vector in the item-tag, < > based on the value of the item-tag>
Figure BDA00039965287400000924
Indicating the user-tag attention vector, μ i ＝ζ T b i Represents an item-tag attention vector, <' > based on the number of items in the list>
Figure BDA0003996528740000101
An element representing an attention vector in the user-tag, is @>
Figure BDA0003996528740000102
An element representing an attention vector in an item-tag; the hadamard points are used to learn the joint embedding ≥ of user-tag>
Figure BDA0003996528740000103
And item-tag associative embedding>
Figure BDA0003996528740000104
And joint embedding with countervailing user-tag +>
Figure BDA0003996528740000105
Figure BDA0003996528740000106
And item-tag joint embedding
Figure BDA0003996528740000107
(5) The triangle inequalities are satisfied using euclidean distance as a distance measure, and a potential relationship vector and a similarity score between different entity information are calculated by modeling the user-tag and item-tag distance measures to form a prediction. The distance between data can be calculated by using Euclidean distance:
Figure BDA0003996528740000108
where ρ is a point (x) 2 ,y 2 ) And point (x) 1 ,y 1 ) The Euclidean distance between, | X | is a point (X) 2 ,y 2 ) Euclidean distance to the origin.
(6) Considering the problem of similarity of data in tag recommendation, a similarity score between a user, an item and a tag is calculated by using Euclidean distance as a distance measure, and a scoring function of the similarity score can be calculated as a formula:
Figure BDA0003996528740000109
wherein the content of the first and second substances,
Figure BDA00039965287400001010
represents the original model parameter, <' > is selected>
Figure BDA00039965287400001011
And &>
Figure BDA00039965287400001012
A vector of potential relationships for the generated user-tag, item-tag. A higher similarity score +>
Figure BDA00039965287400001013
Meaning that the probability that user u will annotate item i with tag i is higher.
(7) By adding the anti-perturbation injection directly to the raw model parameters, the scoring function of the present invention (AMLT) can be described as the formula:
Figure BDA00039965287400001014
wherein the content of the first and second substances,
Figure BDA00039965287400001015
represents fighting a disturbance>
Figure BDA00039965287400001016
And &>
Figure BDA00039965287400001017
To add user-tag potential relationships and item-tag potential relationships generated against the perturbation. Using the formula based on the obtained similarity score: />
Figure BDA0003996528740000111
Returning a rankingThe tag recommendation list of (1).
(8) And constructing an objective function combining metric learning and antagonistic learning, wherein one part is a loss function of the original parameters, and the other part is a loss function with antagonistic disturbance. The core idea is that an additional anti-disturbance regularizer is added under the condition of no disturbance, and the additional anti-disturbance regularizer are optimized together in the training process, so that the model is forced to perform self-defense in the training process, and the robustness of the model is improved. The objective function of the joint training is:
Figure BDA0003996528740000112
/>
where D is an example of training, alpha represents the strength against the perturbation,
Figure BDA0003996528740000113
representing the current model parameters, t' represents the unobserved labels, m represents a fixed margin>
Figure BDA0003996528740000114
For a constant obtained during the training process, is>
Figure BDA0003996528740000115
Figure BDA0003996528740000116
max (0, x) is also known as Relu function. The opposition disturbance Δ is updated by maximization and the original parameter θ is updated by minimization. Antagonistic term α L MLT (θ+Δ adv ) Can be viewed as a term of de-regularization of the model. This training process can be viewed as playing a maximum and minimum game:
Figure BDA0003996528740000117
wherein the content of the first and second substances,
Figure BDA0003996528740000118
Figure BDA0003996528740000119
Figure BDA00039965287400001110
the learning algorithm for the model parameters θ is to minimize the participant, while the process Δ of acquiring the perturbation is taken as the maximizing participant, with the goal of identifying the worst-case perturbation for the current model. The two players alternately play until convergence.
(9) For further analysis, the present invention provides details of solving the infinitesimal optimization, given a training instance (u, v, t, t'), the opposition perturbation Δ can be updated by maximizing:
Figure BDA0003996528740000121
wherein it is present>
Figure BDA0003996528740000122
A constant representing the current model parameter when->
Figure BDA0003996528740000123
The gradient of the opposition perturbation Δ is equal to 0. Otherwise, to maximize Δ exactly, the objective function L is set adv (Delta) is approximated as a linear function, based on the value of>
Figure BDA0003996528740000124
The gradient of (d) is:
Figure BDA0003996528740000125
within a constraint of | < epsilon adv The optimal solution of (a) is:
Figure BDA0003996528740000126
(10) The invention fully considers the updating conditions of all parameters, and the solving of the minimized local objective function of the training example (u, v, t, t') comprises the following steps:
Figure BDA0003996528740000127
for all parameters involved
Figure BDA0003996528740000128
The derivative of (d) can be expressed as:
Figure BDA0003996528740000129
/>
(11) Updating the involved parameters and the countermeasure disturbance by using a stochastic gradient descent method (SGD), comprising the following steps:
Figure BDA00039965287400001210
Figure BDA00039965287400001211
where η represents the learning rate, the original parameter
Figure BDA00039965287400001212
Countering disturbances
Figure BDA00039965287400001213
Figure BDA00039965287400001214
After the parameter updating is finished, the model obtains a scoring function with good stability.
The following specific application example is combined to perform experimental demonstration aiming at a learning label recommendation method and system based on parameter counterattack metric, and specifically comprises the following steps:
1. preparing a standard data set
The invention uses the MovieLens data set as a standard data set to verify the effectiveness of the parameter-based anti-attack metric learning label recommendation method. The movilens dataset is a widely used reference dataset, published by the group research group, which analyzes the relationship between users, tags and movies. The details of the data set are shown in table 1.
TABLE 1 data set-related statistics
Data set Number of users Quantity of items Number of labels Number of training sets Number of test sets
MovieLens 469 1524 1017 30503 6911
2. Evaluation index and parameter setting
Referring to the (u, v) combination as a post, for each post (u, v), the last triplet (u, v, t) is selected as the test set F according to the marked time test And from the setF is cleared, and the observed user-item-tag triplets F remain train ＝F-F test As a training set. The purpose of the tag recommendation system is to provide a tag ordered list of Top-K for a post (u, v).
The invention judges the performance of label recommendation based on the widely used standard indexes in the information retrieval and recommendation system: precision @ K and recall @ K as evaluation indexes are respectively expressed as:
Figure BDA0003996528740000131
Figure BDA0003996528740000132
where R (u, v) represents a set of tags recommended to the user item pair (u, v), T (u, v) is a set of tags assigned to item v by user u, and Test represents a Test set. For these 2 indices, K is set to 5, 10, 20, respectively.
All algorithms are realized by using a TensorFlow framework in a Linux environment, and results are reported in a test set. The number of iterations of training is 1000, the learning rate η =0.001, the number of batches B =1024 and the regularization coefficient λ =0.001, the embedding dimension d =64, the edge m =0.3, the number of memory slices N =25, the strength of the anti-regularizer α =25, and the size of the anti-perturbation ∈ =25.
3. Experiments were performed on standard data sets
In order to verify the effectiveness of the parameter-based anti-attack metric learning label recommendation method, K =5, 10 and 20 are respectively taken from a MovieLens data set for modeling and prediction, and the prediction result and other prediction results are compared on an evaluation index. The results of the experiment are shown in table 2.
Table 2 comparison results of all algorithms in MovieLens dataset
Figure BDA0003996528740000141
It can be observed from table 2 that the present invention (AMLT) shows a better prediction accuracy on movilens dataset compared to the prediction results of other comparative algorithms (CF, PITF, NITF, CML, LRML, ABNT, ATF) on all indices. From the above analysis, the invention, as a label recommendation method combining antagonism learning and metric learning, shows higher accuracy and better stability on most indexes.
The invention relates to a label recommendation method and a system based on parameter counterattack metric learning, wherein the method comprises the following steps: and acquiring a user identifier, an item identifier and a label identifier according to the historical behavior record of the user, representing the user identifier, the item identifier and the label identifier as a binary sparse vector through one-hot coding, and projecting the binary sparse vector onto a low-dimensional dense vector. And reflecting the user preference and the item characteristics in different types of label information interaction by adopting an attention mechanism, and adaptively learning the potential relation between a user-label and an item-label according to different attention scores of the user-label and the item-label so as to improve the geometric flexibility of the model. Similarity scores between users, items, tags, and potential relationship vectors are calculated from euclidean distances. The robustness of the model can be effectively improved by the countermeasure learning, corresponding countermeasure disturbance is added to the parameters related to the method in the metric learning so that the model can carry out countermeasure defense, a novel objective function combining the metric learning and the countermeasure learning is constructed, the original model parameters are minimized while the countermeasure disturbance is maximized, the effectiveness of the proposed method is analyzed in principle, the basic working principle of the method is explained, and the interpretability of the method is improved.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is specific and detailed, but not to be understood as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. The label recommendation method based on parameter counterattack metric learning is characterized by comprising the following steps:
(1) Acquiring historical access record information corresponding to a user identifier, and acquiring the user identifier, an item identifier and a tag identifier according to the historical access record information;
(2) Converting the user identifier, the item identifier and the label identifier into low-dimensional dense vector representations by using One-Hot coding, and respectively generating a user representation, an item representation and a label representation; respectively generating user countermeasure disturbance, project countermeasure disturbance and label countermeasure disturbance according to the user representation, the project representation and the label representation;
(3) Adding the user countermeasure disturbance, the item countermeasure disturbance and the tag countermeasure disturbance to a user representation, an item representation and a tag representation respectively to generate an countermeasure user representation, an countermeasure item representation and an countermeasure tag representation respectively;
(4) Generating a user-label and item-label confrontation potential relationship vector by using the confrontation user representation, the confrontation item representation and the confrontation label representation in the step (3) by using an attention mechanism, and generating a user-label and item-label potential relationship vector according to the user representation, the item representation and the label representation;
(5) Modeling user-label and item-label distance measures using Euclidean distances to the user representation, item representation and label representation, and user-label and item-label potential relationship vectors;
(6) Modeling the confrontation user representation, the confrontation item representation and the confrontation label representation, and the confrontation potential relation vectors of the user-label and the item-label by using Euclidean distance, and returning a top K label recommendation list which is most interesting to the user;
(7) Performing joint training on the distance metric and the confrontation distance metric by utilizing triple loss to solve a maximum and minimum optimization problem; minimizing the original model parameters while maximizing the disturbance rejection, wherein the disturbance rejection is updated through maximization, and the original model parameters are updated through minimization;
(8) And updating parameters and updating the anti-disturbance by using random gradient descent.
2. The method of claim 1, wherein in step (2), the three unique encoding vectors of the user identifier, the item identifier and the tag identifier are respectively associated with the embedding matrix U, V, T U And T V Multiplication results in:
u p ＝U.onehot(u),v q ＝V.onehot(v)
Figure FDA0003996528730000021
wherein onehot (-) represents a one-hot encoding operation,
Figure FDA0003996528730000022
and
Figure FDA0003996528730000023
Figure FDA0003996528730000024
representing a number domain, wherein the U, V and T respectively represent the quantity of users, the quantity of items and the quantity of labels, and d is the dimension of potential features; the user is denoted as u p Item is denoted by v q The label of a particular user is indicated as ≥>
Figure FDA0003996528730000025
The tag of a particular item is denoted as ≥>
Figure FDA0003996528730000026
3. The method of claim 2, wherein in step (3), the user is injected with user opposition perturbation on the user representation, item representation and label representation
Figure FDA0003996528730000027
Item fighting disturbance pick>
Figure FDA0003996528730000028
The tag of a particular user resists a disturbance>
Figure FDA0003996528730000029
The tag of a particular item resists a perturbation>
Figure FDA00039965287300000210
Figure FDA00039965287300000211
The size limits of the user versus disturbance, the item versus disturbance, the tag versus disturbance for a particular user, and the tag versus disturbance for a particular item are:
Figure FDA00039965287300000212
wherein the content of the first and second substances,
Figure FDA00039965287300000213
and &>
Figure FDA00039965287300000214
| | represents L 2 Norm, ε is the magnitude of the control countermeasure disturbance.
4. The method of claim 3, wherein in step (4), the user-tag
Figure FDA00039965287300000215
And item-tag
Figure FDA00039965287300000216
Against the potential relationship vector and the user-tag->
Figure FDA00039965287300000217
And item-tag->
Figure FDA00039965287300000218
The potential relationship of (a) is calculated as:
Figure FDA00039965287300000219
Figure FDA00039965287300000220
wherein, the first and the second end of the pipe are connected with each other,
Figure FDA00039965287300000221
represents a user-tag antagonistic attention vector, < '> based on the user's preference>
Figure FDA00039965287300000222
Stands for item-tag antagonistic attention vector @>
Figure FDA00039965287300000223
An element representing an anti-attention vector in the user-tag, is->
Figure FDA00039965287300000224
Element representing the anti-attention vector in item-tag, w i ＝χ T a i Indicating the user-tag attention vector, μ i ＝ξ T b i Represents an item-tag attention vector, <' > based on the number of items in the list>
Figure FDA00039965287300000225
The elements representing the attention vectors in the user-tag,
Figure FDA00039965287300000226
elements representing attention vectors in item-tags; the hadamard points are used to learn the joint embedding ≥ of user-tag>
Figure FDA0003996528730000031
And item-tag combination embedding>
Figure FDA0003996528730000032
And joint embedding with antagonistic user-tag +>
Figure FDA0003996528730000033
Figure FDA0003996528730000034
And item-tag joint embedding
Figure FDA0003996528730000035
5. The method of claim 4, wherein in step (5), the similarity score between the user representation, the item representation and the tag representation is calculated using Euclidean distances as follows:
Figure FDA0003996528730000036
wherein, the first and the second end of the pipe are connected with each other,
Figure FDA0003996528730000037
represents the original model parameter, <' > is selected>
Figure FDA0003996528730000038
And &>
Figure FDA0003996528730000039
To the generated user-tag potential relationship and item-tag potential relationship.
6. The method of claim 5, wherein in step (6), the Euclidean distance is used to calculate the similarity score between the confrontational user representation, the confrontational item representation and the confrontational tag representation by the following formula:
Figure FDA00039965287300000310
/>
wherein, the first and the second end of the pipe are connected with each other,
Figure FDA00039965287300000311
representing resistance to perturbation>
Figure FDA00039965287300000312
And &>
Figure FDA00039965287300000313
Adding a user-label potential relation and an item-label potential relation generated after resisting disturbance; using the formula based on the obtained similarity score:
Figure FDA00039965287300000314
and returning an ordered tag recommendation list, wherein K represents the length of the tag recommendation list.
7. The method of claim 6, wherein in step (7), the distance metric and the robust distance metric are jointly trained using triple loss, the robust perturbation Δ is updated by maximization, and the parameter θ is updated by minimization:
Figure FDA0003996528730000041
where D is an example of training, α represents the robust regularizer,
Figure FDA0003996528730000042
for a constant obtained in the training process>
Figure FDA0003996528730000043
Representing the parameters of the current model and,
Figure FDA0003996528730000044
Figure FDA0003996528730000045
t' denotes the label not observed, m denotes a fixed margin, max (0, x) is called Relu function, antagonistic component α L MLT (θ+Δ adv ) One term of counterregularization, which is considered a model;
the training process is regarded as a maximum and minimum optimization problem:
Figure FDA0003996528730000046
wherein the content of the first and second substances,
Figure FDA0003996528730000047
and
Figure FDA0003996528730000048
8. the method of claim 7, wherein the counterperturbation is updated by maximization; given a training instance (u, v, t, t'), the counterdisturbance is
Figure FDA0003996528730000049
Comprises the following steps:
Figure FDA00039965287300000410
wherein the content of the first and second substances,
Figure FDA00039965287300000411
a constant representing the current model parameter when->
Figure FDA00039965287300000412
When the gradient of the opposition perturbation Δ is equal to 0; otherwise, oppose disturbance L adv (Δ) is approximated as a linear function, the gradient of Δ being: />
Figure FDA00039965287300000413
Wherein Δ | | | ≦ ε constraint adv The optimal solution of (a) is:
Figure FDA0003996528730000051
the update of the parameter θ is to solve the local objective function of the minimization of the training instance (u, v, t, t'):
Figure FDA0003996528730000052
wherein, delta adv To counter a constant obtained after update of the disturbance, for the parameter concerned
Figure FDA0003996528730000053
Is expressed as:
Figure FDA0003996528730000054
9. the method of claim 8, wherein step (8), the updating of the parameters and the updating of the counterdisturbance using stochastic gradient descent, are calculated as:
Figure FDA0003996528730000055
Figure FDA0003996528730000056
where η represents the learning rate, the original parameter
Figure FDA0003996528730000057
Antagonizes a perturbation>
Figure FDA0003996528730000058
Figure FDA0003996528730000059
The optimum is found by solving a maximum and minimum optimization problem.
10. A tag recommendation system based on the method of any one of claims 1-9, comprising the following modules:
an identification acquisition module: acquiring historical access record information corresponding to a user identifier, and acquiring the user identifier, an item identifier and a tag identifier according to the historical access record information;
the representing and resisting disturbance generating module: converting the user identifier, the project identifier and the label identifier into low-dimensional dense vector representations by utilizing One-Hot, and respectively generating a user representation, a project representation and a label representation; respectively generating user countermeasure disturbance, project countermeasure disturbance and label countermeasure disturbance according to the user representation, the project representation and the label representation;
the confrontation representation generation module: adding the user countermeasure disturbance, the item countermeasure disturbance and the tag countermeasure disturbance to a user representation, an item representation and a tag representation respectively to generate an countermeasure user representation, an countermeasure item representation and an countermeasure tag representation respectively;
a relationship vector generation module: generating a confrontation user representation, a confrontation item representation and a confrontation label representation generated by a confrontation representation generation module to form confrontation potential relation vectors of a user-label and an item-label by using an attention mechanism, and generating the potential relation vectors of the user-label and the item-label according to the user representation, the item representation and the label representation;
a distance metric modeling module: modeling user-label and item-label distance measures using Euclidean distances to the user representation, item representation and label representation, and user-label and item-label potential relationship vectors;
the confrontation distance metric modeling module: modeling the confrontation user representation, the confrontation item representation and the confrontation label representation, and the confrontation potential relation vectors of the user-label and the item-label by using Euclidean distance to measure the confrontation distance of the user-label and the item-label, and returning a top K label recommendation lists which are most interested in the user;
a training module: performing joint training on the distance metric and the confrontation distance metric by utilizing triple loss to solve the maximum and minimum optimization problem; minimizing the original model parameters while maximizing the disturbance rejection, wherein the disturbance rejection is updated through maximization, and the original model parameters are updated through minimization;
an updating module: and updating parameters and updating the countermeasure disturbance by using random gradient descent.
CN202211603826.8A 2022-12-13 2022-12-13 Label recommendation method and system based on parameter anti-attack metric learning Pending CN115860053A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211603826.8A CN115860053A (en) 2022-12-13 2022-12-13 Label recommendation method and system based on parameter anti-attack metric learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211603826.8A CN115860053A (en) 2022-12-13 2022-12-13 Label recommendation method and system based on parameter anti-attack metric learning

Publications (1)

Publication Number Publication Date
CN115860053A true CN115860053A (en) 2023-03-28

Family

ID=85672736

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211603826.8A Pending CN115860053A (en) 2022-12-13 2022-12-13 Label recommendation method and system based on parameter anti-attack metric learning

Country Status (1)

Country Link
CN (1) CN115860053A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117540791A (en) * 2024-01-03 2024-02-09 支付宝(杭州)信息技术有限公司 Method and device for countermeasure training

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117540791A (en) * 2024-01-03 2024-02-09 支付宝(杭州)信息技术有限公司 Method and device for countermeasure training
CN117540791B (en) * 2024-01-03 2024-04-05 支付宝(杭州)信息技术有限公司 Method and device for countermeasure training

Similar Documents

Publication Publication Date Title
Kong et al. VOPRec: Vector representation learning of papers with text information and structural identity for recommendation
Bansal et al. Content driven user profiling for comment-worthy recommendations of news and blog articles
Cao et al. Shilling attack detection utilizing semi-supervised learning method for collaborative recommender system
Qi et al. Mining collective intelligence in diverse groups
Hu et al. A self-training approach for resolving object coreference on the semantic web
Zhou et al. Userrec: A user recommendation framework in social tagging systems
Su et al. Neural graph matching based collaborative filtering
Armandpour et al. Robust negative sampling for network embedding
Ma et al. Hierarchical taxonomy aware network embedding
Zhang et al. Graph embedding-based approach for detecting group shilling attacks in collaborative recommender systems
Jiao et al. A novel learning rate function and its application on the SVD++ recommendation algorithm
Cao et al. An improved collaborative filtering recommendation algorithm based on community detection in social networks
Su et al. Research on personalized recommendation algorithm based on ontological user interest model
Gao et al. CNL: collective network linkage across heterogeneous social platforms
Stanescu et al. A hybrid recommender system: User profiling from keywords and ratings
CN113918833A (en) Product recommendation method realized through graph convolution collaborative filtering of social network relationship
Dai et al. BTR: a feature-based Bayesian task recommendation scheme for crowdsourcing system
CN115860053A (en) Label recommendation method and system based on parameter anti-attack metric learning
Liang et al. Cross-media semantic correlation learning based on deep hash network and semantic expansion for social network cross-media search
Chen et al. DPM-IEDA: dual probabilistic model assisted interactive estimation of distribution algorithm for personalized search
Meng et al. Community discovery in social networks via heterogeneous link association and fusion
CN115840853A (en) Course recommendation system based on knowledge graph and attention network
Liao et al. Accumulative Time Based Ranking Method to Reputation Evaluation in Information Networks
Wei et al. Unsupervised meta-path reduction on heterogeneous information networks
Yuan et al. A weighted one class collaborative filtering with content topic features

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination