CN109410080A - A kind of social image recommended method based on level attention mechanism - Google Patents

A kind of social image recommended method based on level attention mechanism Download PDF

Info

Publication number
CN109410080A
CN109410080A CN201811200827.1A CN201811200827A CN109410080A CN 109410080 A CN109410080 A CN 109410080A CN 201811200827 A CN201811200827 A CN 201811200827A CN 109410080 A CN109410080 A CN 109410080A
Authority
CN
China
Prior art keywords
user
image
matrix
social
formula
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811200827.1A
Other languages
Chinese (zh)
Inventor
吴乐
陈雷
汪萌
洪日昌
杨永晖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hefei University of Technology
Original Assignee
Hefei University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hefei University of Technology filed Critical Hefei University of Technology
Priority to CN201811200827.1A priority Critical patent/CN109410080A/en
Publication of CN109410080A publication Critical patent/CN109410080A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/01—Social networking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Economics (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The invention discloses a kind of social image recommended methods based on level attention mechanism, comprising: 1. stereo isomers data: user is to the rating matrix of image, user to the social networks matrix between the upload information matrix of image, user and user；2. pair isomeric data and image set handle to obtain social embeded matrix, content embeded matrix, style embeded matrix；3. embeded matrix is input in bottom attention network and obtains social semantic information；4. passing through three aspects that top layer attention network obtains social semantic information weight shared when influencing user and selecting image；5. according to eigenmatrix, three aspect social semantic information matrix and the weight computing shared by them obtain score in predicting value, to carry out image recommendation.The present invention does not solve data sparsity problem effectively merely with the social information between the upload information of user and user, more explains user preferences well by level attention mechanism, realizes accurately social image recommendation.

Description

A kind of social image recommended method based on level attention mechanism
Technical field
The present invention relates to image recommendation fields, are specifically a kind of social image recommendation sides based on level attention Method.
Technical background
Social networks based on image has become most popular social networks in recent years.With mobile phone user It quicklys increase, many users, which shoot image and upload in social network-i i-platform, shares their lives.The image largely uploaded is led Image overload is caused, how understanding the hobby of different user and carrying out accurately image recommendation to it becomes urgent need.
The problem of collaborative filtering can be with effective solution image recommendation, it passes through the digging to user's history behavioral data Pick, finds the preference of user and recommends.Although collaborative filtering is widely used, but user-image interbehavior square The sparsity of battle array limits its recommendation performance.In order to solve the data sparsity problem of collaborative filtering, RuiningHe in 2016 Et al. propose VBPR algorithm, the visual signature of image is extracted by deep neural network CNN, the visual information of image is embedded in Into existing collaborative filtering, the inherent attribute of article has been excavated sufficiently to improve recommendation precision.
But the inherent attribute that the collaborative filtering method based on Image Visual Feature only accounts for article does not use The inherent attribute of user.One important feature of social image platform is exactly that there are social networks between different user, therefore When user shows the preference to image in social platform, social networks can be embedded into collaborative filtering and mitigate number System recommendation performance is improved according to sparsity.
Although people, which increasingly pay close attention to, combines carry out image recommendation for various auxiliary embedding datas, how more preferable Ground is improved using the feature of social image platform in comprehensive manner recommends performance still in exploration.In addition to improving social image Recommend how to excavate the hobby of different user except precision, so that making reasonable explain to recommendation results is also to need to solve The problem of.
Summary of the invention
The present invention to solve the deficiencies in the prior art place, propose that a kind of social image based on level attention mechanism pushes away Recommend method, to can social networks sufficiently between user, user to the behavioral data of image and the insertion of Image Visual Feature Effective solution data sparsity problem, to improve social recommendation precision；User is influenced to image selection by definition simultaneously Three aspects semantic information, and illustrate user to the influence power of image selection, thus very using level attention mechanism The good interpretation for solving the problems, such as social image recommendation.
The present invention adopts the following technical scheme that in order to solve the technical problem
A kind of the characteristics of social image recommended method based on level attention mechanism of the invention is to carry out as follows:
Step 1, stereo isomers data, comprising: user is to the rating matrix R of image, user to the upload information square of image Social networks matrix S between battle array L, user and user:
U is enabled to indicate user's collection, and U={ u1,...,ua,...,ub,...,uM, uaIndicate a-th of user, ubIndicate b A user, M indicate total number of users, 1≤a, b≤M；V is enabled to indicate image set, and V={ v1,...,vi,...,vj,...,vN, vi Indicate i-th of image, vjIndicate that j-th of image, N indicate total number of images, 1≤i, j≤N；Enable RaiIndicate a-th of user uaTo I image viScore value, then rating matrix R={ R of the user to imageai}M×N；Enable LaiIndicate a-th of user uaTo i-th Image viUpload information, then upload information matrix L={ L of the user to imageai}M×N；Enable SabIndicate a-th of user uaTo b The u of a userbSocial networks, then the social networks matrix S={ S between user and userab}M×M；
Step 2 handles the isomeric data and image set V to obtain social embeded matrix E, content embeded matrix FcAnd wind Lattice embeded matrix Fs:
Step 2.1 is handled social networks matrix S by Random Walk Algorithm, obtains social embeded matrix E；
Step 2.2, the visual signature that image is extracted by depth convolutional neural networks model VGG:
The output Fc of the last one full articulamentum of the depth convolutional neural networks model VGG is chosen as i-th of figure As viContent be embedded in vector Fi c, to obtain the content embeded matrix F of all imagesc；
Assuming that m-th of convolutional layer has NmA filter, each filter size are Mm, then depth convolutional neural networks The Gram matrix of m-th of convolutional layer feature of model VGGM=1~5；
Feature Mapping relationship between m-th of convolutional layer, p-th of filter and q-th of filter is calculated using formula (1)
In formula (1),Indicate the excitation value of p-th of m-th of convolutional layer, k-th of filter position,Indicate m-th volume The excitation value of q-th of lamination, k-th of filter position, 1≤p, q≤Nm, 1≤k≤Mm；
Vector F is embedded in using the style that formula (2) obtains i-th of imagei s, to obtain the style embeded matrix of all images Fs:
Fi s=[v (G1),...,v(Gl),...,v(G5)] (2)
In formula (2), v (Gm) indicate m-th of convolutional layer feature Gram matrix GlVector quantization；
Step 3, by social embeded matrix E, content embeded matrix FcWith style embeded matrix FsIt is input to bottom attention net Obtain influencing the social semantic information for three aspects that user selects picture in network:
Step 3.1, the upload consistency for calculating user
Step 3.1.1, L is enabledaIndicate a-th of user uaThe image set of upload then obtains j-th of image v using formula (3)j? Calculate a-th of user uaUpload consistency when shared weight αaj:
In formula (3), σ () indicates sigmoid function；PaAnd QaIt is a-th of user u respectivelyaFoundation characteristic vector sum it is auxiliary Help feature vector, WjAnd XjIt is j-th of image v respectivelyjFoundation characteristic vector sum supplemental characteristic vector；EaIt is a-th of user ua Social insertion vector；WithIt is j-th of image v respectivelyjContent insertion vector sum style be embedded in vector；θu=[w2, W1,Wc,Ws] be the first bottom attention network B otNet1 parameter, wherein w2It is the parameter of first sigmoid function, W1It is First matrix parameter, WcIt is to content embeded matrix FcCarry out the parameter of dimensionality reduction, WsIt is to style embeded matrix FsCarry out dimensionality reduction Parameter；It is a-th of user uaThe picture material embeded matrix liked, and obtained by formula (4)；It is a-th of user ua The image style embeded matrix liked, and obtained by formula (5)；
Step 3.1.2, to the jth image vjCalculating a-th of user uaUpload consistency when shared weight αajIt is normalized, the weight after being normalized by formula (6)
Step 3.1.3, a-th of user u is obtained using formula (7)aUpload consistency
Step 3.2 calculates a-th of user uaSocial influence power
Step 3.2.1, S is enabledaIt indicates and a-th of user uaThere is the user of social networks to collect, is then obtained b-th using formula (8) User ubCalculating a-th of user uaSocial influence power when shared weight βab:
βab=σ (w4×(W3×[Pa,Pb,Qa,Qb,Ea,Eb,Fa c,Fa s])) (8)
In formula (7), PbAnd QbIt is b-th of user u respectivelybFoundation characteristic vector sum supplemental characteristic vector；EbIt is b-th User ubSocial insertion vector；WithIt is a-th of user u respectivelyaThe image set liked content embeded matrix and wind Lattice embeded matrix；θs=[w4,W3] be the second bottom attention network B otNet2 parameter, wherein w4It is second sigmoid The parameter of function, W3It is second matrix parameter；
Step 3.2.2, to b-th of user ubCalculating a-th of user uaSocial influence power when shared weight βabIt is normalized, the weight after being normalized by formula (9)
Step 3.2.3, a-th of user u is obtained according to formula (10)aSocial influence power
Step 3.3 enables CiIndicate i-th of image viUploader, and each image only one uploader, thus by i-th A image viUploader influence power be expressed as i-th of image viUploader CiAuxiliary be embedded in vector QCi；
Using the upload consistency, social influence power and uploader influence power as the social semantic information of three aspects；
Step 4, three input A that top layer attention network TopNet is respectively obtained using formula (11)λ, λ=1,2,3:
A-th of user u is being influenced respectively using the social semantic information that formula (12) obtains three aspectsaSelect image when institute The weight γ accounted foraλ:
γaλ=σ (w6×(W5×Aλ)) (12)
In formula (12), θa=[w6,W5] be top layer attention network parameter, w6It is the ginseng of third sigmoid function Number, W5It is third matrix parameter；
Step 5, according to three aspect social semantic information and its shared weight computing user it is pre- to the scoring of image Measured value, to carry out image recommendation to user:
Step 5.1 obtains a-th of user u according to formula (13)aWith i-th of image viCorresponding prediction scoringTo To user to the score in predicting matrix of article
In formula (13),It is i-th of image viFoundation characteristic vector transposed vector；
Step 5.2 establishes the loss function L (θ) as shown in formula (14):
In formula (13), θ=[θ1,θ2] it is parameter to be optimized, θ1=[P, Q, W, X] is eigenmatrix, θ2=[θu,θs, θa] be attention network parameter, λ is regularization term, Da=(i, j) | i ∈ Ra∧j∈V-RaIt is a-th of user uaTraining Data, RaIt is a-th of user uaThe image set liked；
Step 5.3 optimizes the loss function L (θ) by stochastic gradient descent method, so that L (θ) reaches Minimum to obtain optimum prediction rating matrix, and carries out image recommendation to user according to institute's optimum prediction rating matrix.
Compared with the prior art, the invention has the advantages that:
1, the present invention devise a level attention network by between Image Visual Feature, user social networks and user Behavioral data (upload behavior and thumb up behavior) fusion, alleviate data sparsity problem, effectively increase image and push away It recommends precision and explains the preference of user well.
2, the present invention constructs the isomeric data of social image platform by step 1, to greatly alleviate recommendation system Sparse Problem in system.
3, the present invention is not only extracted the content information of image by depth convolutional neural networks, is also extracted the wind of image Lattice information, because the style of image is also to influence a key factor of user's selection, the insertion of image style matrix is significantly mentioned The high precision of social image recommendation.
4, invention defines influence users to the social semantic information of three aspects of image selection, and as one A important feature is embedded into the inherent attribute for going sufficiently to have excavated user in user interest vector, improves recommendation precision.
5, the present invention devises a level attention network to model to social image proposed algorithm；Bottom pays attention to The power network struction social semantic information of user, top layer attention network obtain these three in terms of social semantic information in shadow Weight shared when different user selection image is rung, to solve the problems, such as the interpretation of social platform image very well.
Detailed description of the invention
Fig. 1 is that the present invention is based on the flow charts of the social image recommended method of level attention mechanism；
Fig. 2 a is the HR experimental result picture of the method for the present invention and the 6 kinds of control methods selected on F_S data set；
Fig. 2 b is the NDCG experimental result picture of the method for the present invention and the 6 kinds of control methods selected on F_S data set；
Fig. 3 a is the HR experimental result picture of the method for the present invention and the 6 kinds of control methods selected on F_L data set；
Fig. 3 b is the NDCG experimental result picture of the method for the present invention and the 6 kinds of control methods selected on F_L data set.
Specific embodiment
Referring to Fig. 1, in the present embodiment, a kind of social image recommended method based on level attention mechanism is by following step It is rapid to carry out:
Step 1, stereo isomers data, comprising: user is to the rating matrix R of image, user to the upload information square of image Social networks matrix S between battle array L, user and user:
U is enabled to indicate user's collection, and U={ u1,...,ua,...,ub,...,uM, uaIndicate a-th of user, ubIndicate b A user, M indicate total number of users, 1≤a, b≤M；V is enabled to indicate image set, and V={ v1,...,vi,...,vj,...,vN, vi Indicate i-th of image, vjIndicate that j-th of image, N indicate total number of images, 1≤i, j≤N；Enable RaiIndicate a-th of user uaTo The v of i imageiScore value, then rating matrix R={ R of the user to imageai}M×N；If a-th of user uaTo i-th image viThere is scoring, then Rai=1, otherwise Rai=0.Enable LaiIndicate a-th of user uaTo the v of i-th of imageiUpload information, then use Upload information matrix L={ L of the family to imageai}M×N；If a-th of user uaUpload i-th image vi, then Lai=1, otherwise Lai=0.Enable SabIndicate a-th of user uaTo the u of b-th of userbSocial networks, then social networks between user and user Matrix S={ Sab}M×M.If a-th of user uaTo the u of b-th of userbSocial networks, then Sab=1, otherwise Sab=0.
Step 2 handles isomeric data and image set V to obtain social embeded matrix E, content embeded matrix FcIt is embedding with style Enter matrix Fs:
Step 2.1 passes through Random Walk Algorithm to social networks matrix SM×MIt is handled, obtains social embeded matrix Ed ×M, d=20；
Step 2.2, the visual signature that image is extracted by depth convolutional neural networks model VGG:
4096 dimensional vectors of the last one full articulamentum output of depth convolutional neural networks model VGG19 are chosen as the I image viContent be embedded in vector Fi c, to obtain the content embeded matrix F of all imagesc；
Assuming that m-th of convolutional layer has NmA filter, each filter size are Mm, then depth convolutional neural networks model The Gram matrix of m-th of convolutional layer feature of VGGM=1~5；
Feature Mapping relationship between m-th of convolutional layer, p-th of filter and q-th of filter is calculated using formula (1)
In formula (1),Indicate the excitation value of p-th of m-th of convolutional layer, k-th of filter position,Indicate m-th volume The excitation value of q-th of lamination, k-th of filter position, 1≤p, q≤Nm, 1≤k≤Mm；
Vector is embedded in using the style that formula (2) obtains i-th of imageTo obtain the style embeded matrix of all images Fs:
In formula (2), v (Gm) indicate m-th of convolutional layer feature Gram matrix GlVector quantization；
Step 3, by social embeded matrix E, content embeded matrix FcWith style embeded matrix FsIt is input to bottom attention net Obtain influencing the social semantic information for three aspects that user selects picture in network:
Step 3.1, the upload consistency for calculating user
Step 3.1.1, L is enabledaIndicate a-th of user uaThe image set of upload then obtains j-th of image v using formula (3)j? Calculate a-th of user uaUpload consistency when shared weight αaj:
In formula (3), σ () indicates sigmoid function；PaAnd QaIt is all the d dimensional vector of Gaussian distributed, difference table Show a-th of user uaFoundation characteristic vector sum supplemental characteristic vector, WjAnd XjIt is all the d dimensional vector of Gaussian distributed, point J-th of image v is not indicatedjFoundation characteristic vector sum supplemental characteristic vector；EaIt is a column of social embeded matrix E, indicates a A user uaSocial insertion vector；It is content embeded matrix FcJth column, indicate j-th of image vjContent be embedded in Amount,It is style embeded matrix FsJth column, indicate j-th of image vjStyle be embedded in vector；θu=[w2,W1,Wc,Ws] be The parameter of first bottom attention network B otNet1, wherein w2It is the parameter of first sigmoid function, W1It is first matrix Parameter, WcIt is to content embeded matrix FcCarry out the parameter of dimensionality reduction, WsIt is to style embeded matrix FsCarry out the parameter of dimensionality reduction； It is a-th of user uaThe picture material embeded matrix liked, and obtained by formula (4)；It is a-th of user uaThe image liked Style embeded matrix, and obtained by formula (5)；
Step 3.1.2, to jth image vjCalculating a-th of user uaUpload consistency when shared weight αajInto Row normalized, the weight after being normalized by formula (6)
Step 3.1.3, a-th of user u is obtained using formula (7)aUpload consistency
Step 3.2 calculates a-th of user uaSocial influence power
Step 3.2.1, S is enabledaIt indicates and a-th of user uaThere is the user of social networks to collect, is then obtained b-th using formula (8) User ubCalculating a-th of user uaSocial influence power when shared weight βab:
In formula (7), PbAnd QbAll it is the d dimensional vector of Gaussian distributed, respectively indicates b-th of user ubBasis it is special Levy vector sum supplemental characteristic vector；EbIt is the b column of social embeded matrix E, indicates b-th of user ubSocial insertion vector；θs =[w4,W3] be the second bottom attention network B otNet2 parameter, wherein w4It is the parameter of second sigmoid function, W3 It is second matrix parameter；
Step 3.2.2, to b-th of user ubCalculating a-th of user uaSocial influence power when shared weight βabInto Row normalized, the weight after being normalized by formula (9)
Step 3.2.3, a-th of user u is obtained according to formula (10)aSocial influence power
Step 3.3 enables CiIndicate i-th of image viUploader, and each image only one uploader, thus by i-th A image viUploader influence power be expressed as i-th of image viUploader CiAuxiliary embeded matrix QCi, QCiIt is to obey height The d dimensional vector of this distribution；
To upload consistency, social influence power and uploader influence power as the social semantic information of three aspects；
Step 4, three input A that top layer attention network TopNet is respectively obtained using formula (11)λ, λ=1,2,3:
A-th of user u is being influenced respectively using the social semantic information that formula (12) obtains three aspectsaSelect image when institute The weight γ accounted foraλ:
γaλ=σ (w6×(W5×Aλ)) (12)
In formula (12), θa=[w6,W5] be top layer attention network parameter, w6It is the ginseng of third sigmoid function Number, W5It is third matrix parameter；
Step 5, according to three aspect social semantic information and its shared weight computing user it is pre- to the scoring of image Measured value, to carry out image recommendation to user:
Step 5.1 obtains a-th of user u according to formula (13)aWith i-th of image viCorresponding prediction scoringTo To user to the score in predicting matrix of article
In formula (13),It is the d dimension row vector of Gaussian distributed, indicates i-th of image viFoundation characteristic vector Wi Transposed vector；
Step 5.2 establishes the loss function L (θ) as shown in formula (14):
In formula (13), θ=[θ1,θ2] it is parameter to be optimized, θ1=[P, Q, W, X] is eigenmatrix, θ2=[θu,θs, θa] be attention network parameter, λ is regularization term, Da=(i, j) | i ∈ Ra∧j∈V-RaIt is a-th of user uaTraining Data, RaIt is a-th of user uaThe image set liked；
Step 5.3 optimizes loss function L (θ) by stochastic gradient descent method, so that L (θ) reaches most It is small, to obtain optimum prediction rating matrix, and image recommendation is carried out to user according to institute's optimum prediction rating matrix.
Embodiment:
In order to verify the validity of this method, the present invention has grabbed a large amount of figure from social image sharing platform Flickr As being used as data set, it is from widely used NUS-WIDE data set extension.The social link of screening is less than 2, scoring Record is less than 2 user, and filtered data set is known as F_L data set.Filter F_L data set further to ensure each use Family and every image at least 10 records, to obtain F_S data set.
The present invention uses Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG) As evaluation criterion.Selected 6 kinds of methods come with method proposed in this paper carry out effect comparison, be respectively BPR, VBPR, ACF, SR, ContextMF and VPOI.Specifically, it can be obtained a result as shown in Fig. 2 a, Fig. 2 b according to experimental result, experimental result Being shown in method proposed by the present invention on data set F_L will be good than 6 kinds of method effects of selection.As shown in Figure 3a, 3b, real It tests result and is again shown in method proposed by the present invention on data set F_L and be superior to other on two evaluation criterions HR and NDCG Method, to demonstrate the feasibility of proposition method of the present invention.

Claims (1)

1. a kind of social image recommended method based on level attention mechanism, it is characterized in that carrying out as follows:
Step 1, stereo isomers data, comprising: user to the rating matrix R of image, user to the upload information matrix L of image, Social networks matrix S between user and user:
U is enabled to indicate user's collection, and U={ u1,...,ua,...,ub,...,uM, uaIndicate a-th of user, ubIndicate b-th of use Family, M indicate total number of users, 1≤a, b≤M；V is enabled to indicate image set, and V={ v1,...,vi,...,vj,...,vN, viIt indicates I-th of image, vjIndicate that j-th of image, N indicate total number of images, 1≤i, j≤N；Enable RaiIndicate a-th of user uaTo i-th Image viScore value, then rating matrix R={ R of the user to imageai}M×N；Enable LaiIndicate a-th of user uaTo i-th of image viUpload information, then upload information matrix L={ L of the user to imageai}M×N；Enable SabIndicate a-th of user uaB-th is used The u at familybSocial networks, then the social networks matrix S={ S between user and userab}M×M；
Step 2 handles the isomeric data and image set V to obtain social embeded matrix E, content embeded matrix FcIt is embedding with style Enter matrix Fs:
Step 2.1 is handled social networks matrix S by Random Walk Algorithm, obtains social embeded matrix E；
Step 2.2, the visual signature that image is extracted by depth convolutional neural networks model VGG:
The output Fc of the last one full articulamentum of the depth convolutional neural networks model VGG is chosen as i-th of image vi's Content is embedded in vector Fi c, to obtain the content embeded matrix F of all imagesc；
Assuming that m-th of convolutional layer has NmA filter, each filter size are Mm, then the depth convolutional neural networks model The Gram matrix of m-th of convolutional layer feature of VGG
Feature Mapping relationship between m-th of convolutional layer, p-th of filter and q-th of filter is calculated using formula (1)
In formula (1),Indicate the excitation value of p-th of m-th of convolutional layer, k-th of filter position,Indicate m-th of convolutional layer The excitation value of q-th of filter, k-th of position, 1≤p, q≤Nm, 1≤k≤Mm；
Vector F is embedded in using the style that formula (2) obtains i-th of imagei s, to obtain the style embeded matrix F of all imagess:
Fi s=[v (G1),...,v(Gl),...,v(G5)] (2)
In formula (2), v (Gm) indicate m-th of convolutional layer feature Gram matrix GlVector quantization；
Step 3, by social embeded matrix E, content embeded matrix FcWith style embeded matrix FsIt is input in bottom attention network Obtain influencing the social semantic information for three aspects that user selects picture:
Step 3.1, the upload consistency for calculating user
Step 3.1.1, L is enabledaIndicate a-th of user uaThe image set of upload then obtains j-th of image v using formula (3)jIt is calculating A-th of user uaUpload consistency when shared weight αaj:
In formula (3), σ () indicates sigmoid function；PaAnd QaIt is a-th of user u respectivelyaFoundation characteristic vector sum assist it is special Levy vector, WjAnd XjIt is j-th of image v respectivelyjFoundation characteristic vector sum supplemental characteristic vector；EaIt is a-th of user uaSociety Hand over insertion vector；WithIt is j-th of image v respectivelyjContent insertion vector sum style be embedded in vector；θu=[w2,W1,Wc, Ws] be the first bottom attention network B otNet1 parameter, wherein w2It is the parameter of first sigmoid function, W1It is first A matrix parameter, WcIt is to content embeded matrix FcCarry out the parameter of dimensionality reduction, WsIt is to style embeded matrix FsCarry out the ginseng of dimensionality reduction Number；It is a-th of user uaThe picture material embeded matrix liked, and obtained by formula (4)；It is a-th of user uaLike Image style embeded matrix, and by formula (5) obtain；
Step 3.1.2, to the jth image vjCalculating a-th of user uaUpload consistency when shared weight αajIt carries out Normalized, the weight after being normalized by formula (6)
Step 3.1.3, a-th of user u is obtained using formula (7)aUpload consistency
Step 3.2 calculates a-th of user uaSocial influence power
Step 3.2.1, S is enabledaIt indicates and a-th of user uaThere is the user of social networks to collect, then obtains b-th of user using formula (8) ubCalculating a-th of user uaSocial influence power when shared weight βab:
βab=σ (w4×(W3×[Pa,Pb,Qa,Qb,Ea,Eb,Fa c,Fa s])) (8)
In formula (7), PbAnd QbIt is b-th of user u respectivelybFoundation characteristic vector sum supplemental characteristic vector；EbIt is b-th of user ub Social insertion vector；WithIt is a-th of user u respectivelyaThe image set liked content embeded matrix and style insertion Matrix；θs=[w4,W3] be the second bottom attention network B otNet2 parameter, wherein w4It is second sigmoid function Parameter, W3It is second matrix parameter；
Step 3.2.2, to b-th of user ubCalculating a-th of user uaSocial influence power when shared weight βabIt carries out Normalized, the weight after being normalized by formula (9)
Step 3.2.3, a-th of user u is obtained according to formula (10)aSocial influence power
Step 3.3 enables CiIndicate i-th of image viUploader, and each image only one uploader, thus by i-th of figure As viUploader influence power be expressed as i-th of image viUploader CiAuxiliary be embedded in vector QCi；
Using the upload consistency, social influence power and uploader influence power as the social semantic information of three aspects；
Step 4, three input A that top layer attention network TopNet is respectively obtained using formula (11)λ, λ=1,2,3:
A-th of user u is being influenced respectively using the social semantic information that formula (12) obtains three aspectsaIt is shared when selection image Weight γaλ:
γaλ=σ (w6×(W5×Aλ)) (12)
In formula (12), θa=[w6,W5] be top layer attention network parameter, w6It is the parameter of third sigmoid function, W5It is Third matrix parameter；
Step 5, according to the social semantic informations of three aspects and its shared weight computing user to the score in predicting value of image, To carry out image recommendation to user:
Step 5.1 obtains a-th of user u according to formula (13)aWith i-th of image viCorresponding prediction scoringTo be used Score in predicting matrix of the family to article
In formula (13), Wi TIt is i-th of image viFoundation characteristic vector transposed vector；
Step 5.2 establishes the loss function L (θ) as shown in formula (14):
In formula (13), θ=[θ1,θ2] it is parameter to be optimized, θ1=[P, Q, W, X] is eigenmatrix, θ2=[θu,θs,θa] it is note The parameter of meaning power network, λ is regularization term, Da=(i, j) | i ∈ Ra∧j∈V-RaIt is a-th of user uaTraining data, Ra It is a-th of user uaThe image set liked；
Step 5.3 optimizes the loss function L (θ) by stochastic gradient descent method, so that L (θ) reaches most It is small, to obtain optimum prediction rating matrix, and image recommendation is carried out to user according to institute's optimum prediction rating matrix.
CN201811200827.1A 2018-10-16 2018-10-16 A kind of social image recommended method based on level attention mechanism Pending CN109410080A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811200827.1A CN109410080A (en) 2018-10-16 2018-10-16 A kind of social image recommended method based on level attention mechanism

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811200827.1A CN109410080A (en) 2018-10-16 2018-10-16 A kind of social image recommended method based on level attention mechanism

Publications (1)

Publication Number Publication Date
CN109410080A true CN109410080A (en) 2019-03-01

Family

ID=65467204

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811200827.1A Pending CN109410080A (en) 2018-10-16 2018-10-16 A kind of social image recommended method based on level attention mechanism

Country Status (1)

Country Link
CN (1) CN109410080A (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110148424A (en) * 2019-05-08 2019-08-20 北京达佳互联信息技术有限公司 Method of speech processing, device, electronic equipment and storage medium
CN110245285A (en) * 2019-04-30 2019-09-17 中国科学院信息工程研究所 A kind of personalized recommendation method based on Heterogeneous Information network
CN110555132A (en) * 2019-08-13 2019-12-10 北京工业大学 Noise reduction self-encoder recommendation method based on attention model
CN112700342A (en) * 2020-12-29 2021-04-23 四川大学 Personalized grading prediction model and prediction method for social media information authenticity
CN112818146A (en) * 2021-01-26 2021-05-18 山西三友和智慧信息技术股份有限公司 Recommendation method based on product image style
CN112905894A (en) * 2021-03-24 2021-06-04 合肥工业大学 Collaborative filtering recommendation method based on enhanced graph learning
CN113010796A (en) * 2021-04-15 2021-06-22 重庆大学 Method for item recommendation
CN113205869A (en) * 2021-04-29 2021-08-03 华东师范大学 Mental health detection and intervention method based on user social software head portrait
CN113658013A (en) * 2021-08-16 2021-11-16 上海花千树信息科技有限公司 Method and device for recommending self-photographing on love or social software
CN114677234A (en) * 2022-04-26 2022-06-28 河南大学 Graph convolution neural network social contact recommendation method and system integrating multi-channel attention mechanism
CN114880583A (en) * 2022-05-11 2022-08-09 合肥工业大学 Cross-domain social recommendation method based on self-supervision learning
CN115618131A (en) * 2022-11-11 2023-01-17 河南大学 Article recommendation method and system fusing interaction frequency and attention network

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108021672A (en) * 2017-12-06 2018-05-11 北京奇虎科技有限公司 Social recommendation method, apparatus and computing device based on photograph album
CN108320187A (en) * 2018-02-02 2018-07-24 合肥工业大学 A kind of recommendation method based on depth social networks
CN108319659A (en) * 2018-01-08 2018-07-24 暨南大学 A kind of social discovery method based on encrypted image fast search
CN108509493A (en) * 2018-02-13 2018-09-07 北京邮电大学 A kind of AR visual search recommendation service method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108021672A (en) * 2017-12-06 2018-05-11 北京奇虎科技有限公司 Social recommendation method, apparatus and computing device based on photograph album
CN108319659A (en) * 2018-01-08 2018-07-24 暨南大学 A kind of social discovery method based on encrypted image fast search
CN108320187A (en) * 2018-02-02 2018-07-24 合肥工业大学 A kind of recommendation method based on depth social networks
CN108509493A (en) * 2018-02-13 2018-09-07 北京邮电大学 A kind of AR visual search recommendation service method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
L. WU: "Modeling the evolution of users’ preferences and social links in social networking services", 《IEEE TRANS. KNOWL. DATA ENG.》 *
LE WU: "Explainable Social Contextual Image Recommendation with Hierarchical Attention", 《HTTPS://ARXIV.ORG/PDF/1806.00723V1.PDF》 *
袁寒雪: "基于反卷积网络的图像模式学习及应用", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110245285B (en) * 2019-04-30 2021-07-13 中国科学院信息工程研究所 Personalized recommendation method based on heterogeneous information network
CN110245285A (en) * 2019-04-30 2019-09-17 中国科学院信息工程研究所 A kind of personalized recommendation method based on Heterogeneous Information network
CN110148424A (en) * 2019-05-08 2019-08-20 北京达佳互联信息技术有限公司 Method of speech processing, device, electronic equipment and storage medium
CN110148424B (en) * 2019-05-08 2021-05-25 北京达佳互联信息技术有限公司 Voice processing method and device, electronic equipment and storage medium
CN110555132A (en) * 2019-08-13 2019-12-10 北京工业大学 Noise reduction self-encoder recommendation method based on attention model
CN112700342A (en) * 2020-12-29 2021-04-23 四川大学 Personalized grading prediction model and prediction method for social media information authenticity
CN112818146A (en) * 2021-01-26 2021-05-18 山西三友和智慧信息技术股份有限公司 Recommendation method based on product image style
CN112905894A (en) * 2021-03-24 2021-06-04 合肥工业大学 Collaborative filtering recommendation method based on enhanced graph learning
CN112905894B (en) * 2021-03-24 2022-08-19 合肥工业大学 Collaborative filtering recommendation method based on enhanced graph learning
CN113010796A (en) * 2021-04-15 2021-06-22 重庆大学 Method for item recommendation
CN113205869A (en) * 2021-04-29 2021-08-03 华东师范大学 Mental health detection and intervention method based on user social software head portrait
CN113205869B (en) * 2021-04-29 2022-10-04 华东师范大学 Mental health detection and intervention method based on user social software head portrait
CN113658013A (en) * 2021-08-16 2021-11-16 上海花千树信息科技有限公司 Method and device for recommending self-photographing on love or social software
CN114677234A (en) * 2022-04-26 2022-06-28 河南大学 Graph convolution neural network social contact recommendation method and system integrating multi-channel attention mechanism
CN114677234B (en) * 2022-04-26 2024-04-30 河南大学 Graph convolution neural network social recommendation method and system integrating multichannel attention mechanisms
CN114880583A (en) * 2022-05-11 2022-08-09 合肥工业大学 Cross-domain social recommendation method based on self-supervision learning
CN114880583B (en) * 2022-05-11 2024-03-05 合肥工业大学 Cross-domain social recommendation method based on self-supervision learning
CN115618131A (en) * 2022-11-11 2023-01-17 河南大学 Article recommendation method and system fusing interaction frequency and attention network
CN115618131B (en) * 2022-11-11 2023-05-23 河南大学 Article recommendation method and system integrating interaction frequency and attention network

Similar Documents

Publication Publication Date Title
CN109410080A (en) A kind of social image recommended method based on level attention mechanism
CN111428147B (en) Social recommendation method of heterogeneous graph volume network combining social and interest information
CN110196946A (en) A kind of personalized recommendation method based on deep learning
CN110473592B (en) Multi-view human synthetic lethal gene prediction method
CN103164713B (en) Image classification method and device
CN110472817A (en) A kind of XGBoost of combination deep neural network integrates credit evaluation system and its method
CN107633513A (en) The measure of 3D rendering quality based on deep learning
CN108320187A (en) A kind of recommendation method based on depth social networks
CN109960737B (en) Remote sensing image content retrieval method for semi-supervised depth confrontation self-coding Hash learning
CN110599234A (en) Product sales prediction method
CN108446312B (en) Optical remote sensing image retrieval method based on deep convolution semantic net
CN109496322A (en) Credit assessment method and device and the progressive decision tree parameter regulation means of gradient and device
CN110399819A (en) A kind of remote sensing image residential block extraction method based on deep learning
CN108550077A (en) A kind of individual credit risk appraisal procedure and assessment system towards extensive non-equilibrium collage-credit data
CN111538827A (en) Case recommendation method and device based on content and graph neural network and storage medium
CN109858566A (en) A method of it being added to the scorecard of mould dimension based on multilayered model building
CN109711401A (en) A kind of Method for text detection in natural scene image based on Faster Rcnn
CN110415022A (en) Handle the method and device of user behavior sequence
CN115917577A (en) System and method for next basket recommendation using dynamic attribute modeling
CN108960488A (en) A kind of accurate prediction technique of saturation loading spatial distribution based on deep learning and Multi-source Information Fusion
CN106407477A (en) Multidimensional interconnection recommendation method and system
CN110070116A (en) Segmented based on the tree-shaped Training strategy of depth selects integrated image classification method
CN110503508A (en) A kind of item recommendation method of the more granularity matrix decompositions of level
CN108491864A (en) Based on the classification hyperspectral imagery for automatically determining convolution kernel size convolutional neural networks
CN109272487A (en) The quantity statistics method of crowd in a kind of public domain based on video

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20190301