CN111159543B - Personalized tourist place recommendation method based on multi-level visual similarity - Google Patents

Personalized tourist place recommendation method based on multi-level visual similarity Download PDF

Info

Publication number
CN111159543B
CN111159543B CN201911311868.2A CN201911311868A CN111159543B CN 111159543 B CN111159543 B CN 111159543B CN 201911311868 A CN201911311868 A CN 201911311868A CN 111159543 B CN111159543 B CN 111159543B
Authority
CN
China
Prior art keywords
user
location
visual
photos
place
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911311868.2A
Other languages
Chinese (zh)
Other versions
CN111159543A (en
Inventor
陈岭
吕丹丹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN201911311868.2A priority Critical patent/CN111159543B/en
Publication of CN111159543A publication Critical patent/CN111159543A/en
Application granted granted Critical
Publication of CN111159543B publication Critical patent/CN111159543B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9535—Search customisation based on user profiles and personalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/53—Querying
    • G06F16/535—Filtering based on additional data, e.g. user or group profiles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/55—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/14—Travel agencies

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Tourism & Hospitality (AREA)
  • Library & Information Science (AREA)
  • Health & Medical Sciences (AREA)
  • Economics (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a personalized tourist place recommendation method based on multilevel visual similarity of a geotagged photo, which comprises the following steps: 1) preprocessing the geotagged photo set, clustering to obtain a travel place, and extracting the times of visiting the travel place by a user; 2) obtaining visual characteristics of the photo by using a VGG16 model; 3) calculating weight values for different photos using a self-attention mechanism to obtain visual representations of the user and the travel location; 4) sampling to obtain implicit vectors of the user and the tourist site based on the visual representation of the user and the tourist site, and predicting the times of visiting the tourist site by the user according to the implicit vectors; 5) training the model based on the integral loss formed by quintuple loss, accuracy loss and regular terms to obtain a parameter-optimized model; 6) given a query, the querying user is recommended travel locations that may be of interest to the querying city. The method excavates user tour preferences from the set of geotagged photos and recommends tour locations to which the user may be interested.

Description

Personalized tourist place recommendation method based on multi-level visual similarity
Technical Field
The invention relates to the technical field of information recommendation, in particular to a personalized tourist site recommendation method based on multilevel visual similarity of a geographic marking photo.
Background
In recent years, with the rapid development of mobile internet, smart phones and photo sharing websites (such as Flickr, Panoramio and Instagram), a large number of geotagged photos appear on the internet, and the number of the geotagged photos contributed by the group is on a rapid growth trend. Based on the geographic labeled photos (hereinafter referred to as photos), tourist locations (hereinafter referred to as locations) in a city can be mined and tourist preferences (hereinafter referred to as preferences) of tourists can be analyzed, so that personalized location recommendation service is further provided for users.
In the early photo mining-based place recommendation method, the similarity among users is usually calculated directly based on the number of times that the users visit places, and then the place is recommended to the users by combining a user-based collaborative filtering method. To improve recommendation performance, a place recommendation method introducing various additional information has appeared. With the development of deep neural networks, the visual content of photographs is receiving more and more attention. Existing visual content-based methods typically first extract features from the visual content of the photograph, and then train a recommendation model using these features as priors in combination with the user history. These methods fail to extract visual features suitable for site recommendations because the extraction of visual features is guided primarily by computer vision tasks unrelated to recommendations.
To solve this problem, the predecessor proposed a visual content enhanced point of interest (POI) recommendation method that extracts features from the visual content of the photos, classifies them according to the photographer and place of the photos, and decomposes the user-POI check-in matrix for personalized recommendation. However, given a photograph, this approach may use the user and location information independently to divide other photographs into visually similar or dissimilar groups, and may not fully utilize the user and location information of photographs to provide multiple levels of similarity. Furthermore, this method does not take into account the degree of importance of the different photos to the user or location.
Disclosure of Invention
The technical problem to be solved by the invention is how to fully utilize the visual difference of pictures taken by different users in different places to obtain the user preference and the place characteristics, thereby further providing personalized place recommendation service for the users.
In order to solve the technical problem, the personalized tourist site recommendation method based on the multilevel visual similarity of the geotagged photos provided by the invention comprises the following steps:
(1) preprocessing a photo set labeled by geography, clustering to obtain a travel location set, and extracting a user set and the times of visiting travel locations by the user;
(2) obtaining visual characteristics of the photo by using a VGG16 model;
(3) calculating weight values for different photos by adopting a self-attention mechanism to obtain visual representations of the user and the place, and obtaining hidden vectors of the user and the place according to the visual representations of the user and the place;
(4) predicting the number of times of the user accessing the location according to the user hidden vector and the location hidden vector;
(5) constructing quintuple loss of the photo according to visual features of the photo, constructing a user regular term according to a user hidden vector, constructing a place regular term according to a place hidden vector, constructing accuracy loss according to access times, calculating total loss according to the quintuple loss, the user regular term, the place regular term and the accuracy loss, and iteratively optimizing model parameters of a VGG16 model and a weight coefficient of an attention mechanism by using the total loss;
(6) and (4) searching and obtaining all candidate places in the query city aiming at a query task comprising the query user and the query city, calculating preference values of the query user on the candidate places according to the query user hidden vector and the candidate place hidden vector obtained in the step (3), and accordingly realizing personalized tourist place recommendation.
Compared with the prior art, the method has the advantages that at least:
1) through the information of the users and the places of the crossed photos, multi-level visual similarity is defined, corresponding quintuple loss is introduced to obtain visual representation of the photos, and the visual difference of the photos shot by different users in different places is fully utilized.
2) The self-attention network is utilized to infer the weight of each photo to characterize the user and location, capturing the importance of different photos.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a personalized travel location recommendation method based on multi-level visual similarity of geotagged photos according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the detailed description and specific examples, while indicating the scope of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
Fig. 1 is a flowchart of a personalized travel location recommendation method based on multi-level visual similarity of geotagged photos according to an embodiment of the present invention. Referring to fig. 1, the personalized tourist site recommendation method includes the steps of:
step 1, inputting a photo set P, clustering photos by using a density-based clustering method, and extracting a location set L; and simultaneously extracting a user set U.
Users typically take pictures at locations where they are of more interest, and if a large number of users take pictures at a location, the location may be considered a location. Clustering the photos according to the longitude and latitude position information corresponding to the photos by using a density-based clustering method (such as P-DBSCAN), wherein each obtained cluster represents a place, and the clustering center is the position of the place. Through the process, a site set L ═ L is excavated1,l2,…,l|L|And f, wherein l is (c, g), c is a city where the location l is located, and g is latitude and longitude information of l. Further, a user set U ═ { U } is extracted from the photographer information of the photograph1,u2,…,u|U|}。
And 2, inputting a photo set P, a user set U and a place set L, and extracting a user access history V.
For user location pair (u)i∈U，ljE.l), first for the ith user u according to the time of taking the pictureiAt jth location ljThe pictures taken are sorted. Considering that a user may take several photos in the same visit, if user uiAt a location ljIf the time interval between several consecutive pictures taken is less than a given time threshold Δ t, the pictures are considered to belong to the same visit, and the average of the times of taking the pictures is used as the time t of the visit, the visit can be expressed as (u)i,ljT). All useful for this treatmentThe user access history V { (u) can be obtained by corresponding photos of the user and the placei,ljT) }, in which (u)i,ljT) represents user uiVisit location l at time tj。
And 3, inputting a user access history V, and extracting the times M of the user access to the place.
Counting the number of times each user visits each place according to the user visit history V, thereby obtaining the number of times M { c } that the user visits the placeijI is more than or equal to 1 and less than or equal to | U |, j is more than or equal to 1 and less than or equal to | L |, wherein cijRepresenting user uiVisit location ljThe number of times.
And 4, dividing the user set U and the place set L into N batches, and simultaneously batching the photo set P and the times M of visiting places by the users according to the corresponding users and places in each batch.
The user set U and the place set L are batched according to the total batch number N set manually by experience to form { U1,U2,…,UNAnd { L }1,L2,…,LN}. For each batch of users UmAnd location Lm(m is more than or equal to 1 and less than or equal to N), all U are found from the photo set PmPictures taken by the inner user and all at LmPictures taken at the interior locations to form a batch of pictures Pm(ii) a Finding all U's from the number M of times a user visits a place at the same timemInter-user access LmNumber of places in the list, and number of times M of visiting places by a group of usersm。
Step 5, taking out a batch of training samples U with index m (m is more than or equal to 1 and less than or equal to N) from a plurality of batchesm，Lm，PmAnd Mm。
Step 6, for each photo p in the batchk∈PmInputting the image into a VGG16 model to obtain the visual characteristic v of the photok。
The VGG16 model is a classic deep learning model in the task of picture classification, and comprises 16 hidden layers (13 convolutional layers and 3 full-link layers). The method extracts the picture p by utilizing the first 14 hidden layers (removing the last 2 full connecting layers) of the VGG16 modelkVisual feature v ofk。
Step 7, for each user u in the batchi∈UmAnd 8-9.
Step 8, fusing the users u by using a self-attention mechanismiThe visual characteristics of the picture are taken to obtain a user uiIs a visual representation u ofi。
First, stack user u by the time of taking the photoiThe visual characteristics of the picture are taken, forming a matrix UPiEach row in the matrix corresponds to a visual characteristic of the corresponding photograph. Fusing users u with a self-attention mechanismiThe specific calculation method of the visual characteristics of the shot picture is as follows:
uai＝softmax(wU tanh(VUUPi T))
ui＝uaiUPi
wherein, wUAnd VUAre learnable network parameters, are weights and bias terms for the self-attention mechanism. uaiIs the weight vector of the photograph. The softmax function ensures that the sum of all calculated weights is 1. According to uaiWeight provided, will UPiThe vector summation in (1) to obtain the user uiIs a visual representation u ofi。
Step 9, taking the mean value as uiSum variance
Figure BDA0002324754700000051
The user implicit vector U is obtained by sampling in Gaussian distributioni。
Given that user preferences primarily depend on visual information, but may also be influenced by other factors, assume a user hidden vector UiIs derived from having a mean value of ui(visual information) and variance
Figure BDA0002324754700000061
(other factors) in a Gaussian distribution, where IUIs and uiAll 1 vectors of the same length.
Step 10, for those in the batchEach location lj∈LmAnd performing the steps 11-12.
Step 11, fusing at the location l by using a self-attention mechanismjThe visual characteristics of the picture are taken to obtain a location ljVisual representation of (l)j。
First, the pictures are stacked at a location l according to the shooting time of the picturesjThe visual characteristics of the picture are taken to form a matrix LPjEach row in the matrix corresponds to a visual characteristic of the corresponding photograph. Fusing at location l with a self-attention mechanismjThe specific calculation method of the visual characteristics of the shot picture is as follows:
laj＝softmax(wLtanh(VLLPj T))
lj＝lajLPj
wherein, wLAnd VLAre learnable network parameters, are weights and bias terms for the self-attention mechanism. lajIs the weight vector of the photograph. According to lajWeight provided, will LPjSumming the vectors to obtain the location ljVisual representation of (l)j。
Step 12, from the mean value of ljSum variance
Figure BDA0002324754700000062
The Gaussian distribution is sampled to obtain a location hidden vector Lj。
Assuming a location hidden vector L, considering that location features mainly depend on visual information, but may also be affected by other factorsjIs derived from having a mean value of lj(visual information) and variance
Figure BDA0002324754700000063
(other factors) in a Gaussian distribution, where ILIs a sum ofjAll 1 vectors of the same length.
Step 13, for each number of visits c in the batchij∈MmFrom the mean value of UiLjSum variance σ2Is sampled in a gaussian distributionGet user uiVisit location ljThe number of times.
Assuming that user u depends primarily on user preferences and location characteristics, but may also be affected by noise, consider that user uiVisit location ljIs from the mean value UiLjSum variance σ2(noise) in a gaussian distribution.
Step 14, from the photos P in the batchmInternally mining quintuple sets for training
Figure BDA0002324754700000064
Two pictures p taken by the same user at the same locationo,
Figure BDA0002324754700000071
A picture taken by another user at the same location
Figure BDA0002324754700000072
A picture taken by the same user at another location
Figure BDA0002324754700000073
And a picture taken by another user at another location
Figure BDA0002324754700000074
May constitute a quintuple
Figure BDA0002324754700000075
After training is finished, quintuple
Figure BDA0002324754700000076
Multiple levels of visual similarity should be satisfied, with the corresponding formalization expressed as follows:
Figure BDA0002324754700000077
Figure BDA0002324754700000078
Figure BDA0002324754700000079
Figure BDA00023247547000000710
Figure BDA00023247547000000711
Figure BDA00023247547000000712
wherein v iso，
Figure BDA00023247547000000713
And
Figure BDA00023247547000000714
are each po，
Figure BDA00023247547000000715
And
Figure BDA00023247547000000716
the visual characteristics of (1). m is1，m2，m3，m4，m5And m6Are respectively a photo pair
Figure BDA00023247547000000717
And
Figure BDA00023247547000000718
Figure BDA00023247547000000719
and
Figure BDA00023247547000000720
and
Figure BDA00023247547000000721
and
Figure BDA00023247547000000722
and
Figure BDA00023247547000000723
and
Figure BDA00023247547000000724
and
Figure BDA00023247547000000725
must satisfy the minimum visual distance between, and satisfy m1<m2<m3，m4<m5。
Quintuple, which has satisfied the above multi-level visual similarity, does not contribute to training, resulting in a slow convergence rate. To ensure fast convergence, for any po∈PmSelecting all other photos taken by the same user at the same place as
Figure BDA00023247547000000726
And select PmAll photos satisfying the following inequality are taken as
Figure BDA00023247547000000727
And
Figure BDA00023247547000000728
Figure BDA00023247547000000729
Figure BDA00023247547000000730
Figure BDA00023247547000000731
Figure BDA0002324754700000081
Figure BDA0002324754700000082
Figure BDA0002324754700000083
in this way, PmAll the pictures can obtain quintuple set for training
Figure BDA0002324754700000084
Wherein p iso,
Figure BDA0002324754700000085
Step 15, for each five-tuple excavated
Figure BDA0002324754700000086
Calculating the corresponding quintuple loss LQ。
The triplet loss calculation method corresponding to each inequality representing multi-level visual similarity in the previous step is as follows:
Figure BDA0002324754700000087
Figure BDA0002324754700000088
Figure BDA0002324754700000089
Figure BDA00023247547000000810
Figure BDA00023247547000000811
Figure BDA00023247547000000812
wherein when [ · [ ]]+Internal value being positive, [ ·]+Take this value, otherwise 0.
And adding the triad losses to obtain a final quintuple loss, wherein the specific calculation mode is as follows:
LQ＝L1+L2+L3+L4+L5+L6
step 16, for each number of visits c in the batchij∈MmCalculating the loss of accuracy LH。
Calculating and sampling user uiVisit location ljNumber of times of (2) and number of true accesses cijThe square of the error between, the accuracy loss L is obtainedHThe specific calculation method is as follows:
LH＝(cij-UiLj)2
step 17, for each user u in the batchi∈UmCalculating the user regularization term LU。
Calculating the distance between the user hidden vector and the user visual representation to obtain a user regular term LUThe specific calculation method is as follows:
Figure BDA0002324754700000091
wherein
Figure BDA0002324754700000092
The Frobenius norm of the matrix is represented.
For each location l in the batch, step 18j∈LmComputing a locality regularization term LL。
Calculating the distance between the hidden location vector and the visual location representation to obtain a location regular term LLThe specific calculation method is as follows:
Figure BDA0002324754700000093
and 19, calculating the total loss L of all samples in the batch, and adjusting the network parameters in the whole model.
The total loss L for all samples in the batch was calculated in the following manner:
Figure BDA0002324754700000094
wherein
Figure BDA0002324754700000095
Quintuple loss, accuracy loss, user regularization term, and place regularization term, respectively, for a single sample. Θ represents the parameters of the VGG16 model as well as the weight and bias terms of the self-attention mechanism.
Figure BDA0002324754700000096
λnAnd respectively representing the weight of the user regular term, the location regular term and the parameter regular term. Then, according to the loss L, the network parameters in the whole model are adjusted.
Step 20, repeat steps 6-19 until all batches of the training data set have been engaged in model training.
And step 21, repeating the steps 5-20 until the specified iteration number is reached.
Step 22, given the query q ═ u, c, find all candidate locations in the query city c
Figure BDA0002324754700000097
Step 23, calculating candidate location of query user u
Figure BDA0002324754700000098
And returns the K-top ranked places as recommendation results.
Finding out a hidden vector u and a candidate location corresponding to the query user u
Figure BDA0002324754700000101
Each of which is
Figure BDA0002324754700000102
Corresponding hidden vector
Figure BDA0002324754700000103
Calculating the query user u for each place
Figure BDA0002324754700000104
The specific calculation of the preference value is as follows:
Figure BDA0002324754700000105
and sorting the calculated preference values in a descending order, and returning the place K before the ranking as a recommendation result.
The above-mentioned embodiments are intended to illustrate the technical solutions and advantages of the present invention, and it should be understood that the above-mentioned embodiments are only the most preferred embodiments of the present invention, and are not intended to limit the present invention, and any modifications, additions, equivalents, etc. made within the scope of the principles of the present invention should be included in the scope of the present invention.

Claims (10)

1. A personalized tourist place recommendation method based on multilevel visual similarity of a geographic marking photo comprises the following steps:
(1) preprocessing a photo set labeled by geography, clustering to obtain a travel location set, and extracting a user set and the times of visiting travel locations by the user;
(2) obtaining visual characteristics of the photo by using a VGG16 model;
(3) calculating weight values for different photos by adopting a self-attention mechanism to obtain visual representations of the user and the place, and obtaining hidden vectors of the user and the place according to the visual representations of the user and the place;
(4) predicting the number of times of the user accessing the location according to the user hidden vector and the location hidden vector;
(5) constructing quintuple loss of the photo according to visual features of the photo, constructing a user regular term according to a user hidden vector, constructing a place regular term according to a place hidden vector, constructing accuracy loss according to access times, calculating total loss according to the quintuple loss, the user regular term, the place regular term and the accuracy loss, and iteratively optimizing model parameters of a VGG16 model and a weight coefficient of an attention mechanism by using the total loss;
(6) and (4) searching and obtaining all candidate places in the query city aiming at a query task comprising the query user and the query city, calculating preference values of the query user on the candidate places according to the query user hidden vector and the candidate place hidden vector obtained in the step (3), and accordingly realizing personalized tourist place recommendation.
2. The method for recommending personalized tourist sites based on multilevel visual similarity of geotagged photos as claimed in claim 1, wherein in step (1), the photos are clustered by using a density-based clustering method according to the longitude and latitude position information corresponding to the photos, each obtained cluster represents a site, and the clustering center is the position of the site; through the process, a site set L ═ L is excavated1,l2,…,l|L|Where l ═ c, g)C is the city where the location l is located, and g is the longitude and latitude information of l;
extracting a user set U-U according to the photographer information of the photo1,u2,…,u|U|}。
3. The method for recommending personalized tourist sites based on multi-level visual similarity of geotagged photos according to claim 1, wherein in step (1), the user site pair (u) is pointed outi∈U，ljE.l), first for the ith user u according to the time of taking the pictureiAt jth location ljSequencing the shot photos;
considering that a user may take several photos in the same visit, if user uiAt a location ljIf the time interval between several consecutive pictures taken is less than a given time threshold Δ t, the pictures are considered to belong to the same visit, and the average of the times of taking the pictures is used as the time t of the visit, the visit can be expressed as (u)i,ljT); the user access history V { (u) can be obtainedi,ljT) }, in which (u)i,ljT) represents user uiVisit location l at time tj；
Counting the number of times each user visits each place according to the user visit history V, thereby obtaining the number of times M { c } that the user visits the placeijI is more than or equal to 1 and less than or equal to | U |, j is more than or equal to 1 and less than or equal to | L |, wherein cijRepresenting user uiVisit location ljThe number of times.
4. The method for recommending personalized tourist sites based on multi-level visual similarity of geotagged photos according to claim 1, wherein in step (3),
first, stack user u by the time of taking the photoiThe visual characteristics of the picture are taken, forming a matrix UPiEach row in the matrix corresponds to the visual characteristics of the corresponding photo, and the user u is fused by utilizing a self-attention mechanismiThe specific calculation method of the visual characteristics of the shot picture is as follows:
uai＝softmax(wUtanh(VUUPi T))
ui＝uaiUPi
wherein, wUAnd VUFor learnable network parameters, for weights and bias terms of the self-attention mechanism, uaiIs the weight vector of the photo, the softmax function ensures that the sum of all calculated weights is 1;
then, from the mean value uiSum variance
Figure FDA0003392845340000021
The user implicit vector U is obtained by sampling in Gaussian distributioniIn which IUIs and uiAll 1 vectors of the same length.
5. The method as claimed in claim 1, wherein the personalized tourist spot recommendation method based on the multi-level visual similarity of the geotagged photos is implemented in step (3), and the photos are firstly stacked at the spot l according to the shooting time of the photosjThe visual characteristics of the picture are taken to form a matrix LPjEach row in the matrix corresponds to the visual characteristics of the corresponding photo and is fused at the location l by using a self-attention mechanismjThe specific calculation method of the visual characteristics of the shot picture is as follows:
laj＝softmax(wLtanh(VLLPj T))
lj＝lajLPj
wherein, wLAnd VLFor learnable network parameters, for the weights and bias terms of the self-attention mechanism, lajIs the weight vector of the photograph, according to lajWeight provided, will LPjSumming the vectors to obtain the location ljVisual representation of (l)j；
Then, from the mean value of ljSum variance
Figure FDA0003392845340000031
Is sampled in a gaussian distributionObtaining a location implicit vector LjIn which ILIs a sum ofjAll 1 vectors of the same length.
6. The method of claim 1, wherein in the step (4), the number of visits c to each of the plurality of groups is determinedij∈MmFrom the mean value of UiLjSum variance σ2The user u is obtained by sampling in the Gaussian distributioniVisit location ljThe number of accesses of (c).
7. The method for recommending personalized tourist sites based on multi-level visual similarity of geotagged photos according to claim 1, wherein in step (5),
two pictures p taken by the same user at the same locationo，
Figure FDA0003392845340000032
Finding all batches of users U from photo collection PmPictures taken internally and all at location LmPictures taken at the interior locations to form a batch of pictures PmA picture taken by another user at the same location
Figure FDA0003392845340000033
A picture taken by the same user at another location
Figure FDA0003392845340000034
And a picture taken by another user at another location
Figure FDA0003392845340000035
May constitute a quintuple
Figure FDA0003392845340000036
After training is finished, quintuple
Figure FDA0003392845340000041
Multiple levels of visual similarity should be satisfied, with the corresponding formalization expressed as follows:
Figure FDA0003392845340000042
Figure FDA0003392845340000043
Figure FDA0003392845340000044
Figure FDA0003392845340000045
Figure FDA0003392845340000046
Figure FDA0003392845340000047
wherein v iso，
Figure FDA0003392845340000048
And
Figure FDA0003392845340000049
are each po，
Figure FDA00033928453400000410
And
Figure FDA00033928453400000411
the visual characteristics of (1), m1, m2, m3, m4, m5 and m6 are photo pairs respectively
Figure FDA00033928453400000412
And
Figure FDA00033928453400000413
Figure FDA00033928453400000414
and
Figure FDA00033928453400000415
and
Figure FDA00033928453400000416
and
Figure FDA00033928453400000417
and
Figure FDA00033928453400000418
and
Figure FDA00033928453400000419
and
Figure FDA00033928453400000420
must satisfy the minimum visual distance between, and satisfy m1＜m2＜m3，m4＜m5；
To ensure fast convergence, for any po∈PmSelecting all other photos taken by the same user at the same place as
Figure FDA00033928453400000421
And select PmAll photos satisfying the following inequality are taken as
Figure FDA00033928453400000422
And
Figure FDA00033928453400000423
Figure FDA00033928453400000424
Figure FDA00033928453400000425
Figure FDA00033928453400000426
Figure FDA00033928453400000427
Figure FDA00033928453400000428
Figure FDA00033928453400000429
in this way, PmAll the pictures can obtain quintuple set for training
Figure FDA00033928453400000430
Wherein p iso，
Figure FDA00033928453400000431
The triplet loss calculation for each inequality representing multi-level visual similarity is as follows:
Figure FDA0003392845340000051
Figure FDA0003392845340000052
Figure FDA0003392845340000053
Figure FDA0003392845340000054
Figure FDA0003392845340000055
Figure FDA0003392845340000056
wherein when [ · [ ]]+Internal value being positive, [ ·]+Taking the value, otherwise, taking the value as 0;
and adding the triad losses to obtain a final quintuple loss, wherein the specific calculation mode is as follows:
LQ＝L1+L2+L3+L4+L5+L6。
8. the method as claimed in claim 1, wherein the step (5) of calculating the hidden vector U of the user is performed by using a personalized tourist spot recommendation method based on multi-level visual similarity of the geotagged photosiAnd a user visual representation uiDistance between them, obtaining user regular term LUThe specific calculation method is as follows:
Figure FDA0003392845340000057
computing a locality-hidden vector LjAnd a location visual representation ljDistance between them, get the location regularization term LLThe specific calculation method is as follows:
Figure FDA0003392845340000058
calculating and sampling user uiVisit location ljNumber of times of (2) and number of true accesses cijThe square of the error between, the accuracy loss L is obtainedHThe specific calculation method is as follows:
LH＝(cij-UiLj)2
wherein
Figure FDA0003392845340000059
Frobenius norm representing matrix, i being user index, j being location index, cijRepresenting user uiVisit location ljThe number of times.
9. The method for recommending personalized tourist sites based on multi-level visual similarity of geotagged photos as claimed in claim 1, wherein in step (5), the specific calculation manner of the total loss L is as follows:
Figure FDA0003392845340000061
wherein
Figure FDA0003392845340000062
Quintuple loss, accuracy loss, user regularization term and location regularization term, respectively, of a single sample, Θ represents the parameters of the VGG16 model and the weight and bias terms of the self-attention mechanism,
Figure FDA0003392845340000063
λnand respectively representing the weight of the user regular term, the location regular term and the parameter regular term.
10. The method as claimed in claim 1, wherein in step (6), the hidden vector u and the candidate location corresponding to the query user u are found
Figure FDA0003392845340000064
Each of which is
Figure FDA0003392845340000065
Corresponding hidden vector
Figure FDA0003392845340000066
Calculating the query user u for each place
Figure FDA0003392845340000067
The specific calculation of the preference value is as follows:
Figure FDA0003392845340000068
and sorting the calculated preference values in a descending order, and returning the place K before the ranking as a recommendation result.
CN201911311868.2A 2019-12-18 2019-12-18 Personalized tourist place recommendation method based on multi-level visual similarity Active CN111159543B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911311868.2A CN111159543B (en) 2019-12-18 2019-12-18 Personalized tourist place recommendation method based on multi-level visual similarity

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911311868.2A CN111159543B (en) 2019-12-18 2019-12-18 Personalized tourist place recommendation method based on multi-level visual similarity

Publications (2)

Publication Number Publication Date
CN111159543A CN111159543A (en) 2020-05-15
CN111159543B true CN111159543B (en) 2022-04-05

Family

ID=70557245

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911311868.2A Active CN111159543B (en) 2019-12-18 2019-12-18 Personalized tourist place recommendation method based on multi-level visual similarity

Country Status (1)

Country Link
CN (1) CN111159543B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117390290B (en) * 2023-12-08 2024-03-15 安徽省立医院(中国科学技术大学附属第一医院) Method for learning dynamic user interests based on language model of content enhancement

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103064924A (en) * 2012-12-17 2013-04-24 浙江鸿程计算机系统有限公司 Travel destination situation recommendation method based on geotagged photo excavation
CN110134885A (en) * 2019-05-22 2019-08-16 广东工业大学 A kind of point of interest recommended method, device, equipment and computer storage medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103064924A (en) * 2012-12-17 2013-04-24 浙江鸿程计算机系统有限公司 Travel destination situation recommendation method based on geotagged photo excavation
CN110134885A (en) * 2019-05-22 2019-08-16 广东工业大学 A kind of point of interest recommended method, device, equipment and computer storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Weighted multi-information constrained matrix factorization for personalized travel location recommendation based on geo-tagged photos;Dandan Lyu等;《Springer Science+Business Media》;20191024;第1-15页 *

Also Published As

Publication number Publication date
CN111159543A (en) 2020-05-15

Similar Documents

Publication Publication Date Title
CN110162593B (en) Search result processing and similarity model training method and device
CN110598130B (en) Movie recommendation method integrating heterogeneous information network and deep learning
CN110647904B (en) Cross-modal retrieval method and system based on unmarked data migration
Xing et al. Points-of-interest recommendation based on convolution matrix factorization
CN114298122B (en) Data classification method, apparatus, device, storage medium and computer program product
CN109947987B (en) Cross collaborative filtering recommendation method
CN109471982B (en) Web service recommendation method based on QoS (quality of service) perception of user and service clustering
CN108897791B (en) Image retrieval method based on depth convolution characteristics and semantic similarity measurement
CN108897750B (en) Personalized place recommendation method and device integrating multiple contextual information
CN113268669B (en) Relation mining-oriented interest point recommendation method based on joint neural network
CN113255714A (en) Image clustering method and device, electronic equipment and computer readable storage medium
Li et al. Where you instagram? associating your instagram photos with points of interest
CN111382283A (en) Resource category label labeling method and device, computer equipment and storage medium
Zhuang et al. Anaba: An obscure sightseeing spots discovering system
CN115408618B (en) Point-of-interest recommendation method based on social relation fusion position dynamic popularity and geographic features
CN115712780A (en) Information pushing method and device based on cloud computing and big data
CN113537304A (en) Cross-modal semantic clustering method based on bidirectional CNN
CN115422441A (en) Continuous interest point recommendation method based on social space-time information and user preference
CN113821657A (en) Artificial intelligence-based image processing model training method and image processing method
CN110598126B (en) Cross-social network user identity recognition method based on behavior habits
CN117312681A (en) Meta universe oriented user preference product recommendation method and system
CN115600017A (en) Feature coding model training method and device and media object recommendation method and device
CN111538916A (en) Interest point recommendation method based on neural network and geographic influence
CN111159543B (en) Personalized tourist place recommendation method based on multi-level visual similarity
CN110543601B (en) Method and system for recommending context-aware interest points based on intelligent set

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant