CN115827974A - Next interest point recommendation system based on spatio-temporal information representation - Google Patents
Next interest point recommendation system based on spatio-temporal information representation Download PDFInfo
- Publication number
- CN115827974A CN115827974A CN202211552543.5A CN202211552543A CN115827974A CN 115827974 A CN115827974 A CN 115827974A CN 202211552543 A CN202211552543 A CN 202211552543A CN 115827974 A CN115827974 A CN 115827974A
- Authority
- CN
- China
- Prior art keywords
- user
- representation
- time
- check
- granularity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a next interest point recommendation system based on spatio-temporal information representation, which relates to the technical field of recommendation and comprises a time personalized module, wherein the time personalized representation T of the time T to a user u is obtained u (t); a time coding module for obtaining a time coding representation phi (T; a user sign-in sequence module, with T u (t), the embedded expression of phi (t and the sign-in interest point of the user u) is used as input, and the embedded expression S of the sign-in sequence of the user is calculated u (ii) a A causal convolution enhancement module; a new check-in sequence module of the user u calculates the new check-in sequence representation Z of the user u u (ii) a And an output module. The invention designs personalized multi-granularity periodic representation and calculates check-inThe representation of the time interval and the geographic distance is considered in attention so as to utilize the space-time information in modeling the long-term preference of the user, and the causal convolution is utilized for local information enhancement to improve the recommendation performance of the next interest point.
Description
Technical Field
The invention relates to the technical field of recommendation, in particular to a next interest point recommendation system based on spatio-temporal information representation.
Background
The popularity of smart devices and mobile internet has promoted the growing popularity of Location-based Social Networks (lbs ns), such as Foursquare, gowalla, yelp, twitter, wechat, and micro blogging, attracting a large number of users. People sign in to places through the sign-in function provided by the LBSs platform, share own dynamic and real-time positions with friends, and issue information such as opinions, photos and comments related to the places to interact with the friends, and the social contact mode increasingly permeates the daily life of the public and gradually develops into an important communication mode in the life of people.
As one of the core functions of the lbs ns, the next point of interest recommendation technology has a wide application scenario and plays a crucial role in the life of people. For the government, by predicting the next point of interest to be visited by people, the government can design more reasonable traffic planning and scheduling strategies to alleviate traffic congestion and crowd gathering; for platforms such as taxi taking, carpooling and delivery, the next interest point prediction technology can accurately help a driver or a takeaway rider to effectively avoid a congested road section and plan a trip in advance; for merchants, the store information and the coupons can be accurately distributed to target users who are likely to visit, so that blind large-scale advertising is avoided, targeted advertising is realized, and the advertising cost is saved; for the user, the next interest point recommendation technology can assist the user in making decisions and improve the user experience. As an independent sub-field of a recommendation system, the next interest point recommendation is widely applied, and can provide better user experience and third-party service for users, so that the system has gained wide attention in academia and industry in recent years.
In the next point of interest recommendation task, the user's interests may be divided into long-term preferences and short-term preferences. The long-term preference refers to the comprehensive interests of the user mined from the historical track of the user and depends on all historical check-in records; the short-term preference means that the interest preference in the short-term range of the user is more inclined to the latest check-in record under the influence of the recently accessed interest points. The time of the user's check-in contains two pieces of information. On one hand, the timestamp reflects the absolute time of the user for accessing the interest point, including the periods of different granularities of year, month, week, day, hour and the like; on the other hand, the time interval between check-ins reflects the degree of correlation between two check-ins. Therefore, the movement rule of the user can be better mined by reasonably utilizing the time information. To model the periodicity of human movement with time information, some models propose embedded representation of time, such as TMCA partitioning time in hours as granularity, and differentiating weekdays and weekends, thereby representing time as a one-hot vector of dimension 48 as an input to the model. But the model only considers the period of hours and weeks, ignores the information of the period of months and days, and considers that adjacent periods are independent from each other and does not consider the proximity of the adjacent periods. In addition, methods such as TMCA, tiSASRec and STAN in the recommendation task utilize spatio-temporal information by learning the representation of time interval and geographical distance in model design, but the current model has limited capability of learning the representation of time interval and geographical distance and cannot reveal the influence of real time or geographical distance effect on human movement.
Different from pure online interactive digital product recommendation such as commodity recommendation, news recommendation and music recommendation, implicit feedback data such as browsing and clicking are not available in check-in behaviors of users, human activities show a complex transfer rule under the influence of practical factors, and a data set has sparsity, so that a next interest point recommendation task has great challenge.
Disclosure of Invention
In view of the foregoing defects of the prior art, the present invention provides a next point of interest recommendation system based on spatio-temporal information representation to improve the performance of next point of interest recommendation.
The invention provides a next interest point recommendation system based on spatio-temporal information representation, which comprises:
the time individuation module is used for individualizing the cycle information of four granularities of month, week, day and hour of the sign-in timestampIn the periodic representation of any one of the four granularities, the time interval of the check-in time t is combined with the representation of the adjacent time interval in a self-adaptive manner by adopting an attention mechanism to obtain the personalized representation of the time interval of the granularity to the user u
By using the said
Calculating to obtain the time personalized representation T of the time T to the user u by adopting an attention mechanism u (t)；
The time coding module is used for mapping the sign-in timestamp to a vector space by using a time coding function to obtain a time coding representation phi (t);
user sign-in sequence module, with the T u (t), the embedded expression of phi (t) and the sign-in interest point of the user u is used as input, and the embedded expression S of the sign-in sequence of the user is calculated u ；
And the causal convolution enhancement module is used for taking the embedded expression of the check-in interest point of the user u as input, combining the causal convolution and the Transformer to enhance the local information of the check-in sequence of the user to obtain the embedded expression S 'after the causal convolution enhancement' u ；
New sign-in sequence module of user, and S u S 'to' u And embedded representation of spatial relationships E Δ Calculating a new sign-in sequence representation Z for said user u ；
And the output module is used for calculating the preference of the user u on the interest point at the moment t by using the new sign-in sequence representation of the user u, predicting and recommending the next interest point.
The technical effects are as follows:
aiming at the next interest point recommendation problem based on spatio-temporal information representation, the invention designs personalized four granularity period representations, combines time coding representation and embedded representation of geographic distance, considers the representation of time interval and geographic distance when calculating attention between check-in, so as to utilize spatio-temporal information and cause-effect convolution to enhance local information when modeling long-term preference of users, and improve the performance of next interest point recommendation.
The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.
Drawings
FIG. 1 is a block diagram of a next point of interest recommendation system in an embodiment of the invention;
FIG. 2 is a schematic diagram of a single user continuous check-in activity in accordance with an embodiment of the present invention
FIG. 3 is a schematic diagram of the STIRSAN model of the present invention;
FIG. 4.A is a thermodynamic diagram of a check-in point of interest category at month granularity in a NYC dataset according to an embodiment of the present invention;
FIG. 4.B is a thermodynamic diagram of a check-in point of interest category at a date granularity in a NYC dataset according to an embodiment of the present invention;
FIG. 4.C is a thermodynamic diagram of a check-in point of interest category at week granularity in an NYC dataset according to an embodiment of the present invention;
FIG. 4.D is a thermodynamic diagram of a check-in point of interest category in NYC data set at hour granularity in accordance with an embodiment of the present invention;
FIG. 5 is a schematic diagram of a causal convolution according to an embodiment of the present invention;
FIG. 6.A is a trend graph of Recall @10 in the NYC dataset as a function of dimension according to an embodiment of the present invention;
FIG. 6.B is a trend graph of the change of Recall @20 with dimension in the NYC dataset according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating the impact of the TKY dataset dimension on STIRLSAN according to an embodiment of the present invention;
FIG. 8 is a diagram illustrating the effect of sequence length on STIRLSAN in a TKY dataset according to an embodiment of the present invention;
FIG. 9.A is a graph showing a similarity thermodynamic diagram in a Month cycle in an embodiment of the present invention;
FIG. 9.B is a graph showing a similarity thermodynamic diagram in Date cycles in accordance with an embodiment of the present invention;
FIG. 9.c is a thermodynamic diagram illustrating the similarity in DayofWeek cycles in accordance with an embodiment of the present invention;
FIG. 9.D is a graphical representation of a similarity thermodynamic diagram in Hour cycles in accordance with an embodiment of the present invention;
FIG. 10.A is a thermodynamic diagram of example 1 of a geographical distance versus space relationship in an embodiment of the present invention;
FIG. 10.B is a thermodynamic diagram of example 2 of a geographical distance and spatial relationship in an embodiment of the present invention;
FIG. 10.C is an example 3 thermodynamic diagram of geographic distance versus space relationships in an embodiment of the invention;
fig. 10.D is a thermodynamic diagram of example 4 of a geographical distance and spatial relationship in an embodiment of the invention.
Detailed Description
The technical contents of the preferred embodiments of the present invention will be more clearly and easily understood by referring to the drawings attached to the specification. The present invention may be embodied in many different forms of embodiments and the scope of the invention is not limited to the embodiments set forth herein.
In the embodiment of the invention, the variables and the mathematical expression thereof are defined as follows: with U = { U = 1 ,u 2 ,...u |U| The user is checked in, U represents one element of the set of users, and the total number of the users is | U |; with L = { L 1 ,l 2 ,...l |L| Defining a set of interest points in the check-in data, wherein the total number of the checked-in interest points is | L |; by p k ＝(lon k ,lat k ) Represents a point of interest l k I.e. latitude and longitude. Sign in the k time of user u
Is marked as
Indicates that user u is at t k At all times visit the place
(ii) a Record the track of each user as
. Tailoring user trajectory intoFixed length
Where N is the maximum length of the defined track. If N < m, only the most recent N check-ins are considered; if N > m, 0 is added to the left side of the sequence until the length of the sequence is N. In addition, the time interval between the i-th and j-th check-in is denoted as Δ T ij ＝|t i -t j | geographical distance is recorded as Δ D ij ＝Haversine(p i ,p j ) Wherein Haverine formula is as follows:
wherein R represents the earth radius and is 6371km. The geographic separation Δ D between each two within a check-in sequence is therefore expressed as:
the next point of interest recommendation refers to the check-in record tra (u) of a given user u, and predicts the point of interest that the user will visit next time.
As shown in fig. 1, an embodiment of the present invention provides a next point of interest recommendation system based on spatio-temporal information representation, including:
the time individuation module individualizes cycle information of four granularities of month, week, day and hour of the sign-in timestamp, adopts an attention mechanism to adaptively combine a time interval of a sign-in time t with the representation of an adjacent time interval for the cycle representation of any one of the four granularities to obtain the individuation representation of the time interval of the granularity for a user u
By using the said
Calculating to obtain a time personalized table of the time t for the user u by adopting an attention mechanismShow T u (t)；
The time coding module is used for mapping the sign-in timestamp to a vector space by using a time coding function to obtain a time coding representation phi (t);
user sign-in sequence module, with the T u (t), the embedded expression of phi (t) and the sign-in interest point of the user u is used as input, and the embedded expression S of the sign-in sequence of the user is calculated u ；
And the causal convolution enhancement module is used for combining causal convolution and Transformer to enhance local information of the user check-in sequence by taking the embedded type of the check-in interest point of the user u as input to obtain an embedded representation S 'after the causal convolution enhancement' u ；
New user' S check-in sequence module, and S u S 'to' u And embedded representation of spatial relationships E Δ Calculating a new sign-in sequence representation Z of the user u u ；
And the output module is used for calculating the preference of the user u to the interest points at the moment t by using the new sign-in sequence representation of the user u, predicting and recommending the next interest point.
The embodiment of the invention designs four personalized granularity period representations by utilizing the multi-granularity periodicity presented by people when going out and considering the personalized influence of multi-granularity period information on users, and performs coded representation and embedded representation on the time interval and the geographic distance between check-in based on the Bochner theory and the AutoDis method respectively by combining the coded representation of time and the embedded representation of the geographic distance, thereby overcoming the defects of the existing work and considering the representation of the time interval and the geographic distance between check-in to capture the spatio-temporal effect when calculating self-attitude. In addition, the method utilizes the causal convolution to enhance the perception of the Transformer on the local sequence, and utilizes the causal convolution to enhance the local information, thereby improving the performance of recommending the next interest point.
The following describes the present embodiment with reference to the drawings, and fig. 2 is a schematic diagram of a single user check-in activity in an embodiment data set. Wherein p is 1 ,2,3,..., i Is a point of interest sequence; Δ t 1 ,Δt 2 ,…,Δt i-1 Is adjacent toThe time interval between two check-in records; Δ d 1 ,Δd 2 ,…,Δd i-1 Is the geographic distance between two adjacent points of interest. The embodiment of the invention only considers the latest N times of sign-in data of the user to model the spatio-temporal information representation of the learning data and predicts the next interesting place.
FIG. 3 is a schematic diagram of a model structure according to a preferred embodiment of the present invention, where the labels are defined as: distance Interval Embedding: a geographic distance embedded representation; inputs: inputting; input Embedding: an embedded representation of a check-in point of interest; causal Convolition Layer: a causal convolutional layer; user embedding: a user embedded representation; time Interval Encoding: a time-coded representation; PMGP: coding the personalized representation of the user u at the moment t; positional Encoding: a position-coded representation; self-orientation Layer: a self-attentive layer; feed-Forward Network: a feed-forward network layer; dropout: generalization of the model; layerNorm Layer normalization. In the Dropout, during forward propagation, the activation value of a certain neuron stops working with a certain probability p, so that the model generalization is stronger, and the occurrence of an overfitting phenomenon is relieved.
The original data in the embodiment of the invention is the sign-in record of the user and comprises information such as user number, interest location type, interest location longitude and latitude and the like. The check-in record of the user can be regarded as a sequence of interest points arranged according to time, and the next interest point recommending task can be essentially regarded as a sequence prediction problem. And grouping the original data sets according to the user numbers, and sequencing according to the sign-in time sequence to generate sign-in sequence data of each user. For each user, sign-ins for the [2, N ] th time are taken as a test set, and sign-in interest points for the Nth time are predicted by taking sign-ins for the [2, N-1] th time as an input sequence.
For user u, the sub-attributes Month, dayOfWeek, date, hour of the check-in time are extracted. The user u is subjected to embedded expression to obtain e u And randomly initializing embedded expressions of four granularities of month, week, day and hour to obtain personalized information expression of four granularities of the user u.
As can be seen with reference to fig. 4.a-4.d, the categories of check-in points of interest exhibited by adjacent time periods are similar, and therefore adjacent time periods should have a relatively similar representation. Therefore, in the embodiment of the present invention, for the granularity s, an attention mechanism is adopted to adaptively combine the time period of the time t with the representation of the adjacent time period, so as to obtain the personalized representation of the time period of the granularity to the user u.
And calculating to obtain the time personalized representation of the moment t to the user u by adopting an attention mechanism for the period representations of the four granularities.
Since time intervals play an important role in expressing time effects and revealing sequence patterns, embodiments of the present invention consider the representation of time intervals to represent Φ as a time code (t counts the time interval to measure the relationship between two timestamps.
To overcome the disadvantages of the Transformer, stirlsan combines causal convolution with the Transformer to enhance local information in the sequence, since Transformer is naturally adept at capturing long-term, global dependencies in the sequence, but Transformer is not adept at extracting fine-grained local information, while convolution operations can explore local information well. Different from the traditional convolution operation, the causal convolution aims at ensuring that information leakage at a future moment does not exist in the calculation process, in order to realize the causal convolution, the length of the sequence is required to be complemented to be 0 (kernel size-1), and the final redundant output of the sequence is required to be truncated to ensure that the lengths of the sequence before and after the convolution are consistent. Thus, the input of the convolution at the time t only contains the information of the input at the time before the time t and does not contain the information at the time t +1 and later. FIG. 5 is a diagram of a causal convolution with a convolution kernel size of 3, resulting in a causal convolution of
Input of causal convolution
Sign-in interests for each of user uAn embedded representation of the point.
An embedded representation of geographic distance is another important factor in the next point of interest recommendation problem. For the embedded representation of continuous numerical features, a simple solution is to use the numerical features as classification features and allocate an independent embedded vector to each numerical feature, but the method has serious defects such as large parameter quantity, insufficient low-frequency feature training and the like. However, the capacity of the method model is low, so that the performance is reduced. Therefore, the invention designs a meta-embedding H epsilon R K×d Shared for all geographical distances. Each element is embedded in h v ∈R d Can be viewed as a subspace within the hidden space to improve the expressive power and capacity of the model. In order to capture the complex relation between the geographic distance and the meta embedding, a differentiable automatic discrete module is designed, and the geographic distance delta D between the ith check-in point of interest and the jth check-in point of interest is obtained in a weighted average mode ij Embedded representation of
The weighted averaging approach may make the relevant meta-embedding more advantageous for providing rich information, while irrelevant meta-embedding may be largely ignored, thereby resulting in an embedded representation E of the geographic distance Δ D between user u checks ΔD And further obtaining an embedded representation E of the spatial relationship Δ 。
STIRSAN aggregates representations of historical check-ins by computing self-attributes to assign different weights to each check-in the trace. The input of spatio-temporal perception self-attention is an embedded expression S of the user check-in sequence u The embedded expression of the user check-in sequence local information subjected to causal convolution enhancement is S' u Embedded representation of said spatial relationship E Δ In addition to calculating the similarity between the check-in points of interest in the process of calculating attention, the method also takes into accountThe influence of the time interval and the geographic distance between each check-in obtains a new check-in sequence representation Z of the user u u 。
Calculating the preference of the user u to the interest point at the moment t by using a potential factor model according to the new sign-in sequence representation of the user u; and carrying out model training, learning model parameters and predicting the next interest point.
The set of four granularity period information is represented by S = { Month, dayOfWeek, date, hour }, S ∈ S, S represents an element in the set, namely, one of the four granularities, and in a preferred embodiment of the present invention, for granularity S, the embedded representation of the user u at the time t is represented as
Wherein, when s represents month, it is represented as
Wherein E is Month (t)∈R d Representing an embedded representation of the month of said time t, R representing a real number field, d being the dimension of the embedded representation,
representing the Hadamard product, W Month ∈R d×d Weight coefficients for linear transformation, obtained by random initialization and continuously optimized in iteration, W Month The embedded representation of the user can be correspondingly transformed to be used for carrying out personalized processing on the month representation, and the personalized embedded representation of the personalized week, day and hour granularity cycle information can be obtained in the same way
For granularity s, the personalization of m periods adjacent to the time t to the user u is represented as
The following relationship exists:
wherein the content of the first and second substances,
wherein, the first and the second end of the pipe are connected with each other,
for the embedded representation of the user u, at the time t, granularity s is adjacent to m periods when m<0, representing the m-th time interval before the time interval of the time t; when m is>When 0, the m-th time interval after the time interval of the time t is represented; when s represents a month, t s Represents the month in which the time t is located, and f is 12; when s represents week, t s F represents the cycle of the time t and is 7; when s represents day, t s F represents the day of the moment t and is 31; when s represents hour, t s F is 24, but different from the calculation of month, week and day, for hours, since one day starts from 0, when s represents hours, Δ represents hours s (t,m)＝(t s + m) mod f, mod is the modulus operator.
For granularity s, an attention mechanism is adopted to adaptively combine the time period of the time t with the representation of the adjacent m time period, and the personalized representation of the time period of the granularity s for the user u is as follows:
wherein, in correspondence with the particle size s,
for the personalized representation of the time period t for the user u, corresponding to the four granularities of month, week, day and hour, the personalized representation for the user u is respectively obtained as follows:
personalized representation of the m time periods adjacent to the time t for the user u: attention coefficient α t,m To represent
And
the similarity of (c).
Preferably, the attention coefficient α t,m The calculation formula is as follows:
wherein r is s Representing the window size of the granularity s adjacent the period, r s The method comprises the following steps: s represents month, r s =2; s represents week, r s =1; s represents day, r s =6,s represents hour, r s =5,exp denotes an exponential function with a natural constant e as base, n represents a value within a window of adjacent time periods of the granularity s.
In another preferred embodiment of the invention, the time-personalized representation T of the time T for the user u is calculated using an attention mechanism u (t)∈R d×1 Comprises the following steps:
wherein the content of the first and second substances,
the period representing the user u versus the granularity s represents the attention coefficient, W u ∈R d×d Transformation of representation to user representation, e u ∈R d An embedded representation for user u;
T u (t)∈R d×1 as a final time-personalized representation for the user u at time t, it contains a representation of personalized multi-granularity period information.
The time interval of the user sign-in reflects the degree of association between two sign-ins, plays an important role in expressing time effect and revealing sequence mode, reasonably utilizes time information to better mine the movement rule of the user, and a timestamp reflects the absolute time of the user for visiting an interest point, and the timestamp information needs to be processed to express the time interval. t → R d The time interval can then be expressed as a dot product of the corresponding time-coded representation:
ψ(t 1 -t 2 )＝K(t 1 ,t 2 ):＝<Φ(t 1 ),Φ(t 2 )>(1)
wherein, K is a time kernel function,<,>representing a dot product operation. Psi (t) 1 -t 2 ) The relationship between two timestamps is measured. Because the time coding function directly codes and represents the time stamp, the time coding function can be popularized to any time stamp, and can obtain the representation of any time interval:
based on the Bochner theory and Monte Carlo integration, Φ (t) in equation (1) can be determined by Φ in equation (2) d (implementation. Where ω = [ ω ]) 1 ,...,ω d ] T Parameters to be learned for the model. So the code at time t is represented as phi d (t, and the time interval is obtained by the vector inner product shown in equation (1).
In another preferred embodiment of the present invention, with the time personalized representation of the user u, the time coded representation, and the embedded representation of each check-in point of interest of the user u, the embedded representation of the check-in sequence of the user u is calculated as:
wherein N is the check-in times of the user u,
representation of the kth check-in for user u:
wherein the content of the first and second substances,
for the user u at t k Interest point signed in at moment
Embedded representation of, T u (t k ) Is said t k The time personalized expression of the time integrates the periodic information of the four granularities, phi (t) k ) Is said t k Time of day check-in time coded representation, e pk Is said t k Coded representation of the location of the time check-in, e u ∈R d Is an embedded representation of user u.
In another preferred embodiment of the invention, in order to capture the complex association between geographic distance and meta-embedding, a differentiable automatic discretization module is designed:
γ ij ＝ReLu(W 1 ΔD ij )
wherein V is the number of element insertions, Δ D ij The geographic distance between the ith check-in and the jth check-in,
W 1 ∈R V×1 、W 2 ∈R V×V parameters to be learned for the model. Alpha controls the ratio of the residual connection. Gamma ray ij Representing the results obtained by a layer of neural network conversion,
representing a geographical distance Δ D ij The relevance of the meta-embedding H to,
wherein the content of the first and second substances,
representing a geographical distance Δ D ij Embedding with the v-th element v The correlation of (c).
Thus, the geographic distance Δ D between the ith check-in and the jth check-in point of interest ij Embedded representation of
Comprises the following steps:
obtaining the geographic distance delta D between the ith check-in and the jth check-in interest point in a weighted average mode ij Embedded representation of
The embedded representation of the geographic distance Δ D between user u's check-ins is denoted E ΔD ∈R N＝N×d Then embedded representation of the spatial relationship E Δ ∈R N×N Comprises the following steps:
E Δ ＝E ΔD W D
wherein the mapping matrix W D ∈R d×1 For pair E ΔD And carrying out dimensional transformation.
The weighted averaging approach makes the relevant meta-embedding more favorable for providing rich information, while irrelevant meta-embedding is largely ignored.
In another preferred embodiment of the invention, the embedded representation in conjunction with the user check-in sequence is S u The embedded expression of the user check-in sequence local information subjected to causal convolution enhancement is S' u Embedded representation of said spatial relationship E Δ Calculating a new sign-in sequence representation Z of the user u u ：
Wherein the content of the first and second substances,
mapping matrix W Q 、W K 、W V ∈R d×d ，
And (3) representing a Hadamard product, and introducing a lower triangular mask matrix M with an element of 1 in order to ensure causality when calculating self-attribute, namely, not utilizing information of T +1 and later time when predicting an interest point at the time T, wherein T represents transposition operation.
In another preferred embodiment of the present invention, a Feed-Forward neural Network (FFN) is used to represent Z for the new check-in sequence of the user u u Introducing nonlinearity:
wherein the content of the first and second substances,
the weight parameters of the neural network are set,
for the threshold parameter of the neural network, f1, f2 denote the first layer network and the second layer network of the FFN.
In order to accelerate the training process and prevent overfitting, gradient disappearance and other phenomena in the training process, layerNorm, dropout, residual error connection and other skills are introduced, and the optimized representation of the check-in sequence of the user u
Comprises the following steps:
wherein N is the check-in times of the user u,
at a time t k And (4) the optimized sign-in representation of a user u, wherein k is the kth sign-in and t k The time of the kth check-in.
Preferably, a potential factor model is used for calculating the t of the user u k Point of interest for time
Preference of
Wherein the content of the first and second substances,
as points of interest
Is shown in。
At a time t k-1 Optimized sign-in representation for user u, t k-1 The time of the (k-1) th check-in.
And performing model training, predicting the next interest point, outputting the interest points according to the sequence of the preference values from high to low after the preference of the user u to each interest point at the time t is obtained, and recommending the next interest point to be accessed by the user or predicting each interest point.
To learn the parameters of the model, the BPR loss function is employed:
wherein, the training set sample D = (u, l) i ,l j ,t)|(u,l i ,t)∈I u ，l j ∈I\I u Wherein the positive sample (u, l) i T) represents the point of interest l that the user u checked in at the time t i And negative examples (u, l) j And t) randomly sampling interest points which are not checked in by the user u.
Including all learnable parameters, penalty terms, of the model
To prevent overfitting of the model. Sigma is sigmoid function.
In the following, for the stir san model established in the embodiment of the present invention, two evaluation indexes, namely recall @ K and Mean Reciprocal Rank (MRR), of top K are used to evaluate the performance of the model.
(1) Recall rate recalling @ K
Recall refers to the ratio of positive samples predicted by the model to all positive samples. The next interest point recommendation refers to the ratio of the labels at top K after the predicted values of the models for the interest points are arranged in a descending order. The calculation of Recall @ K is:
wherein K ∈ {1,5,10,20},
and S label Respectively representing the topK points of interest recommended to the user by the model and the points of interest actually visited by the user, namely the tags. Obviously, in the next interest point recommendation task, only one interest point is accessed by the user at the next time step, i.e. | S label |＝1。
(2)MRR
MRR refers to the average of the inverses of the ranking of the positive samples in all samples, reflecting the ranking capability of the model as a whole. The MRR is calculated as:
wherein, rank u The ranking of the label representing user u in the model recommendation list. The higher the label rank in the recommendation list, the higher the MRR value, and the better the model.
According to the embodiment of the invention, the STIRSAN model is evaluated in two real world data sets of Foursquare-NYC and Foursquare-TKY. The data set is pre-processed, removing inactive users who have checked in less than 5 times, and points of interest that have been checked in less than 5 times. Data set statistics the statistics are as follows:
table 1 data set statistics
| dataset | #User | #POI | #Check-in |
| Foursquare-NYC | 1083 | 9989 | 179468 |
| Foursquare-TKY | 2293 | 15177 | 494807 |
The data set is divided into a training set and a testing set for each user using only their most recent N visits in the experiment, wherein each user's [1, N-1] sign-ins are used as the training set and [2, N ] sign-ins are used as the testing set, and wherein the N-th sign-in interest point is predicted using the [2, N-1] sign-ins as the input sequence.
Results of the experiment
The invention mainly utilizes the space-time information to represent to recommend the next interest point, and the experimental part mainly comprises two experiments, (1) the comparative experiment of the embodiment model and 5 baseline models. (2) And designing an ablation experiment, and verifying the effectiveness of each module of the model of the embodiment of the invention. (3) Design experiments verify the robustness and interpretability of the model.
In a comparison test of the recommended performance of the next point of interest, we compared the stirlsan model established in the example of the present invention with the following model:
(1) TMCA model: with an LSTM-based encoder-decoder architecture, two mechanisms of attention are proposed to adaptively select the relevant historical check-in and context factors.
(2) DeepMove model: it is proposed to use an attention mechanism to obtain relevant information from historical check-ins to model long-term user preferences, and RNNs to model short-term preferences.
(3) LSTPM model: modeling long-term preferences using temporal and spatial correlations of current and historical trajectories, and capturing geographical links between non-consecutive check-ins using geo-scaled RNN when modeling short-term preferences.
(4) STAN model: and (3) learning representations of different time intervals and geographic distances by utilizing a linear interpolation method by considering the time intervals and the geographic distances between the discontinuous sign-ins based on a next interest point recommendation model of a Transformer.
(5) TiSASRec model: based on a Transformer sequence recommendation model, time intervals of different users are subjected to personalized processing to obtain relative time intervals, and the influence of different time intervals is considered when self-attention is calculated.
The super parameters of the STIRSAN model are set as follows: r is a radical of hydrogen M ＝2，r W ＝1，r D ＝6，r H =5, α =0.1, the number of meta-embedded representations K is set to 30, and the convolution kernel size of the causal convolution is set to 5. The input sequence length is 100, the dimension of the embedded representation is 100, the dimension of the hidden layer is also set to 100, the number of CAC-Transfomer layers is 2, and lambda is 5e-5. The parameters of each model are shown in Table 2.
TABLE 2 comparison of the model parameters
| Models | TMCA | DeepMove | LSTPM | STAN | TiSASRec | STIRSAN |
| Amount of ginseng | 2,420,753 | 4,207,563 | 3,257,863 | 1,795,000 | 1,172,400 | 1,268,492 |
In both the NYC and TKY data sets, the experimental results of stirlsan and baseline model in the next point of interest recommendation task are shown in tables 3 and 4, with the data in the table being the highest experimental results.
TABLE 3 comparison of recommended Performance on NYC dataset
Table 4 recommended Performance comparison experiments on TKY dataset
As can be seen from tables 3 and 4, the performance of the stirlsan model according to the embodiment of the present invention is significantly higher than that of the baseline model. In the NYC dataset, STIRSAN was 1.7%, 4.07%, 3.32%, 0.56% and 3.14% above baseline model at 5 evaluation indices Recall @1, recall @5, recall @10, recall @20 and MRR, respectively; in the TKY dataset, STIRLSAN was higher than baseline models by 2.48%, 2.23% and 1.11% in Recall @5, recall @10 and MRR indices, respectively, and similar to the optimal baseline model in Recall @1 and Recall @20, with a difference of 0.37% and 0.92%, respectively.
In combination with table 2, the parameters of the stirlsan model are significantly lower than those of TMCA, deepMove, LSTPM and STAN, and are equivalent to those of the TiSASRec model. The STIRSAN obtains remarkable superior performance by using less parameter quantity, which shows that the STIRSAN model performance is improved by mining the rule inside the check-in data, but not by fitting excessive parameters.
In order to verify the effectiveness of each module, the following variants of the stirlsan model were set up for the ablation experiments:
(1) STIRSAN w/o CAC: variants of the stirlsan model, i.e. removing the causal convolution that enhances local sequence information;
(2) STIRLSAN w/o PMGP: a variant of the stirlsan model, namely, removing the personalized multi-granularity period representation;
(3) STIRSAN w/o TIE: a variant of the STIRSAN model, i.e., to remove the effect of the time interval between check-ins;
(4) STIRSAN w/o DIE: a variant of the STIRSAN model, namely, removing the embedded representation of the geographic distance between check-ins.
The results of the ablation experiments are shown in tables 5 and 6.
TABLE 5 results of the ablation experiments on the NYC dataset
| Evaluation index | R@1 | R@5 | R@10 | R@20 | MRR |
| STIRSAN | 0.1837 | 0.4414 | 0.5272 | 0.5753 | 0.2954 |
| STIRSAN w/o PMGP | 0.1801 | 0.4201 | 0.5115 | 0.5734 | 0.2899 |
| STIRSAN w/o TIE | 0.1810 | 0.4146 | 0.4940 | 0.5457 | 0.2881 |
| STIRSAN w/o DIE | 0.1837 | 0.4183 | 0.5032 | 0.5688 | 0.2906 |
| STIRSAN w/o CAC | 0.1625 | 0.4090 | 0.4903 | 0.5531 | 0.2722 |
TABLE 6 results of the ablation experiments on TKY data set
The results of the experiments on the variant models of STIRLSAN are shown in tables 5 and 6. It can be seen that the data sets of NYC and TKY show a more consistent trend, that is, modules such as causal convolution, multi-granularity cycle information representation learning, and representation of time interval and geographic distance all contribute to the improvement of the stirlsan model performance.
The enhancement of model performance is maximized by carrying out local sequence enhancement on the Transformer by using causal convolution, and the enhancement is respectively increased by 2.12 percent and 2.22 percent for the NYC data sets Recall @1 and MRR; the TKY data set is improved by 2.18% and 2.1% respectively. Combining tables 3, 4, it can be seen that both Recall @10 and Recall @20 of STIRSANw/oCAC without causal convolution in both datasets are higher than LSTM-based DeepMove and LSTPM, but Recall @1, recall @5 and MRR indices are lower than LSTM-based comparative model. In addition to the stirlsan proposed herein, the same thing is shown for the transform-based TiSASRec model type, i.e. recall @10 and recall @20 are significantly higher than the LSTM-based DeepMove, LSTPM, etc. comparative models, but for the better-described recommendations recall @1 and recall @5, the transform-based model is inferior to the LSTM-based model. This indicates that Transformer excels in capturing long-term dependence and is insensitive to local information. For the next interest point recommendation task, the user's recent visit, i.e. short-term preference, is crucial for the next interest point prediction. The Transformer based model is therefore inferior to the LSTM based model in terms of accurate recommendations. Therefore, the STIRLAN combines the causal convolution and the Transformer to strengthen local information in the check-in sequence, so as to further mine the short-term preference of the user, and the experimental result also proves that the Transformer enhanced by the causal convolution greatly improves the accuracy of recommendation.
Next, to verify the robustness of the model, embodiments of the invention compare SIRSAN to the baseline model in different dimensions and sequence lengths, respectively. And visualizing the periodic representation and spatial relationships learned by the stirlsan model to demonstrate model interpretability.
(1) Influence of dimensionality on the results of the experiment
6.a-6.b visually illustrate the trend of Recall @10 and Recall @20 of each model along with dimensional changes on the NYC dataset; fig. 7 shows the performance of the stirlsan model in the TKY dataset in different vector dimensions. Overall, the results of stirlsan in both data sets are relatively smooth with dimensional changes, and in most cases, are higher than the baseline model, which indicates that the stirlsan model is relatively robust to vector dimensional changes.
(2) Effect of sequence Length on the results of the experiment
Fig. 8 visually shows the influence of the change of the sequence length in the TKY data set on the stirlsan model, and as the sequence length decreases, the performance of the stirlsan decreases, but the change is more gradual, so that the stirlsan model is more robust to the sequence length.
(3) Visualization of periodic relationships of various granularities
Fig. 9.a to 9.d respectively show thermodynamic diagrams of cosine similarity expressed by each granularity cycle learned by the stirlsan model, and it can be observed from the diagrams that the expressions of adjacent periods in each cycle are relatively similar. Taking fig. 8.D as an example, it can be clearly seen that 24 hours are roughly divided into 4-14 hours and 15-3 hours, wherein the 24 hours can be further divided into 8-11 hours, 11-14 hours, 14-18 hours, 19-23 hours, etc. This is also consistent with the division of time in daily life.
(4) Spatial relationship E Δ Is visualized by
FIGS. 10.a-10.d show the thermodynamic diagram of the geographical distance between check-ins of four randomly chosen users and the spatial relationship E learned by the STIRSAN model Δ In a thermodynamic diagram of (1). The spatial relationship E between the geographical distance thermodynamic diagram and the model can be obviously seen Δ The textures of the thermodynamic diagram are close, while the values show the opposite relationship. I.e. the greater the geographical distance in fig. 9.a, 9.b, the corresponding E Δ The smaller the value; the greater the geographic distanceSmall, E corresponding to it Δ The larger. This indicates that the model learns a larger association between two check-ins for smaller geographic distances and vice versa.
In summary, the next interest point recommendation model based on the spatio-temporal information representation provided by the embodiment of the present invention is superior to other comparative experiments in recommendation performance, so that the effectiveness of the embodiment of the present invention is demonstrated, and the model can be applied to the recommendation task of the next interest point; in addition, the effectiveness of the multi-granularity periodic representation, the representation of the time interval between check-ins, the geographic distance and the enhanced causal convolution of the local sequence information proposed by the embodiment of the invention is verified through ablation experiments.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the prior art according to the concepts of the present invention should be within the scope of protection determined by the claims.
Claims (9)
1. A next point of interest recommendation system based on spatio-temporal information representation, comprising:
the time individuation module is used for individualizing cycle information of four granularities of month, week, day and hour of the check-in timestamp, for the cycle representation of any one of the four granularities, the time interval of the check-in time t is combined with the representation of the adjacent time interval in a self-adaptive manner by adopting an attention mechanism, and the individuation representation of the time interval of the granularity to the user u is obtained
By using the said
Calculating to obtain a time personalized representation T of the moment T to the user u by adopting an attention mechanism u (t)；
The time coding module is used for mapping the sign-in timestamp to a vector space by using a time coding function to obtain a time coding representation phi (t);
user sign-in sequence module, with the T u (t), the embedded expression of phi (t) and the sign-in interest point of the user u is used as input, and the embedded expression S of the sign-in sequence of the user is calculated u ；
And the causal convolution enhancement module is used for taking the embedded expression of the check-in interest point of the user u as an input, combining the causal convolution and the Transformer to enhance the local information of the user check-in sequence to obtain an embedded expression S 'after the causal convolution enhancement' u ；
New user' S check-in sequence module, and S u S 'to' u And embedded representation of spatial relationships E Δ Calculating a new sign-in sequence representation Z for said user u ；
And the output module is used for calculating the preference of the user u on the interest point at the moment t by using the new sign-in sequence representation of the user u, predicting and recommending the next interest point.
2. The spatio-temporal information representation-based next point-of-interest recommendation method of claim 1, wherein the granularity is a personalized representation of the time period for user u
Comprises the following steps:
wherein S ∈ S, S = { Month, dayOfWeek, date, hour } represents a set of four kinds of granularity period information, S represents an element in the set, i.e., one of the four granularities, R represents a real number domain, d is a dimension of embedded representation, for granularity S,
for the granularity adjacent to m periodsPersonalized representation of user u: when m is<0, representing the m-th granularity period before the period of the time t; when m is>When 0, it represents the m-th granularity period after the period of the time t; attention coefficient α t,m To represent
And
similarity of (2), r s Representing the granularity s the window size of the adjacent period.
3. The spatio-temporal information representation-based next point-of-interest recommendation method of claim 2, wherein the granular m-period-adjacent personalized representation for user u
Comprises the following steps:
wherein the content of the first and second substances,
wherein the content of the first and second substances,
for user u, the embedded representation of granularity s at said time t,
for the embedded representation of the granularity s adjacent m periods at said time instant t, Δ s (t, m) represents the m-th granularity period adjacent at said time instant t; when s represents a month, t s Indicates that the time t isMonth of (d), f is 12; when s represents week, t s F represents the cycle of the time t and is 7; when s represents day, t s F represents the day of the time t and is 31; when s represents hour, t s F is 24, but different from the calculation of month, week and day, for hours, since one day starts from 0, when s represents hours, Δ represents hours s (t,m)＝(t s +m)mod f。
4. The spatio-temporal information representation-based next point of interest recommendation method of claim 3, wherein the attention coefficient α is t,m The calculation formula is as follows:
wherein r is s Representing the granularity s the window size of the adjacent period.
5. The spatio-temporal information representation-based next point-of-interest recommendation method according to claim 1, characterized in that the time T represents T for the temporal personalization of the user u u (t)∈R d×1 Comprises the following steps:
wherein S ∈ S, S = { Month, dayOfWeek, date, hour denotes a set of four granularity period information, S denotes an element in the set, i.e., one of the four granularities,
the period representing the user u versus the granularity s represents the attention coefficient,
representing the personality of the user u in the time period of the granularity sChemical representation of, W u ∈R d×d Transformation of representation into user representation, e u ∈R d Is an embedded representation of user u.
6. The spatio-temporal information representation-based next point-of-interest recommendation method of claim 1, characterized in that the user u signs in an embedded representation S of a sequence u Comprises the following steps:
wherein N is the check-in times of the user u,
for user u at t k Representation of a time check-in:
wherein the content of the first and second substances,
for the user u at t k Interest point signed in at moment
Embedded representation of, T u (t k ) Is said t k The time personalized expression of the time integrates the periodic information of the four granularities, phi (t) k ) Is said t k Time of day check-in time coded representation, e pk Is said t k Coded representation of the location of the time check-in, e u ∈R d Is an embedded representation of user u.
7. The spatio-temporal information representation-based next point-of-interest recommendation method of claim 1, wherein the embedded representation of spatial relationship E Δ ∈R N×N Comprises the following steps:
E Δ ＝E ΔD W D
wherein E is ΔD ∈R N×N×d Mapping matrix W for an embedded representation of geographic distance between user u's check-ins D ∈R d×1 For pair E ΔD And D, carrying out dimension transformation, wherein N is the sign-in times of the user u.
8. The spatio-temporal information representation-based next point-of-interest recommendation method of claim 1, wherein the new check-in sequence of user u represents Z u ：
Wherein the content of the first and second substances,
mapping matrix W Q 、W K 、W V ∈R d×d ，
Representing the hadamard product, M being the lower triangular mask matrix with element 1, T representing the transposition operation.
9. The spatio-temporal information representation-based next point of interest recommendation method of any one of claims 1 or 8, characterized in that a feed-forward neural network (FFN) is used to introduce non-linearity for the model:
wherein the content of the first and second substances,
the weight parameters of the neural network are set,
f1 and f2 represent a first layer network and a second layer network of the FFN, wherein the threshold parameters of the neural network are represented by f1 and f 2;
optimized representation of the check-in sequence of the user u
Comprises the following steps:
wherein N is the check-in times of the user u,
is shown at time t k Optimized check-in representation for user u.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211552543.5A CN115827974A (en) | 2022-12-05 | 2022-12-05 | Next interest point recommendation system based on spatio-temporal information representation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211552543.5A CN115827974A (en) | 2022-12-05 | 2022-12-05 | Next interest point recommendation system based on spatio-temporal information representation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115827974A true CN115827974A (en) | 2023-03-21 |
Family
ID=85544119
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211552543.5A Pending CN115827974A (en) | 2022-12-05 | 2022-12-05 | Next interest point recommendation system based on spatio-temporal information representation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115827974A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117573986A (en) * | 2024-01-16 | 2024-02-20 | 广东工业大学 | Interest point recommendation method based on sequential and ground understanding coupling characterization |
| CN117633371A (en) * | 2024-01-25 | 2024-03-01 | 云南大学 | Recommendation method, device and readable storage medium based on multi-attention mechanism |
| CN117745349A (en) * | 2023-12-22 | 2024-03-22 | 广州市禾赢文化传播有限公司 | Personalized coupon pushing method and system based on user characteristics |
-
2022
- 2022-12-05 CN CN202211552543.5A patent/CN115827974A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117745349A (en) * | 2023-12-22 | 2024-03-22 | 广州市禾赢文化传播有限公司 | Personalized coupon pushing method and system based on user characteristics |
| CN117573986A (en) * | 2024-01-16 | 2024-02-20 | 广东工业大学 | Interest point recommendation method based on sequential and ground understanding coupling characterization |
| CN117573986B (en) * | 2024-01-16 | 2024-03-26 | 广东工业大学 | Interest point recommendation method based on sequential and ground understanding coupling characterization |
| CN117633371A (en) * | 2024-01-25 | 2024-03-01 | 云南大学 | Recommendation method, device and readable storage medium based on multi-attention mechanism |
| CN117633371B (en) * | 2024-01-25 | 2024-04-12 | 云南大学 | Recommendation method, device and readable storage medium based on multi-attention mechanism |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111400620B (en) | User trajectory position prediction method based on space-time embedded Self-orientation | |
| Yu et al. | A category-aware deep model for successive POI recommendation on sparse check-in data | |
| CN115827974A (en) | Next interest point recommendation system based on spatio-temporal information representation | |
| Niu et al. | An online-traffic-prediction based route finding mechanism for smart city | |
| CN109062962B (en) | Weather information fused gated cyclic neural network interest point recommendation method | |
| Liao et al. | Multi-context integrated deep neural network model for next location prediction | |
| Lin et al. | A spatial-temporal hybrid model for short-term traffic prediction | |
| Chen et al. | CEM: A convolutional embedding model for predicting next locations | |
| Zhou et al. | Improving human mobility identification with trajectory augmentation | |
| Ahmed et al. | Knowledge graph based trajectory outlier detection in sustainable smart cities | |
| CN114971748B (en) | Prediction data generation method, model training method, computer device, and storage medium | |
| Liu et al. | Stock prices prediction using deep learning models | |
| CN112053188B (en) | Internet advertisement recommendation method based on hybrid deep neural network model | |
| Singh et al. | A review of bus arrival time prediction using artificial intelligence | |
| CN115080868A (en) | Product pushing method, product pushing device, computer equipment, storage medium and program product | |
| Chu et al. | Predicting online user purchase behavior based on browsing history | |
| Yu et al. | Using information entropy and a multi-layer neural network with trajectory data to identify transportation modes | |
| Linardatos et al. | CO2 concentration forecasting in smart cities using a hybrid ARIMA–TFT model on multivariate time series IoT data | |
| Blanco-Justicia et al. | Generation of synthetic trajectory microdata from language models | |
| CN115952355A (en) | Next interest point recommendation system based on space-time power law attention | |
| Wang et al. | A Spatial‐Temporal Self‐Attention Network (STSAN) for Location Prediction | |
| CN115982477A (en) | Next interest point recommendation method based on spatio-temporal information representation | |
| Wu et al. | Mining trajectory patterns with point-of-interest and behavior-of-interest | |
| US20220343243A1 (en) | Systems and methods for resource allocation in ride-hailing platforms | |
| Eum et al. | Imputation of missing time-activity data with long-term gaps: A multi-scale residual CNN-LSTM network model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |