WO2022007526A1

WO2022007526A1 - Travel package recommendation method based on multi-view attention mechanism

Info

Publication number: WO2022007526A1
Application number: PCT/CN2021/095763
Authority: WO
Inventors: 曹杰; 朱桂祥; 申冬琴; 陈蕾; 梁伟超
Original assignee: 云境商务智能研究院南京有限公司; 南京理工大学
Priority date: 2020-07-07
Filing date: 2021-05-25
Publication date: 2022-01-13
Also published as: CN111915395A

Abstract

Disclosed are a travel package recommendation method based on a multi-view attention mechanism. A unified representation of a travel package is learned by means of deep learning technology, and a preference representation of a user is learned according to long-term and short-term click stream data of an online travel user to generate a recommendation. The method includes travel package encoding and user preference encoding. In a travel package encoding module, important words and views are selected from attributes of the travel package by using attention networks at a word level and a view level, so as to learn a unified travel package representation; and in a user preference encoding module, the dynamic evolution of a user preference is learned by means of a recurrent neural network (RNN) with an attention mechanism at a travel package level. Further provided is a gate structure fusion method, which is used for integrating long-term and short-term preferences of a user in order to learn a representation of the user. By means of the travel package recommendation method based on a multi-view attention mechanism in the present invention, useful information in a negative sample can be effectively mined, and the computing costs of model training can be reduced.

Description

A travel package recommendation method based on multi-view attention mechanism

technical field

The invention relates to the field of information science, and provides a travel package recommendation method based on a multi-view attention mechanism.

Background technique

In recent years, with the rapid development of the Internet, tourism is one of the most successful and beneficial industries in the field of e-commerce. In travel e-commerce, more and more tourists collect richer, comprehensive and personalized travel information through various online platforms for their travel itinerary planning. Therefore, a large amount of online travel data has been generated, and the e-commerce travel platform (Online Travel Agency, OTA) also urgently needs to realize the potential of business through some novel data analysis and mining technical means. A travel package usually refers to a set of integrated packages that contain a series of travel-related content, such as departure and destination cities, travel route text description, travel cost, travel days, transportation, accommodation, classification, etc. Numerous studies have been conducted on the personalized recommendation of travel packages. Based on consumption data provided by offline travel agencies, they found that travel packages have distinct domain characteristics compared to traditional products such as movies, books, and groceries. Most travel package recommendation models based on user clickstream (session) only utilize the user's current real-time clickstream data, while ignoring historical clickstream data. Another recommendation model is the user-based personalized recommendation model. In fact, the user's interactive behavior naturally forms a behavior sequence over time, and the user's long-term stable preference can be characterized by historical behavior data, short-term motivation and Requirements can be characterized by current behavioral data. Therefore, a relatively complete recommendation model needs to take into account the dynamic changes of user preferences, that is, not only the interaction information of the user's current session, but also the user's historical behavior information.

Deep learning is an important research direction in the field of machine learning. In recent years, breakthroughs have been made in the fields of pattern recognition, natural language processing, image recognition and autonomous driving. Up to now, deep learning has become a craze in artificial intelligence, bringing new opportunities and horizons to the research of recommender systems. Deep learning can characterize massive data related to users and items by learning a deep nonlinear network structure. On the other hand, deep learning can obtain a unified representation of data by performing automatic feature learning from multi-source heterogeneous data, thereby mapping different data to the same latent space. At present, in the recommendation models of most e-commerce platforms (such as Taobao, Tmall and Amazon), more recommendation models based on collaborative filtering are used. Such models can only consider the static interests of users, but cannot capture the user's Dynamic interest. Therefore, more and more scholars integrate deep learning into recommendation systems, and study how to integrate massive multi-source heterogeneous data to build a user model that is more in line with user preferences, so as to improve the performance of recommendation systems and user satisfaction. As one of the contributions of the present invention, the present invention will present a specific model for personalized recommendation of travel packages, which integrates multi-view learning and attention mechanism learning.

SUMMARY OF THE INVENTION

Purpose of the invention: In order to overcome the deficiencies in the prior art, the exemplary implementation of the present invention makes full use of the long-term and short-term clickstream data of users in online travel websites, and studies a travel package recommendation method based on a multi-view attention mechanism . Specifically, NATR is mainly composed of two core modules: Travel Package Encoder and User Preference Encoder. Specifically, the research contributions of this chapter are summarized as follows: (i) In order to accurately learn the travel package representation from the four attributes of the travel package, a multi-view attention mechanism method is proposed in the Travel Package Encoder module to learn Unified travel package characterization. Different from existing research methods that utilize special auxiliary information to improve recommendation accuracy, this module adopts word-level and view-level attention mechanisms to effectively select important and informative words and views, respectively; (ii) In order to capture the dynamic evolution of interest from the user's overall sequence behavior data, two bidirectional long-short-term memory neural networks (Bi-LSTM) with attention mechanism are designed in the User Preference Encoder module. Dynamically learn users' long-term and short-term preferences. Different from traditional session-based recommendation methods, the Bi-LSTM neural network with attention mechanism can select important travel packages from the user's continuous clickstream to accurately represent the user's preference; (iii) in order to better integrate the user Long-term preference and short-term preference, we further propose a gate-structure fusion network that fuses their relevant information instead of simply combining them. Unlike the weights of the attention network scalar, the gate structure vector has stronger representational power to control the importance of these two preferences.

Technical solutions:

A travel package recommendation method based on a multi-view attention mechanism, comprising the following steps:

Step 1): constructing a travel package coding module, i.e. the representation learning module of a unified travel package, obtaining the representation vector after the learning of the travel package coding module is r _j ;

Step 2): constructing a user long-term preference coding module and a user short-term preference coding module, that is, the user's long-term interest representation learning module and the user's short-term interest representation learning module, to obtain a short-term behavior representation vector s ^u and a long-term behavior representation vector l ^u ;

Step 3): design a gate structure fusion network module, use the short-term behavior representation vector s ^u and long-term behavior representation vector l ^u obtained in step 2 as the input of the gate structure fusion network module, and obtain the preference representation vector O _{u of the} user u;

Step 4): Calculate the recommendation score through the NATR model, that is, the Neural Attentive Travel package Recommendation model: z _k =O _u ^T r _j , use the Adam optimizer to minimize the loss function, and adjust the parameters in the NATR model to the optimal configuration .

Further, in step 1), the travel bag coding module carries out representation learning according to four attributes of the travel bag, and the four attributes include the title, destination, travel area and travel type of the travel bag, and this module adopts word-level and The view-level attention mechanism effectively selects words and views respectively. Given a travel package x _j , x _j =<Title,Destination,Categories>, the title i.e. Title includes country/city, attractions, hotel transportation and travel days, etc. , Destination is an identifier consisting of one or a few words, usually represented by a city or country name, and Category is a category including Travel Region and Travel Type. After learning from the travel package coding module The characterization vector of is r _j .

Further, in step 2), the user's long-term and short-term preference coding modules respectively design two bidirectional long-short-term memory neural networks (Bi-directional Long Short-term Memory, Bi-LSTM) with an attention mechanism. and the current clickstream to dynamically learn the user’s long-term and short-term preferences, the Bi-LSTM neural network with attention mechanism can select important travel packages from the user’s continuous clickstream to accurately represent the user’s preference, and the user’s long-term and short-term preference encoding module The representation vectors of long-term and short-term preferences of users are obtained by the sum of the travel package context representation vectors with attention weights. Given a target user u, let its short-term behavior and long-term behavior be denoted as S ^u and ^Lu , respectively, then short and long term behavior of the user characterization vectors and are denoted as s ^u l ^u.

Further, step 3) is specifically, let the user preference query vector be q _u , take the short-term preference vector s ^u and the long-term preference vector ^lu as input, and the gate structure vector F _{u is} used to control the contribution of long-term and short-term preference:

F _u =sigmod(W _q q _u +W _s s _u +W _l l _u +b _u ),

Wherein W _q, W _s, W _l and B _u are parameters of the neural network projection learning model Q _u represent, s ^{^u,} l ^u weight matrix, and a bias vector, preference characterizing the user of the final output of u The vector O _u can be calculated by:

O _u =(1-F _u )⊙s _u +F _u ⊙l _u ,

where ⊙ is the vector inner product symbol.

Further, step 4) is specifically, in the NATR model training stage, the positive label is the next real purchased travel package

while negative labels are removed from the travel pack set X

The set of travel packages formed by log-uniform sampling, after obtaining the user preference vector O _u and the characterization vector r _j of the travel package, the travel package z in the candidate travel package z={z ₁ ,z ₂ ,...,z _{| The} recommended score z _k can be calculated, and the recommended score is calculated as follows:

z _k =O _u ^T r _j ,

κ is the subset containing positive and negative labels sampled from X, ︳︳ represents the norm of the set, and X represents the set of travel packages;

The predicted probability of the travel package in κ is

Application softmax function to obtain an output NATR model, i.e., ^{y - = softmax (z),} y - is the probability that all travel at a session S ^u respectively purchased,

is the probability that the |k|th travel package is purchased.

For each user, the loss function based on the purchase probability defined by cross entropy and the real situation can be calculated by the following method:

where y _j is the probability distribution that the travel package x _j is actually purchased, specifically, if x _j is a positive label, then y _j = 1, otherwise y _j = 0

Beneficial effects: The present invention constructs a novel multi-view attention model Neural Attentive Travel package Recommendation (NATR) for personalized travel package recommendation, which can perform representation learning on complex travel package description information and dynamically capture the dynamic changes of user interests , which can effectively integrate users’ long-term and short-term interest preferences. In the user interest coding module, the dynamic evolution of user preferences is learned through a recurrent neural network (RNN) with an attention mechanism at the tourist package level, instead of using traditional methods in the matching stage. Collaborative filtering and matrix factorization models. The gate structure fusion network better integrates the user's long-term preferences and short-term preferences. This gate structure network fuses their related information instead of simply combining them. Different from the weight of the attention network scalar, the gate structure vector has more The importance of strong representational ability to control both preferences. The present invention minimizes the loss function with the help of the Adam optimizer

Therefore, the parameters in the NATR model are adjusted to the optimal configuration. Compared with the existing recommendation method, the method can effectively mine the useful information in the negative samples, and further reduce the computational cost of model training. Therefore, the method of the present invention NATR models can be more easily trained on large amounts of e-commerce travel clickstream data.

Description of drawings

Figure 1 is a sample display of the travel package;

Figure 2 is a travel package recommendation that integrates long-term and short-term behaviors of users;

Figure 3 is the click stream division of long-term and short-term behaviors;

Figure 4 is the framework of the NATR travel package recommendation model.

detailed description

The present invention will be further described below in conjunction with the accompanying drawings:

As shown in FIG. 1 , according to an exemplary embodiment of the present invention, a method for recommending travel packages based on a multi-view attention mechanism is provided, including the following steps:

Step 2): constructing a user's long-term preference coding module and a user's short-term preference coding module, namely the user's long-term and short-term interest representation learning module and the user's short-term interest representation learning module;

Step 3): design a gate structure fusion network to obtain the preference representation vector O _{u of} user u;

Step 4): Calculate the recommendation score through the NATR (Neural Attentive Travel package Recommendation) model: z _k =O _u ^T r _j , use the Adam optimizer to minimize the loss function, and adjust the parameters in the NATR model to the optimal configuration , the parameters are the projection parameters W _q , W _s , W _l and _bu in the gate structure fusion network.

In an exemplary embodiment, the travel package x _j generally refers to a set containing some necessary elements related to travel, which can be represented by a tuple x _j =<Title,Destination,Categories>, and the title (Title) is a travel A brief description of attributes, including country/city, attractions, hotel transportation, and travel days, etc. A destination is an identifier consisting of one or a few words, usually represented by a city or country name. Category (Categories) is a description of the type of travel package from the perspective of travel region (Travel Region) and travel type (Travel Type). Specifically, the first category is to divide travel packages according to the travel area, so each travel package is divided into local/surrounding tours, domestic short-distance/long-distance tours and overseas short-distance/long-distance tours, and the other type is based on the type of travel. Travel packages are divided, including Tuniu special tours, group tours, self-driving tours, self-guided tours, company package tours, local tours, etc.

The package coding module treats the title, destination and category of tours as different representational views of the tour package, and aims to learn a unified representation of the tour package from these views. Since different types of travel package information should be handled differently, the importance of different words in the same travel package title is different. The present invention designs a word-level and view-level attention network to select important words and views. Representation for learning travel packages.

The input of the title encoding module is the sequence {w ₁ ,w ₂ ,...,w _I } after word segmentation of the travel title, where I is the number of words contained in the title. The present invention first uses a Google open source word2ve project to convert titles into embedded vectors

Secondly, the embedded vector of the title is used as the input of the Bi-LSTM model, and the output of the network can be obtained respectively through the forward and reverse LSTM, namely: the vector of the final hidden layer of the title

Finally, embed the identifier of user u into a representation vector as the user's preference vector q _u , let

Represents the attention weight of the i-th word in the title of the travel package, which is calculated as follows:

Among them, W _t and b _t are the parameters learned in the neural network, representing the weight matrix and bias vector in the title encoding module, respectively, and _{the title representation of the final travel package x j} is the sum of all word context representation vectors with weights :

The input to the destination encoding module is the identifier of the travel destination. The present invention first converts discrete destination identifiers into low-dimensional representation vectors e ^c ; then uses a Multi-layer Perceptron (MLP) model to learn the destination representation vectors of _{travel package x j:}

where W _c and b _c are the parameters learned in the neural network and represent the weight matrix and bias vector ^{of e c in the destination encoding module, respectively.}

The input of the category coding module is the travel region (Travel Region, TR) and the travel type (Travel Type, TT) of the travel package. The present invention firstly converts the discrete identifiers of travel region (TR) and travel type (TT) into low-dimensional dense representation vectors, denoted as et ^tr and ^{et tt} ; and then also uses an MLP model to learn the category travel regions ( TR) and category travel type (TT) representation vector

and

The present invention uses a multi-view attention (View-level Attention) module to model the information content of different types of travel packages, so as to obtain the representation vector of the travel packages. Let α _t , α _c , α _tr and α _tt represent the attention weights of the title, destination, travel area and travel type of the travel package x _{j, respectively.} Attention to the right to view the title weight α _t as an example, which is calculated as follows:

Among them, W _v and b _v are the parameters learned in the neural network, representing

The matrix weights and bias vectors of .

The unified representation vector of the final travel package x _{j is} the sum of representation vectors on each view with attention weights, calculated as follows:

In an exemplary embodiment, the user preferences encoding module long and short and long-term behavioral intended data S L ^u ^u user data to learn the short and long term behavior characterization vector of user u. Taking the short-term interest coding module as an example, the present invention first obtains the short-term behavior representation vector through the travel package coding module.

Second, use the Bi-LSTM model to learn its short-term interest motivation to obtain the final output state

is the representation vector of the travel package x _j. In order to model different information amounts of the same travel package for different users, the present invention also designs a personalized attention network to learn the representation of travel packages clicked or purchased by the same user. The present invention records the weight of the jth travel package clicked or purchased by the user u as

This weight can be measured by the interaction importance between the user preference and the travel package representation vector:

where W _p and b _p are the parameters of the projection,

where m=1 to M, representing the summation function. The representation vector of the short-term preference of the end user can be obtained by summing the representation vector of the travel package context with the attention weight:

where |S ^u | is the number of travel packages included in the short-term behavior data S ^u.

Similarly, the present invention also uses Bi-LSTM personalized attention and learning mechanism to the end user preferences characterized by long-l _u, l _u long term preference characterization and calculation methods as short, but different data input.

In an exemplary embodiment, in order to integrate the long-term and short-term preferences of user u, the present invention designs a gate-structure fusion network to measure the importance of the long-term and short-term preference vectors, and integrate these information accordingly. Taking the short-term preference vector s _u and the long-term preference vector l _u as input, the gate structure vector F _{u is} used to control the contribution of the long-term and short-term preference:

o _u =(1-F _u )⊙s _u +F _u ⊙l _u

where ⊙ is the vector inner product symbol.

In an exemplary embodiment, the travel package representation vector is r _j and the user u preference representation vector O _u described in claim 4, and the goal of the NATR model is to predict a package containing top-K items _{according to O u} and r _j The recommendation candidate set of the travel package, the recommendation score is calculated as follows:

z _k =O _u ^T r _j .

In an exemplary embodiment, the NATR model during training, the positive label is the next real purchased travel package

while negative labels are removed from the travel pack set X

A collection of travel packages formed by log-uniform sampling. After obtaining the user preference vector characterizing vector r O _u travel packages and _j, z candidate travel packages travel package _{_{= {z 1, z 2,}} ..., z | k |} calculated recommended value _K obtained z out, where κ is the subset sampled from X containing positive and negative labels,

Κ is the predicted probability of the travel bag of the present invention is then applied to obtain an output NATR softmax function model, i.e., y ^- = softmax (z), where y ^- is the probability of travel at a session S ^u being purchased.

where y _j is the probability distribution that the travel package x _j is actually purchased. Specifically, if x _j is a positive label, then y _j =1, otherwise y _j =0. Here, the present invention minimizes the loss function with the help of the Adam optimizer

Thereby, the parameters in the NATR model are adjusted to the optimal configuration.

The e-commerce travel data set used in the present invention is provided by Tuniu, one of the largest online travel platforms in China. Tuniu can provide more than one million travel products and has provided online travel booking services for 15 million customers. The Tuniu dataset mainly consists of clickstream data from page browsing on server logs, which is actually a common setting for studying online shopping behavior analysis.

Figure 1 shows three classic travel package examples. A complete travel package usually refers to a series of necessary travel-related elements (such as transportation, attractions, accommodation, features and travel days, etc.) to form different sets, usually consisting of OTA conducts unified customization based on factors such as resource integration, market demand, and cost control. For example, it can be seen from Figure 1 that each travel package is mainly composed of four attributes: title, destination, travel area and travel type. Specifically, the textual information of the title attribute is much longer and more detailed than other attributes, while the destination and category (i.e. travel area and travel type) attributes usually consist of several simple identifiers.

Figure 2 shows the clickstream segmentation of long- and short-term actions. Here, the session is automatically identified by the SessionID field in the click stream provided by Tuniu. Let U and X denote the sets of users and travel packages, respectively. For any user u∈U, the present invention can obtain its interaction sequence according to the order of time

in

represents the nth session of user u, and N=|S ^u | is the number of sessions held by user u. each session of user u

It can be expressed as

in

is a session

the number of packages included,

represents the jth travel package operated by user u,

Represents the type of action (eg click or buy). As shown in FIG. 2, the present invention selects the most recent session before the user purchases

As the short-term behavior of user u, denoted as ^Su , and the remaining session as the long-term behavior of user u, denoted as

Figure 3 shows a classic travel package recommendation scenario of OTA. First, the interests of online users change dynamically over time. Intuitively, user behaviors naturally form clickstreams over time, and these historical and current clickstreams can dynamically reveal users’ long- and short-term preferences. Here, from the user's current click stream, the user may be interested in the travel package of Nanjing short-term tour. If the present invention adopts a session-based recommendation method that only uses the current click stream as input (as shown by the red box in Fig. 3), another series of popular travel packages related to Nanjing tour will be recommended. Conversely, from a long-term perspective, the user's historical clickstream information suggests that he may be interested in short-term travel packages in Shanghai. However, the existing researches all use content-based recommendation methods that rely on collaborative filtering and matrix factorization, which can only model users' static interests and cannot capture user dynamics from users' complete clickstream sequence data. changes in interest.

Furthermore, the long-term preferences contained in historical clickstream data of online users always influence users' current decisions. For example, if the present invention simply splices long-term and short-term clickstreams into the model of collaborative filtering and matrix factorization, other travel packages similar to Nanjing and Shanghai will be recommended (as shown in the green box in Figure 3). ). In fact, the user shown may plan to purchase a package that includes a series of city tours adjacent to the historical and current clickstream (shown in the blue box in Figure 5.1). However, most of the traditional methods are based on the interaction matrix of users and items in short sessions, and how to perfectly combine long-term preferences with short-term preferences remains to be explored.

Figure 4 shows a novel NATR model framework designed based on deep learning architecture. The NATR model can be used for personalized travel package recommendation. The model consists of two core components, namely, Travel Package Encoder and User Interest Encoder. Preference Encoder). Different from existing methods, in the Travel Package Encoder module, a unified travel package representation is learned by using word-level and view-level attention networks to select important words and views from the attributes of the travel package. Meanwhile, in the User Preference Encoder module, the dynamic evolution of user preferences is learned through a recurrent neural network (RNN) with an attention mechanism at the tourist bag level, instead of using traditional collaborative filtering and matrix factorization models in the matching stage. Furthermore, a gate structure fusion method is proposed to integrate users' long-term and short-term preferences to learn user representations.

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. It should be regarded as the protection scope of the present invention.

Claims

A kind of travel package recommendation method based on multi-view attention mechanism, is characterized in that, comprises the steps:

Step 1): constructing a travel package coding module, i.e. the representation learning module of a unified travel package, obtaining the representation vector after the learning of the travel package coding module is r j ;

Step 2): constructing a user long-term preference coding module and a user short-term preference coding module, that is, the user's long-term interest representation learning module and the user's short-term interest representation learning module, to obtain a short-term behavior representation vector s u and a long-term behavior representation vector l u ;

Step 3): design a gate structure fusion network module, use the short-term behavior representation vector s u and long-term behavior representation vector l u obtained in step 2 as the input of the gate structure fusion network module, and obtain the preference representation vector O u of the user u;

Step 4): Calculate the recommendation score through the NATR model, that is, the NeuralAttentive Travel package Recommendation model: z k =O u T r j , use the Adam optimizer to minimize the loss function, and adjust the parameters in the NATR model to the optimal configuration.
The method for recommending travel packages based on a multi-view attention mechanism according to claim 1, wherein in step 1), the travel package encoding module performs representation learning according to four attributes of the travel package, and the four attributes include travel The title, destination, travel area and travel type of the package. This module adopts the word-level and view-level attention mechanisms to effectively select words and views respectively. Given a travel package x j , x j =<Title,Destination ,Categories>, Title includes country/city, attractions, hotel transportation and travel days, etc. Destination includes one or a few words, usually represented by city or country name, Category includes travel Region (Travel Region) and Travel Type (Travel Type), the representation vector learned by the travel package encoding module is r j .
The method for recommending travel packages based on a multi-view attention mechanism according to claim 1, wherein in step 2), the user's long-term and short-term preference coding modules respectively design two bidirectional long-short-term memory neurons with an attention mechanism The network (Bi-directional Long Short-term Memory, Bi-LSTM) dynamically learns the user's long-term and short-term preferences from the user's history and the current click stream, respectively. The Bi-LSTM neural network with an attention mechanism can learn from the user's continuous click stream. Select important travel packages to accurately represent the user's preferences. The representation vector of the user's long-term and short-term preferences of the user's long-term and short-term preference encoding module is obtained by summing the travel package context representation vectors with attention weights. Given a target user u, so that it short-term and long-term behavior are denoted by L and S u u, the short and long term behavior of the user characterization vectors and are denoted as s u l u.
The method for recommending travel packages based on a multi-view attention mechanism according to claim 3, wherein step 3) is specifically: let the user preference query vector be q u , and combine the short-term preference vector s u with the long-term preference vector l With u as input, the gate structure vector F u is used to control the contribution of long-term and short-term preferences:

F u =sigmod(W q q u +W s s u +W l l u +b u ),

Wherein W q, W s, W l and B u are parameters of the neural network projection learning model Q u represent, s u, l u weight matrix, and a bias vector, preference characterizing the user of the final output of u The vector O u can be calculated by:

O u =(1-F u )⊙s u +F u ⊙l u ,

where ⊙ is the vector inner product symbol.
The method for recommending travel packages based on a multi-view attention mechanism according to claim 4, wherein step 4) is specifically: in the NATR model training stage, the positive label is the next really purchased travel package
while negative labels are removed from the travel pack set X
The set of travel packages formed by log-uniform sampling, after obtaining the user preference vector O u and the characterization vector r j of the travel package, the travel package z in the candidate travel package z={z 1 ,z 2 ,...,z | κ| } The recommended score z k can be calculated, and the recommended score is calculated as follows:

z k =O u T r j ,

κ is the subset containing positive and negative labels sampled from X, || represents the norm of the set, and X represents the set of travel packages;

The predicted probability of the travel package in κ is
Application softmax function to obtain an output NATR model, i.e., y - = softmax (z), y - is the probability that all travel at a session S u respectively purchased,
is the probability that the |k|th travel package is purchased.

For each user, the loss function based on the purchase probability defined by cross entropy and the real situation can be calculated by the following method:

where y j is the probability distribution that the travel package x j is actually purchased, specifically, if x j is a positive label, then y j =1, otherwise y j =0.