Disclosure of Invention
In view of the foregoing, it is necessary to provide a cross-domain recommendation method and implementation system thereof that enables a fusion tag and attention mechanism with more diversified features of a recommendation structure by analyzing user preferences in different domains.
A cross-domain recommendation method integrating labels and attention mechanisms comprises the following steps:
selecting and constructing a cross-domain fusion label, fusing two similar domains into a new domain, taking the new domain as a source domain recommended by the cross-domain, and respectively carrying out weighted summation on label vectors of the source domain and the target domain to obtain a resource vector of each resource;
secondly, learning a time sequence relation between a user and a resource through a long-short-term memory algorithm LSTM according to an interest mining algorithm based on an attention mechanism, and introducing the attention mechanism to obtain the preference of the user in a specified time period of a source field and a target field;
thirdly, according to a cross-domain label mapping algorithm based on the BP neural network, a label mapping between a source domain and a target domain is learned through a three-layer BP neural network, and after a preference vector of a user source domain is mapped to the target domain, the preference vector of the user source domain and the preference vector of the user in the target domain are weighted and summed to obtain comprehensive preference of the user in the target domain;
and step four, a cross-domain recommendation algorithm integrating tag mapping and an attention mechanism calculates the similarity between the resource vector which is not browsed by the user in the target domain and the comprehensive preference of the user according to the comprehensive preference of the user in the source domain and the target domain, and recommends the first N items to the user.
Further, the selecting and constructing a cross-domain fusion label flow in the step one includes:
step 1-1, preprocessing data in a source field and a target field;
collecting commonly used labels in two similar fields A and B and a target field C as custom labels DT, retrieving resources related to the custom labels to obtain resource labels RT corresponding to the resources, removing repeated labels to obtain RT-DT matrixes, wherein each column of the matrixes is custom label DT vectors, each action resource label RT vector, and uniformly measuring the rest custom label vectors;
step 1-2, selecting and constructing a cross-domain fusion tag;
analyzing similarity of the same custom tag DT vector and the same resource tag RT vector in the A, B field, and calculating similarity of the same DT vector and the same RT vector in the user A and B resources by using cosine similarity, wherein the similarity is shown in a formula (1):
wherein ,
and
Represents the same DT or RT in the A-domain and B-domain, respectively,>
、
labels->
and
Vector representations of (a); to ensure recommended quality, labels with similarity below a threshold are culled;
in each field of constructed tag vectors, the custom tag DT and the resource tag RT are filtered, and the same resource tag RT vector is weighted and summed according to the interests of users in each field to obtain an A-B field tag vector matrix, as shown in the formula (2) and the formula (3):
wherein ,
to fuse tags across domains->
For labels->
Is a vector representation of (c).
The resource labels RT of the resources are ordered according to the number of times the labels are marked by users, the labels in front are more closely related to the resources than the labels in back, and the labels of each resource are assigned with weights according to the formula (4):
the corresponding weight of each tag vector of all the resources is obtained by using the formula (4) for the A-B field and the C field, and then the weight summation of the tag vectors is carried out to obtain the resource vector of each resource, as shown in the formula (5):
wherein ,
tag vector for each tag of a resource, +.>
Is a resource vector of resources.
Further, the data preprocessing of the source domain and the target domain in the step 1-1 comprises the following steps:
step 1-1-1, respectively acquiring labels from texts of A, B domain resources by using a TF-IDF technology, and extracting labels which are commonly used by m users and are simultaneously displayed in two domains, namely a custom label DT, from the acquired labels, wherein the labels can better represent resource characteristics and are marked as DTs; then searching the resources related to DT in A, B field, displaying the label corresponding to each resource, namely resource label RT, and recording as RTs;
step 1-1-2, the labels commonly used by N users are collected from the C field by using TF-IDF technology and are marked as DTt. Then searching the resources related to the DT, and displaying the label corresponding to each resource, and marking the label as RTt;
step 1-1-3, for the collected labels, finishing word segmentation by using an NLPIR Chinese word segmentation system, removing repeated labels, and counting the occurrence frequency of each RT in the resources corresponding to each DT, wherein the larger the vector value is, the tighter the relation between RT and DT is;
step 1-1-4, because the total number of resources retrieved by different DTs is different, dividing each component of all DT vectors by the largest component of the vector to obtain a DT vector with uniform metrics.
Further, the interest mining algorithm based on the attention mechanism in the second step includes the following steps:
learning a timing relationship between a user and a resource using a long-short-term memory algorithm, assuming that an update interval of a memory cell layer is a time step
,
Is the memory cell layer at time->
Is input to the computer;
And->
Is a weight matrix;
Is a bias vector; the method comprises the following steps:
step 2-1, at each time step
Multiplying the input information of the input gate by the weight value, adding the offset, and calculating to obtain a control variable +.>
And a new input vector->
Specifically, the method is shown as a formula (6) and a formula (7):
step 2-2, at each time step
Calculating the control variable +.>
And candidate state->
Multiplication, namely multiplying forgetting gate input information by weight, adding the product of the input gate input information and the weight, and then taking the memory unit state from +.>
Update to->
Specifically, the method is shown as a formula (8) and a formula (9):
step 2-3, continuously calculating the value of the output gate in the updated memory cell state, specifically as shown in the following formulas (10) and (11):
step 2-4, calculating a loss function and minimizing it using a gradient descent method, as shown in formula (12):
after being calculated by a long-short-term memory algorithm LSTM layer, each time step hidden state of the LSTM is used as an output result to be output to an attention layer so as to capture the dependency relationship among sequences, and a context vector representation corresponding to an output sequence i is obtained after weighted summation
Description of specific formulasAs shown in the formula (13) and the formula (14):
wherein ,
is LSTM No
iOutput of individual time steps,/->
Represent the first
iTime step and No. H
jThe normalized weight is output in each time step, and the similarity calculation function adopts matrix transformation represented by W.
Further, according to the interest mining algorithm based on the attention mechanism, after all resource vectors of the histories of the user in the source domain and the target domain are calculated through the LSTM layer and the attention layer respectively, preference vectors of the user in the source domain and the target domain are obtained.
Further, the cross-domain label mapping algorithm based on the BP neural network in the third step comprises the following procedures:
the label mapping between different fields is learned through a three-layer BP neural network, firstly, n labels which are used most by a user are respectively collected in an A-B field and a C field, the weight of each label in the two fields is respectively calculated according to a formula (4), and then the feature vectors of the user in the A-B field and the C field are respectively calculated through the weights, wherein the feature vectors are specifically shown as a formula (15) and a formula (16):
wherein ,
for the feature vector of the user in the A-B field, < >>
and
The weight of each label in the A-B field and the characteristic vector of the label are obtained;
For the feature vector of the user in the C field, +.>
and
The weight of each label in the C field and the characteristic vector of the label are obtained;
then, by
For inputting vectors, ++>
For the actual output vector, learning +.>
and
The mapping is shown as the following formulas (17) - (21):
the mapping relation between the input layer and the hidden layer is shown in the formula (17):
the activation function through the hidden layer is shown in equation (18):
the mapping relationship from the hidden layer to the output layer is shown in the formula (19):
the activation function through the output layer is shown in equation (20):
the loss function of the BP neural network is shown as a formula (21):
wherein in the formulas (14) and (16),
;
nthe number of hidden layer units for BP neural network, i.e.
, wherein ,
For the number of input layer units, < > for>
Is the number of output layer elements. />
Further, the number of hidden layer units of the BP neural networknThe method is determined by adopting a golden section method.
Further, the cross-domain recommendation algorithm fusing the tag mapping and the attention mechanism in the fourth step specifically includes the following steps:
step 4-1, setting the user preference vector of the history record in the A-B field after being processed by the LSTM layer and the attention layer as
The user preference vector of the history record in the C field after being processed by the LSTM network is +.>
The mapping network between the source domain and the target domain is +.>
Will be +.>
and
Weighted summation to obtain final user resource vector:
wherein ,
resource vector representing user->
Representing the number of target domain resources in the user history,
representing the number of source domain resources in the user history;
step 4-2, after the final user resource vector is obtained by calculation, calculating the similarity between the user resource vector and the resource vector in the target field in the resource library, wherein the similarity is specifically shown as a formula (23):
wherein ,
representing user resource vector, ++>
A resource vector representing a target domain in a resource pool.
Further, in the calculation of the final user resource vector, according to different browsing amounts of the user in the target field, the similarity of the resource vector which is not browsed in the target field by the user is directly affected; when the data volume browsed by the user in the target field is small, the preference mapping of the user in the source field can be used for obtaining the preference calculation of the user in the target field, and the similarity of the resource vectors not browsed by the user in the target field; when the browsing data quantity of the user in the target field is far greater than that of the user in the source field, the data sparsity does not exist, which is equivalent to obtaining a final recommendation result directly according to the browsed resource vector in the target field.
And a system for implementing a cross-domain recommendation method of a fusion tag and an attention mechanism, where the system is configured to implement a cross-domain recommendation method of a fusion tag and an attention mechanism according to any one of the above, and the system includes:
the method comprises the steps of obtaining resource labels of resources corresponding to the custom labels by utilizing commonly used custom labels in two similar fields A and B based on a label cross-domain resource fusion algorithm module, analyzing the similarity of the custom label DT vector and the same resource label RT vector in the A, B field, and eliminating the resource label vector with the similarity lower than a threshold value; respectively carrying out weighted summation on resource tag vectors of a source field and a target field C which are formed by two similar fields A and B to obtain resource vectors of each resource of the source field and the target field;
the interest mining algorithm module based on the attention mechanism obtains preference vectors of the user in the source field and the target field by utilizing all resource vectors of the history records of the user in the source field and the target field and through deep learning of an LSTM layer and an attention layer of the long-term memory algorithm;
the cross-domain label mapping algorithm module based on the BP neural network is used for mapping the preference vector of the user source domain to the target domain by using the three-layer BP neural network to obtain the comprehensive preference of the user in the target domain;
and the cross-domain recommendation algorithm module is used for combining the preferences of the user in the source domain and the target domain to obtain the comprehensive preferences of the user in the target domain, and then calculating the similarity between the resource vector which is not browsed in the target domain by the user and the comprehensive preferences of the user to obtain a recommendation list of the user in the target domain.
In the cross-domain recommendation method of the fusion tag and the attention mechanism and the implementation system thereof, the cross-domain fusion tag is constructed by fusing the item tags of two similar domains, the two similar domains are fused into a new domain, the new domain is used as a source domain of the cross-domain recommendation, and then a domain with lower correlation degree is used as a target domain, so that the data of the source domain of the cross-domain recommendation is richer. The current preference of the user is calculated in the source field and the target field respectively through the attention introducing mechanism, so that the recommendation result is more time-efficient. The mapping relation of the overall preference of the user in the source field and the target field is obtained through the BP neural network, the mapping relation is applied to the current preference of the user, the mapping relation is mapped from the source field to the target field, and the final recommendation result is more accurate by combining with the independent recommendation result of the target field.
Detailed Description
In this embodiment, a cross-domain recommendation method and an implementation system thereof that integrate a label and an attention mechanism are taken as an example, and the present invention will be described in detail below with reference to specific embodiments and accompanying drawings.
Referring to fig. 1 and fig. 2, a cross-domain recommendation method for fusing a tag and an attention mechanism is shown in an embodiment of the present invention.
The cross-domain resource recommendation is realized by using the labels and the attention mechanism, and the comprehensive preference of the user in the target domain is obtained by mapping the preference of the user in the source domain to the target domain and combining the preference of the user in the target domain.
Because of the sparsity of the data in a single field, the accuracy of recommendation is reduced, and if the data in a plurality of fields can be combined, the reliability of a recommendation result can be greatly improved. If the data of the user in the target field is sparse, the preference of the source field can be obtained through the mapping network to obtain the corresponding target field preference, and if the data of the user in the source field is sparse, the data of the user in the source field can be solved by fusing a similar field. If the data richness of the user in the target field is much higher than the data of the user in the fused source field, the preference of the user can be learned directly through the data of the user in the target field.
Two similar fields are fused into a new field by constructing a cross-field label vector, and the new field is used as a source field of cross-field recommendation. Firstly, extracting labels of each field by using a natural language processing method, customizing corresponding label vectors, fusing the label vectors with similarity exceeding a specified threshold in the two fields, and customizing cross-field fusion labels. Second, by introducing an attention mechanism in combination with LSTM, user preferences over a specified period of time are calculated in the source domain and the target domain, respectively. And thirdly, a BP neural network is used for learning a mapping relation between the source domain label and the target domain label, and data of the target domain and the source domain are combined, so that a recommendation result is more accurate. Finally, an overall framework of a cross-domain recommendation model is described, the obtained user source domain preferences are mapped to the target domain through a learned mapping network, and a final result is obtained by combining the user source domain preferences with the user target domain preferences.
1. Cross-domain resource fusion of labels
The fields are not isolated, a certain relation exists between the fields, and when the similarity exceeds a certain degree, the fields can be combined into a new field. The number of resources in the new domain is the sum of the number of resources in two similar domains, and the sum is used as a source domain recommended across domains, so that the information of the source domain can be enriched.
1.1 data Pre-processing
The method is characterized in that A and B are two similar fields, the combined field of A and B is taken as a source field, the other field C with larger span is taken as a target field, and the processing steps are as follows:
1) The TF-IDF technology is used for respectively acquiring labels from texts of the A and B domain resources, and then m labels which are commonly used by users and are simultaneously displayed in the two domains, namely custom labels (DT for short), are extracted from the acquired labels, and the labels can well represent the resource characteristics and are marked as DTs. Then, the resources related to DT are searched in A, B field, and then the label corresponding to each resource, i.e. the resource label (RT for short), is displayed and is recorded as the RTs.
2) The TF-IDF technique was used to collect N labels commonly used by users from the C field, denoted DTt. And then searching the resources related to the DT, and displaying the label corresponding to each resource, and recording as RTt.
3) And for the collected labels, finishing word segmentation by using an NLPIR Chinese word segmentation system, removing repeated labels, and counting the occurrence frequency of each RT in the resources corresponding to each DT, wherein the larger the vector value is, the tighter the relation between RT and DT is.
4) Because the total number of resources retrieved by different DTs is different, dividing each component of all DT vectors by the largest component of the vector results in a uniformly measured DT vector.
1.2 selection and construction of Cross-Domain fusion tags
We need to analyze the correlation of the same DT vector and the same RT vector in A, B domain, which can be exploited and cross-domain recommended only if the correlation is large enough. If the same DT vector in the a-field and the B-field or the single components of the same RT vector are similar they cannot be said to be highly correlated, but if all the components are similar, then the correlation of the two is considered to be sufficiently high. And calculating the similarity of the same DT vector and the same RT vector in the resources of the user A and the user B by using the cosine similarity, as shown in the formula (1).
wherein ,
and
Represents the same DT or RT in the A-domain and B-domain, respectively,>
、
labels->
and
Is a vector representation of (c). To ensure recommended quality, labels with similarity below a threshold are culled.
After the label vector of each field is constructed and the DT and RT are screened, the same RT vector is weighted and summed according to the interests of the user in each field, and the A-B field label vector matrix is obtained, as shown in the formula (2) and the formula (3).
wherein ,
to fuse tags across domains->
For labels->
Is a vector representation of (c).
The RT of the resource is ordered according to the number of times the tag is marked by the user, the preceding tag is more closely related to the resource than the following tag, and the tag of each resource is assigned a weight according to equation (4).
And (3) respectively using the formula (4) to obtain the weight corresponding to each tag vector of all the resources in the A-B field and the C field, and then carrying out weighted summation on the tag vectors to obtain the resource vector of each resource, wherein the weight is shown in the formula (5).
wherein ,
tag vector for each tag of a resource, +.>
As a resource vector of resources, the process is shown in algorithm 1.
2. Interest mining algorithm based on attention mechanism
The attention points of different people to the resources are different, some people may like the resources of the history subject, some people like the resources of the science fiction class, and some people like the resources of the reasoning class. To accurately discover the user's preferences, the present software uses LSTM to learn the timing relationship between the user and the resource. See in particular the model below.
Assume that each time step is
The memory cell layer will be updated and assume:
1)
is the memory cell layer at time->
I.e. the user's browsing record.
2)
And->
Is a weight matrix.
3)
Is a bias vector. The method comprises the following steps:
(1) at each time step
Multiplying the input information of the input gate by the weight value, adding the offset, and calculating to obtain a control variable ∈of the input gate>
And a new input vector->
Specifically, the formula (6) and the formula (7) are shown. />
(2) At each time step
The control variable +.>
And candidate state->
Multiplication, namely multiplying forgetting gate input information by weight, adding the product of the input gate input information and the weight, and then taking the memory unit state from +.>
Update to->
Specifically, the formula (8) and the formula (9) are shown.
(3) After the updated memory cell state, the output gate value can be continuously calculated, specifically shown in the following formulas (10) and (11).
(4) The loss function is calculated and minimized using a gradient descent method as shown in equation (12).
After passing through the LSTM layer, the hidden state of each time step of the LSTM is an output result, and at the moment, each time step of the LSTM output is output to the attention layer, so that the dependency relationship among the sequences can be captured, and after weighted summation, the context vector representation corresponding to the output sequence i is obtained
The specific formula descriptions are shown in the formula (13) and the formula (14).
wherein ,
is LSTM No
iOutput of individual time steps,/->
Represent the first
iTime step and No. H
jThe normalized weights of the outputs of the time steps are used, and the similarity calculation function is represented by matrix transformation, which is represented by W. And finally, a full connection layer is used to obtain the final output probability. This process is shown in algorithm 2. />
All resource vectors of the histories of the user in the source domain and the target domain respectively pass through the LSTM layer and the attention layer to obtain preference vectors of the user in the source domain and the target domain, the preference vectors of the user in the source domain are mapped to the target domain, and a final result is obtained by combining the preference of the user in the target domain, and the mapping process is specifically described in the next part.
3. Cross-domain label mapping algorithm based on BP neural network
There is also a certain correspondence between the interests of the user among different fields. For example, users who like to watch comedy are mostly biased to listen to rock-and-roll type music, while users who like to watch horror generally like to listen to music with more pronounced melody changes. The label mapping among different fields is learned through a three-layer BP neural network, firstly, n labels which are used most by a user are respectively collected in the A-B field and the C field, the weight of each label in the two fields is respectively calculated according to a formula (4), and then the feature vectors of the user in the A-B field and the C field are respectively calculated in a weighted mode, as shown in a formula (15) and a formula (16).
wherein ,
for the feature vector of the user in the A-B field, < >>
and
Weights for each tag in A-B domain and the sameFeature vectors of the tags;
For the feature vector of the user in the C field, +.>
and
The weight of each label in the C field and the characteristic vector of the label are obtained.
Then, by
For inputting vectors, ++>
For the actual output vector, learning +.>
and
The mapping is shown as the following formulas (17) - (21):
1) The mapping relation between the input layer and the hidden layer is shown in the formula (17):
2) The activation function through the hidden layer is shown in equation (18):
3) The mapping relationship from the hidden layer to the output layer is shown in the formula (19):
4) The activation function through the output layer is shown in equation (20):
5) The loss function of the BP neural network is shown as a formula (21):
wherein in the formulas (14) and (16),
a concrete model structure is shown in fig. 2.
In the BP neural network, the number of hidden layer units is directly related to the requirements of the problem and the number of input and output units. Too few numbers and too few information are acquired, under-fitting occurs. Too many, increasing training time, overfitting is likely to occur and generalization ability is poor. The software uses the golden section method to determine the number of hidden layer units of the BP neural network. That is to say,
, wherein ,
For the number of input layer units, < > for>
Is an output layer unitNumber of parts. Finally, a gradient descent method is used to minimize the loss function. This process is shown in algorithm 3.
After mapping the preference vector of the user source domain to the target domain, the preference vector of the user in the target domain is weighted and summed with the preference vector of the user in the target domain in a certain manner to obtain the comprehensive preference of the user in the target domain, and the process is described in detail in the next part.
4. Cross-domain recommendation framework integrating tag mapping and attention mechanisms
From the first three sections, a cross-domain recommendation algorithm that fuses the tag mapping and the attention mechanism will be given below. After mapping the preference vector of the user source domain to the target domain, it needs to be weighted and summed with the preference vector of the user target domain in a certain way.
The meaning of cross-domain recommendation is to solve the problem of sparse data of users in the target domain. The data of the user in the target area is rich enough relative to the data of the source area, so that the data of the user in the target area is used more, the weight can be determined according to the resource number ratio of the user in the source area and the target area, and the whole framework of the algorithm is shown in figure 1.
Recording the user preference vector of the history record in the A-B field after being processed by the LSTM layer and the attention layer as
The user preference vector of the history record in the C field after being processed by the LSTM network is +.>
The mapping network between the source domain and the target domain is +.>
Will be +.>
and
And (5) weighting and summing to obtain a final user resource vector.
wherein ,
resource vector representing user->
Representing the number of target domain resources in the user history,
representing the number of source domain resources in the user history. If the data volume browsed by the user in the target field is small, the preference in the target field can be obtained through the preference mapping in the source field; if the data of the user in the target field is far more than the data of the user in the source field, the data sparsity does not exist, which is equivalent to obtaining a final result directly according to the result recommended in the single field.
After the final user resource vector is calculated, the similarity between the user resource vector and the resource vector of the target field in the resource library is calculated by using the formula (23), and the TOPN list is recommended to the user.
wherein ,
representing user resource vector, ++>
A resource vector representing a target domain in a resource pool. This process is shown in algorithm 4. />
And the algorithm 4 combines the preference of the user source field with the preference of the target field to obtain the comprehensive preference of the user in the target field, calculates the similarity between the comprehensive preference and the resource vector which is not browsed by the user in the target field, and recommends the first N items to the user. The more resource rich areas have a greater impact on the end result.
And a system for implementing a cross-domain recommendation method of a fusion tag and an attention mechanism, where the system is configured to implement a cross-domain recommendation method of a fusion tag and an attention mechanism according to any one of the above, and the system includes:
the method comprises the steps of obtaining resource labels of resources corresponding to the custom labels by utilizing commonly used custom labels in two similar fields A and B based on a label cross-domain resource fusion algorithm module, analyzing the similarity of the custom label DT vector and the same resource label RT vector in the A, B field, and eliminating the resource label vector with the similarity lower than a threshold value; respectively carrying out weighted summation on resource tag vectors of a source field and a target field C which are formed by two similar fields A and B to obtain resource vectors of each resource of the source field and the target field;
the interest mining algorithm module based on the attention mechanism obtains preference vectors of the user in the source field and the target field by utilizing all resource vectors of the history records of the user in the source field and the target field and through deep learning of an LSTM layer and an attention layer of the long-term memory algorithm;
the cross-domain label mapping algorithm module based on the BP neural network is used for mapping the preference vector of the user source domain to the target domain by using the three-layer BP neural network to obtain the comprehensive preference of the user in the target domain;
and the cross-domain recommendation algorithm module is used for combining the preferences of the user in the source domain and the target domain to obtain the comprehensive preferences of the user in the target domain, and then calculating the similarity between the resource vector which is not browsed in the target domain by the user and the comprehensive preferences of the user to obtain a recommendation list of the user in the target domain.
In the cross-domain recommendation method of the fusion tag and the attention mechanism and the implementation system thereof, the cross-domain fusion tag is constructed by fusing the item tags of two similar domains, the two similar domains are fused into a new domain, the new domain is used as a source domain of the cross-domain recommendation, and then a domain with lower correlation degree is used as a target domain, so that the data of the source domain of the cross-domain recommendation is richer. The current preference of the user is calculated in the source field and the target field respectively through the attention introducing mechanism, so that the recommendation result is more time-efficient. The mapping relation of the overall preference of the user in the source field and the target field is obtained through the BP neural network, the mapping relation is applied to the current preference of the user, the mapping relation is mapped from the source field to the target field, and the final recommendation result is more accurate by combining with the independent recommendation result of the target field.
It should be noted that the above-mentioned embodiments are merely preferred embodiments of the present invention, and are not intended to limit the present invention, but various modifications and variations of the present invention will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.