JP7393060B2 - Personalized search method and search system combining attention mechanism - Google Patents

Description

The present invention belongs to the technical field of data mining, and specifically relates to a personalized search method and search system.

With the rapid development of big data, cloud computing, ubiquitous network and other technologies, the scale and number of users of the Internet have increased rapidly, and users have already become active creators of data, generating a large amount of multi-source heterogeneous users. Content is gathering, and various information is becoming complicated and increasing explosively. User-generated content contains a large amount of dynamically evolving, complex multi-source heterogeneous data, has characteristics such as source and structure diversification, sparsity, multi-mode, incompleteness, social propagation, etc., and has rich value. It has a large amount of information and huge mining potential, and is also an important source for various Internet platforms and mobile application companies to obtain information and improve their business performance and services, making it a typical big data environment. . However, while these complex multi-source heterogeneous user-generated contents bring new information to users, they also lead to increased difficulty for users to filter, sort and process information and ultimately make decisions, i.e., “information overload”. This will cause a load problem. Personalized search and recommendation algorithms, as a bridge between users and information, make full use of large amounts of multi-source heterogeneous user-generated data, and adjust user behavior and development dynamics according to users' latent needs and cognitive preferences. Anticipate and help users filter content that matches their needs and hobbies from a large amount of information as much as possible, effectively reducing "information overload" and improving the user experience and the commercial interests of the website platform. can be improved.

The essence of the personalized search task for user-generated content is to search for an optimal goal that satisfies users' needs and personalized preferences in a dynamic evolving space consisting of multi-source heterogeneous user-generated data, i.e., dynamic It is a qualitative index optimization problem. This complex qualitative index optimization problem not only cannot accurately describe its objective function and performance index with mathematical functions, but also the decision variables of the optimization problem are not simple structured data and are often subject to subjectivity. , since there is a large degree of ambiguity, uncertainty and inconsistency, and users need to perform qualitative analysis, evaluation and decisions on the items to be searched based on their experiential knowledge and hobbies and preferences, certain exact mathematics are not required. It is difficult to construct and describe a model. In recent years, an interactive co-evolutionary calculation that combines human intelligent evaluation has been proposed, which combines the user's subjective cognitive experience, intelligent evaluation/decision, and traditional evolutionary calculation to optimize the qualitative indicators for the above-mentioned complex personalized search. It is an effective way to handle the problem.

The Chinese patent with application number CN2020102165574 discloses a restricted Boltzmann machine-driven based interactive personalized search method, in which the construction of a user's taste preference model is such that the decision variables of different item attributes are different for user preferences. Because it applies the same weight to the decision variables of the used items without considering their influence, the influence of each decision variable on user preferences cannot be fully reflected, thereby creating a more accurate user preference model. They are difficult to construct and further reduce the effectiveness of personalized searches for users.

Regarding the purpose of the invention, to address the problems existing in the prior art, the present invention provides a personalized search method and search system that integrates an attention mechanism, and the search method is designed so that different determining components have different influences on user preferences. By considering what you have, you can help users conduct personalized searches more effectively.

Regarding the technical solution, according to one aspect of the present invention, a personalized search method integrated with an attention mechanism is disclosed,
Collect and retrieve user-generated content including all items that user u has rated, scores and text comments for each item, images for each item, and other users' usefulness evaluation scores for user u's ratings, and text comments. Step 1 of vectorizing the item image, extracting features from the item image, and obtaining the eigenvector;
Step 2 of configuring a dominant item group D including user preferences with items having a user score larger than a predetermined score threshold and a reliability larger than a predetermined confidence threshold, the items in D forming a set S; , S={(u, x _i , C _i , T _i , G _i )}, where x _i ∈D, C _i is the category label vector of item x _i , and T _i is the item is a vectorized representation of the user's text comment for x _i , G _i is a vectorized representation of the image features of item x _i , i = 1, 2, ..., |D|, and |D| Step 2 representing the number of items in D;
A user preference sensing model that combines attention mechanisms is constructed, and the model is composed of three layers of restricted Boltzmann machines based on a deep belief network, and the visible layer of the first layer of restricted Boltzmann machines is the first set of visible layers. unit v ₁ , a second set of visible units v ₂ and a third set of visible units v ₃ , the hidden layer is h ₁ , and h ₁ is a visible layer that includes a second layer RBM together with a hidden layer h ₂ . The parameters of _the user preference _sensing model that combines the above attention mechanisms are θ={θ ₁ , θ ₂ , θ ₃ }={ w ₁ , a ₁ , b ₁ , w ₂ , a ₂ , b ₂ , w ₃ , a ₃ , b ₃ },
Using the dominant item group D, train the first layer RBM in the user preference sensing model that combines the attention mechanism with the contrastive divergence learning algorithm, and set the model parameters θ ₁ = {w ₁ , a ₁ , b ₁ } get
After the training of the first layer RBM model is completed, given the state of the hidden unit, the activation state of each visible unit is conditionally independent, and the vector representation of some item x _i [C _i , T _i , G _i ] is input to the visible layer, and the activation probabilities of the first set, second set, and third set of visible units are respectively

, where a _1,j , a _1,k and a _1,l represent the biases of the first, second and third sets of visible units, respectively;
The information entropy of various multi-source heterogeneous data is calculated, and the information entropy of item category label is
and
The information entropy of the text comment vector is
and
The information entropy of the item image feature vector is
and
Here, c _ij represents the j-th element of the category label vector C _i of item x _i , and p ₍ c _ij) is the activation of the visible unit corresponding to the j-th element of the vector representation of the item category label in RBM1. represents the probability,
t _ik represents the kth element of the vectorized representation T _i of user u's text comment for item x _i , and p ₍ t _ik) is the visible vector representation T i of the vectorized representation of user u's text comment in RBM1. represents the activation probability of the unit,
g _il represents the lth element of the vectorized representation G _i of the image features of item x _i , and p _(gil) is the activation of the visible unit corresponding to the lth element of the vectorized representation of the item image features in RBM1. represents the probability,
Next, calculate the ratio of various information entropies to the total information entropy as a weighting factor,
Here, H(x _i )=H(C _i )+H(T _i )+H(G _i ),
When vectors C _i , T _i , G _i are combined to form the decision vector Ψ _i of item x _i and inputted to each visible unit in v ₁ , v ₂ , v ₃ , the activation of each hidden unit in hidden layer h ₁ is The activation states are conditionally independent, and the activation probability of the m _- th hidden unit is
and
Here, m ₁ =1, 2,..., M ₁ ,
is the bias of the m _-th hidden unit in h ₁ , v _1j is the state of the j-th visible unit in the first set of visible units v ₁ of RMB1, and v _2k is the state of the j-th visible unit in the second set of visible units of RMB1. v ₂ is the state of the k-th visible unit, v _3l is the state of the l-th visible unit in the third set of visible units v ₃ of RMB1,
is the element value in w ₁ and represents the connection weight between the nth visible unit and m _1st hidden unit in RBM1, where n = 1, 2, ..., Φ,
represents the state of the m _- th hidden unit in the hidden layer _h1 ,
is the sigmoid activation function,
After the training of RBM1 is completed, the state of each hidden unit corresponding to item x _i is obtained according to equation (9), and the user's preference degree for each determining component of each item in dominant item group D, that is, the state of the visible layer Obtain the activation probability of the unit as an attention weighting coefficient at _n (x _i ),
here,
If Ψ _i is the state of each visible unit in the visible layer of RBM1, represents the state of the m _i -th hidden unit in the hidden layer h _i , and at _n (x _i ) is each determinant component ψ _in of item x _i represents the attention weight of
Encoding is performed on the item x _i in the dominant item group D based on the attention mechanism using the attention weight coefficient at _n (x _i ) as the weight coefficient of each determining component of the item x _i , and after encoding, it is expressed as x _ati ,
x _ati is input to RBM1 after pre-training to obtain the activation probability V _RBM1 (x _ati ) of the visible unit,
here,
is the n'th element of x _ati ,
Perform a self-attention mechanism calculation using the activation probability V _RBM1 (x _ati ) of the visible unit of RBM1, dynamically learn the user preference attention weight vector A (x _ati ) for each item,
Here, the softmax() function ensures that the sum of all weighting coefficients is 1, and the function a(V _RBM1 (x _ati ), w ₁ ) measures the attention weighting coefficient of item x _i with respect to user preference features. And the calculation is
and
Combining the user preference attention weight vector A(x _ati ) and the original decision vector C _i , T _i , G _i of item x _i to generate an item decision vector that fuses the attention mechanism;
Item decision vector fused with attention mechanism
construct a training set, train the RBM1, RBM2, and RBM3 models in the DBN layer by layer, and after the training is completed, obtain the DBN-based user preference sensing model that fuses the attention mechanism and its optimization model parameters θ. Step 3 and
Build a user preference-based distribution estimation probability model P(x) based on the DBN-based user preference sensing model fused with the trained attention mechanism and its model parameters;
P(x)=[P(ψ ₁ ), P(ψ ₂ ), ..., P(ψ _n ), ..., P(ψ _Φ )] (17)
Here, (ψ ₁ , ψ ₂ , ..., ψ _n , ..., ψ _Φ ) is the original decision vector of item x, and P (ψ _n ) is the user's decision vector for the nth decision component of item Step 4 representing the preference probability of
Set the population size N, use the user preference-based distribution estimation probability model P(x), and adopt the distribution estimation algorithm to generate N new individuals, each individual with one item. Yes, the category label vector of the vth new individual
The setting step for (v=1, 2,...,N) is
(5.1) Let v=1,
(5.2) Generate a random number z of [0, 1], and if z≦P (ψ _j =1), the category label vector of the vth new individual
The jth element of is 1, otherwise 0,
(5.3) Step 5, which is to add 1 to v and repeat step (5.2) until v>N;
Category label vector of N new individuals in search space
Step 6 of selecting the N items with the highest similarity to constitute a recommendation target item set S ^u ;
Adaptation value of each item in the recommendation target item set S ^u
Calculate,
here,
as well as
respectively represent the maximum value and minimum value of the energy function of the item in the recommendation target item set S ^u ,
is the energy function of item x ^* , x ^* ∈S ^u , and its calculation is
and
here,
is the nth determining component of item x ^* , step 7;
Selecting the top N items with the highest adaptation value in S ^u as the search results, TopN<N, step 8;
With the promotion of the user's interactive search process and the dynamic evolution of user behavior, we will update the dominant item group D according to the latest evaluation data of the current user, and retrain the user preference sensing model that combines the attention mechanism. , dynamically updates the extracted user preference features, and at the same time updates the user preference-based distribution estimation probability model P(x).

According to another aspect, the present invention further discloses a search system that implements the personalized search method,
Collecting and retrieving content generated by user u, including all items rated by user u, scores and text comments for each item, images of each item, and other users' usefulness evaluation scores for user u's ratings; a user-generated content acquisition module that vectorizes text comments, performs feature extraction on item images, and acquires eigenvectors;
a superior item group construction module that configures a superior item group D including user preferences with items whose user score is greater than a predetermined score threshold and whose reliability is greater than a predetermined reliability threshold;
A user preference sensing model building and training module that builds and trains a user preference sensing model that integrates an attention mechanism, the model is composed of a three-layer restricted Boltzmann machine based on a deep belief network, and the first layer The visible layer of the restricted Boltzmann machine includes a first set of visible units v ₁ , a second set of visible units v ₂ and a third set of visible units v ₃ , the hidden layer is h ₁ , and h ₁ is the visible layer. , a second-layer restricted Boltzmann machine is constructed with the hidden layer _h2 , _h2 is the visible layer, and a third-layer restricted Boltzmann machine is constructed with the hidden layer _h3 . The parameters of the preference sensing model are θ={θ ₁ , θ ₂ , θ ₃ }={w ₁ , a ₁ , b ₁ , w 2 , a ₂ , b _{2 ,} w ₃ , a ₃ , b ₃ } A preference sensing model construction training module;
Build a user preference-based distribution estimation probability model P(x) based on a deep belief network-based user preference sensing model that combines a trained attention mechanism and its model parameters,
P(x)=[P(ψ ₁ ), P(ψ ₂ ), ..., P(ψ _n ), ..., P(ψ _Φ )] (17)
Here, (ψ ₁ , ψ ₂ , ..., ψ _n , ..., ψ _Φ ) is the original decision vector of item x, and P (ψ _n ) is the user's decision vector for the nth decision component of item a user preference-based distribution estimation probability model construction module representing preference probabilities;
Using the user preference-based distribution estimation probability model P(x), a distribution estimation algorithm is adopted to generate N (N is the size of a given population) new individuals, each of which is one item. and a population generation module that sets a category label vector for each new individual;
Category label vector of N new individuals in search space
a recommendation target item set construction module that selects N items having the highest degree of similarity to constitute a recommendation target item set S ^u ;
an adaptation value calculation module that calculates the adaptation value of each item in the recommendation target item set S ^u ;
A search result selection module that selects TopN items with the highest adaptation value in ^Su as search results, and where TopN<N.

For beneficial effects, the personalized search method disclosed in the present invention makes full use of multi-source heterogeneous user-generated content including user scores, item category labels, user text comments, rating reliability and item image information; Build a user preference sensing model that fuses the attention mechanism, and based on this user preference sensing model, build a user preference-based distribution estimation probability model, generate a new executable solution item that includes user preferences, and calculate the adaptive value. Select the highest multiple items as the final search result. The method can well handle the personalized search task for multi-source heterogeneous user-generated content in big data environment, effectively guide users to perform personalized searches, and help users search for satisfactory solutions as soon as possible. improve the overall performance of personalized search algorithms.

FIG. 1 is a flowchart of the personalized search method integrated with the attention mechanism disclosed in the present invention. Figure 2 is a schematic structural diagram of a user preference sensing model that combines attention mechanisms. FIG. 3 is a schematic diagram of the configuration of a personalized search system that combines an attention mechanism.

The invention will now be further described with reference to the drawings and specific embodiments.

As shown in FIG. 1, the present invention discloses a personalized search method integrated with an attention mechanism, which includes the following steps 1-8.

In step 1, user-generated content is collected and obtained, the user-generated content includes all items rated by user u, scores and text comments for each item, images of each item, and other information for user u's ratings. Contains the user's usefulness evaluation score, vectorizes the text comment, performs feature extraction on the item image, and obtains the eigenvector.
In this embodiment, the step of vectorized representation of text comments involves removing stop words, punctuation marks, etc. in text comments, performing data preprocessing, and performing data preprocessing as described in the literature Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [J]. arXiv:1810.04805v2 [cs. CL] 24 May 2019. The first step is to perform a vectorized representation on user text comments using the BERT model.

Feature extraction of item images is described in the literature Krizhevsky A, Sutskever I, Hinton G E. Image Net classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada, USA: Curran Associates Inc. , 2012. 1097-1105. Feature extraction and vectorization expression are performed on item images using the AlexNet model.

Another user's usefulness evaluation of user u's evaluation means that other users evaluate the usefulness of user u's current evaluation information for a certain item, and when it is determined that it is useful, 1 If not, mark it with 0, and calculate the ratings that all other users give to the current user u's rating information for a certain item, and the total number of items marked with 1 is the user u's rating. This is the usefulness rating score of other users. For example, current user u makes a rating for item x, user A and user B give a usefulness rating for the rating, and the rating reflects the reliability of the current user's rating for item x. , invalid ratings or fake comments can be filtered by statistics on the usefulness ratings that all other users give to the current user u's ratings for item x.

The ratio of the usefulness evaluation score of other users to the user u's evaluation and the total number of items evaluated by the user u is the reliability of the user u's evaluation of the item.

In step 2, construct a superior item group D of user preferences,
An item whose user score is greater than a predetermined score threshold and whose reliability is greater than a predetermined reliability threshold is a user preference item. Since user preferences have the characteristics of ambiguity, uncertainty and dynamic change, this embodiment expands the user selection range by introducing a certain randomness into the existing user preference item groups, thereby User choices are not unduly restricted within current preference information and can adapt to dynamic changes in the actual environment and user preferences. As a result, a dominant item group D is composed of items whose scores are larger than a predetermined score threshold and whose reliability is larger than a predetermined reliability threshold, and a plurality of new items randomly sampled in the search space. New items added to the superior item group D may or may not include user preferences, and are random, increasing the diversity of the item group. The ratio of new items to the dominant item group D is 30% or less, and in this embodiment, the new items account for 10% of the total number of items in the dominant item group D.

Since the new item is randomly sampled in the search space, the current user u may or may not have rated it. If the current user u has not rated the new item, the text comment made by the current user u's similar user u' to the new item is used as the user u's rating for the new item, and If a plurality of similar users have all evaluated the new item, the evaluation of the user with the greatest degree of similarity to user u is selected. If none of the users similar to the current user u have rated the new item, user u's evaluation of the new item uses a method of randomly assigning values.

A user similar to user u is a user who has a common score item with user u and whose degree of similarity is greater than a predetermined similarity threshold. For user u' who has the same score item as user u, u'≠u, and the similarity Sim(u, u') between u and u' is
and
here,
represents the item set that both users u and u' evaluated,
teeth
is user u's score for item x' in
is the score of user u' for x',
is the average score of all items rated by user u,
is the average score of all items rated by user u'.

Dominant item group D constitutes set S, S={(u, x _i , C _i , T _i , G _i )}, x _i ∈D, and C _i is the category label vector of item x _i , the length is the total number of categories n _i , each element c _ij in C _i is a binary variable, c _ij =1 represents that the item x _i has j types of labels, and j = 1 , ₂ , . T _i is a vectorized representation of the user's text comment for item x _i and has length n ₂ ; G _i is a vectorized representation of the image features of item x _i and has length n ₃ ; i=1, 2, L, |D|, where |D| represents the number of items in D.

Vectors C _i , T _i , G _i are combined to form a vector Ψ _i of length Φ and constitute the original decision vector of item x _i , each element Ψ _in of which is a decision component of item x _i . , Φ=n ₁ +n ₂ +n ₃ , n=1, 2, . . . , Φ.

In step 3, a user preference sensing model that combines the attention mechanism is constructed, and as shown in Figure 2, the model is a three-layer Restricted Boltzmann machine based on a deep belief network (DBN). Machine, RBM), and the visible layer of the first layer restricted Boltzmann machine RBM1 includes a first set of visible units v ₁ , a second set of visible units v ₂ and a third set of visible units v ₃ , The hidden layer is h ₁ , the first set of visible units v ₁ has n ₁ units, each unit is a binary variable, and the second and third sets of visible units v ₂ and v ₃ have n ₂ and n ₃ units, respectively, each unit is a real variable, h ₁ constitutes the second layer restricted Boltzmann machine RBM2 together with the hidden layer h ₂ as the visible layer, and h ₂ constitutes the third layer restricted Boltzmann machine RBM3 together with the hidden layer _h3 as a visible layer. h ₁ , h ₂ and h ₃ each have M ₁ , M ₂ and M ₃ hidden units, each hidden unit is a real variable, and for each RBM, the number of hidden units is equal to the number of visible units. It is selected from 0.8 to 1.2 times the total number, and in this embodiment, it is set to 0.8 times. Thereby, the number M ₁ of hidden units in h ₁ is M ₁ =┌0.8*Φ┐, Φn ₁ +n ₂ +n ₃ , ┌g┐ is the round-up operation, and the hidden units in h ₂ The number M ₂ of hidden units in h 3 is M ₂ =┌0.8*M ₁ ┐, and the number M ₃ of hidden units in h ₃ is M ₃ =┌0.8*M ₂ ┐. The parameters of the user preference sensing model that combines the attention mechanism are θ={θ ₁ , θ ₂ , θ ₃ }={w ₁ , a ₁ , b ₁ , w ₂ , a ₂ , b ₂ , w ₃ , a ₃ , b ₃ }, where {w ₁ , a ₁ , b ₁ }, {w ₂ , a ₂ , b ₂ } and {w ₃ , a ₃ , b ₃ } are RBM1, RBM2, and RBM3, respectively. represents the model parameters, w _τ represents the connection weight between the τ-th RBM visible unit and hidden unit, a _τ and b _τ represent the bias of the τ-th RBM visible unit and hidden unit, respectively, and τ ∈{1, 2, 3}.

Using the dominant item group D, train the first-layer restricted Boltzmann machine RBM1 in the user preference sensing model that combines the attention mechanism with the contrastive divergence learning algorithm, and calculate its model parameters θ ₁ = {w ₁ , a ₁ , b ₁ }. In this step, training only RBM1 may be considered as pre-training of RBM1, and in subsequent steps, RBM1, RBM2, RBM3 are trained layer by layer again. The decision vector Ψ _i of item x ₁ is a combination of C _i , T _i , and G _i , and the user preference information contained in C _i , T _i , and G _i is different, for example, the length of the category label vector C _i s n _i is usually the length _n of the vectorized representation G _i of the item's image feature. If each component in the item's decision vector is treated equally, data containing a large amount of information will be compared to data containing a small amount of preference information. Data containing such little preference information is a useful addition to the construction of a user preference sensing model and cannot be ignored. Therefore, the present invention utilizes weights to adjust the components of various multi-source heterogeneous data input to the neural unit of the visible layer of the user preference sensing model, in combination with the information entropy represented by each data type. ensuring that different types of data can effectively contribute to building user preference sensing models.

After the training of the first layer RBM model is completed, given the state of hidden units, the activation state of each visible unit is conditionally independent, and the vector representation of some item x ₁ [C _i , T _i , G _i ] is input to the visible layer, and the activation probabilities of the first set, second set, and third set of visible units are respectively
, where a _1,j , a _1,k and a _1,l represent the biases of the first, second and third sets of visible units, respectively, and a _1,j , a _1,k and a _1,l are combined to form a ₁ , j=1,2,..., _n1 ,k=1,2,..., _n2 ,l=1,2,..., It is _n3 .

Information entropy formula
Compute the information entropy of various multi-source heterogeneous data according to
The information entropy of the item category label is
and
The information entropy of the text comment vector is
and
The information entropy of the item image feature vector is
and
Here, c _ij represents the j-th element of the category label vector C _i of item x _i , and p ₍ c _ij) is the activation of the visible unit corresponding to the j-th element of the vector representation of the item category label in RBM1. represents the probability,
t _ik represents the kth element of the vectorized representation T _i of user u's text comment for item x _i , and p ₍ t _ik) is the visible vector representation T i of the vectorized representation of user u's text comment in RBM1. represents the activation probability of the unit,
g _il represents the lth element of the vectorized representation G _i of the image features of item x _i , and p _(gil) is the activation of the visible unit corresponding to the lth element of the vectorized representation of the item image features in RBM1. represents the probability,
Next, further calculate the ratio of various information entropies to the total information entropy as a weighting factor,
Here, H(x _i )=H(C _i )+H(T _i )+H(G _i ),
Given the states of the visible units, i.e., combining the vectors C _i , T _i , G _i to form the decision vector Ψ _i of item x _i to input each visible unit in v ₁ , v ₂ , v ₃ , The activation state of each hidden unit in the hidden layer _h1 is conditionally independent, and the activation probability of the _m1th hidden unit is
and
Here, m ₁ =1, 2,..., M ₁ ,
is the bias of the m _-th hidden unit in h ₁ , and v _1j is the state of the j-th visible unit in the first set of visible units v ₁ of RMB1, i.e., the value of the j-th element of C _i . , v _2k is the state of the k-th visible unit in the second set of visible units v ₂ of RMB1, i.e. the value of the k-th element of T _i , and v _3l is the state of the k-th visible unit in the second set of visible units v 2 of RMB1; v is the state of the l-th visible unit at ₃ , i.e. the value of the l-th element of G _i ,
is the element value in w ₁ and represents the connection weight between the nth visible unit and m _1st hidden unit in RBM1, where n = 1, 2, ..., Φ,
represents the state of the m _- th hidden unit in the hidden layer _h1 ,
is the sigmoid activation function.

Given the state of the hidden unit, the activation state of each visible unit is also conditionally independent, and the activation probability of the nth visible unit is
and
Here, a _1,n represents the bias of the nth visible unit in the visible layer.

After the training of RBM1 is completed, the state of each hidden unit corresponding to _item The activation probability of a unit can be obtained as an attention weighting coefficient at _n (x _i ),
here,
When Ψ _i is the state of each visible unit in the visible layer of RBM1, represents the state of the m _i -th hidden unit in the hidden layer h _i , and at _n (x _i ) is each determinant component ψ _in of item x _i represents the attentional weight of , and reflects the characteristics of self-adaptation.

Encoding is performed on the item x _i in the dominant item group D based on the attention mechanism using the attention weight coefficient at _n (x _i ) as the weight coefficient of each determining component of the item x _i , and after encoding, it is expressed as x _ati ,
Here, i=1, 2, L, |D|,
x _ati is input to RBM1 after pre-training to obtain the activation probability V _RBM1 (x _ati ) of the visible unit,

here,
is the n'th element of x _ati .

Equation (9) actually nests the activation probabilities of hidden units and visible units, i.e.,
and
The activation probability V _RBM1 (x _ati ) of the visible unit in the obtained RBM1 model and the literature Li J, Wang Y, Mcauley J. Time Interval Aware Self-Attention for Sequential Recommendation. In: WSDM '20: The Thirteenth ACM International Conference on Web Search and Data Mining. ACM, 2020. Using _{the self -} attention mechanism provided in learn from
Here, the softmax() function ensures that the sum of all weighting factors is one. The function a(V _RBM1 (x _ati ), w ₁ ) measures the attention weighting coefficient of item x _i with respect to user preference features, and the calculation is
and
Combining the user preference attention weight vector A(x _ati ) and the original decision vector C _i , T _i , G _i of item x _i to generate an item decision vector that fuses the attention mechanism;
Item decision vector fused with attention mechanism
Construct a training set, and train RBM1, RBM2, and RBM3 models in DBN layer by layer. First, train RBM1, obtain parameters {w ₁ , a ₁ , b ₁ }, and set b ₁ to a in RBM2. ₂ , train RBM2, obtain the optimization parameters {w ₂ , a ₂ , b ₂ }, transfer b ₂ to a ₃ in RBM3, train RBM3, and obtain the optimization parameters { w ₃ , a ₃ , b ₃ }, thereby realizing the mutual influence and mutual association of the three-layer RBM model in the DBN network to form the entire network. After the training is completed, the DBN-based user preference sensing model fused with the attention mechanism and its optimization model parameters θ are obtained.

The DBN model training method here is an improved attention mechanism-based DBN model training method, which uses self-adaptive weight information to extract user preference features, focus attention on important features, and improve practical application. The purpose is to more appropriately represent the influence of different types of attribute determining components of each item in a scene on user preference features, and to express user preference features in more detail.

In step 4, a user preference-based distribution estimation probability model P(x) is constructed based on the DBN-based user preference sensing model that combines the trained attention mechanism and its model parameters;
P(x)=[P(ψ ₁ ), P(ψ ₂ ), ..., P(ψ _n ), ..., P(ψ _Φ )] (17)
Here, (ψ ₁ , ψ ₂ , ..., ψ _n , ..., ψ _Φ ) is the original decision vector of item x, and P (ψ _n ) is the user's decision vector for the nth decision component of item represents the preference probability of, and its calculation is,
First, a user preference-based probability distribution model P(x) is calculated based on the dominant item group D,
Here, p(x) is a Φ-dimensional vector, its nth element P(ψ _n ) is the activation probability of the nth determining component of the user preference item, and a lower bound constraint is applied to P(ψ _n ). , and the value after the constraint is the user's preference probability P(ψ _n ) for the nth determining component of the item, that is,
and
Here, ε is a predetermined lower bound threshold, and in this example, ε=0.1, that is, for a determining component whose activation probability calculated according to equation (18) is less than 0.1, The activation probability value is set to 0.1, and the constraint is that when the activation probability of a determining component is small, the determined component is randomly sampled with a certain probability value. Increase the diversity of the group and prevent the optimal solution from being missed by early convergence of the evolutionary optimization algorithm.

In step 5, the population size N is set, and N new individuals are generated using a distribution estimation algorithm (EDA) using the user preference-based distribution estimation probability model P(x). , each individual is one item, and the category label vector of the vth new individual is
The setting step for (v=1, 2,..., N) is
(5.1) Let v=1,
(5.2) Generate a random number z of [0, 1], and if z≦P (ψ _j =1), the category label vector of the vth new individual
The jth element of is 1, otherwise 0,
(5.3) Add 1 to v and repeat step (5.2) until v>N.

In step 6, category label vectors of N new individuals are added in the search space.
In this example, the Euclidean distance ^between two vectors is used to calculate the similarity. The smaller the distance, the higher the degree of similarity between the two.

In step 7, the adaptation value of each item in the recommendation target item set S ^u is calculated,
In the present invention, the adaptive value of the item is calculated using the energy function, and the adaptive value of the item x ^* in the recommendation target item set S ^u is
The calculation of
and
here,
as well as
respectively represent the maximum value and minimum value of the energy function of the item in the recommendation target item set S ^u ,
is the energy function (x ^* ∈ S ^u ) of item x ^* , and its calculation is
and
Here, a _1,n represents the bias of the nth visible unit in the visible layer of RBM1,
is the nth determining component of item x ^* ,
is the bias of the m _- th hidden unit in h ₁ ,
is the element value in w ₁ and represents the connection weight between the nth visible unit m and the _m1th hidden unit in RBM1.

In step 8, TopN items with the highest adaptation values in S ^u are selected as the search results, and TopN<N.

Due to the dynamic evolution characteristics of multi-source heterogeneous user-generated content and the uncertainty of user tastes and preferences, the user preference information contained in the dominant item group D is insufficient at the early stage of the personalized evolutionary search process, and therefore, The user preference features extracted based on this trained user preference sensing model are rough. Along with this, along with the promotion of the user's interactive search process and the dynamic evolution of user behavior, the dominant item group D is updated according to the latest evaluation data of the current user, and a user preference sensing model that combines an attention mechanism is developed. By re-training and dynamically updating the extracted user preference features to track changes in user preferences in a timely manner, and at the same time updating the user preference-based distribution estimation probability model P(x), personalized evolution can be achieved. It effectively guides the search direction, helps users search for satisfactory solutions as quickly as possible, and smoothly completes personalized search tasks in complex environments.

This embodiment further discloses a personalized search system that combines an attention mechanism to realize the above-mentioned personalized search method, and as shown in FIG. 3,
Collect and retrieve user u-generated content, including all items rated by user u, scores and text comments for each item, images of each item, and other users' usefulness evaluation scores for user u's ratings, and a user-generated content acquisition module 1 that vectorizes comments, performs feature extraction on item images, and acquires eigenvectors;
a superior item group construction module 2 that configures a superior item group D including user preferences with items whose user score is greater than a predetermined score threshold and whose reliability is greater than a predetermined reliability threshold;
A user preference sensing model building and training module for building and training a user preference sensing model fused with an attention mechanism according to step 3, the model comprising three layers of restricted Boltzmann machines based on a deep belief network; The visible layer of the Boltzmann machine with limited layers includes a first set of visible units v ₁ , a second set of visible units v ₂ and a third set of visible units v ₃ , and the hidden layers are h ₁ and h ₁ constitutes _a second-layer restricted Boltzmann machine with hidden layer _h2 as a visible layer, and constitutes a third-layer restricted Boltzmann machine with hidden layer _h3 as a visible layer, and the above attention mechanism is The parameters of the fused user preference sensing model are θ={θ ₁ , θ ₂ , θ ₃ }={w ₁ , a ₁ , b ₁ , w ₂ , a ₂ , b ₂ , w ₃ , a ₃ , b ₃ } User preference sensing model construction training module 3, which is
Build a user preference-based distribution estimation probability model P(x) based on a deep belief network-based user preference sensing model that combines a trained attention mechanism and its model parameters,
P(x)=[P(ψ ₁ ), P(ψ ₂ ), ..., P(ψ _n ), ..., P(ψ _Φ )] (17)
Here, (ψ ₁ , ψ ₂ , ..., ψ _n , ..., ψ _Φ ) is the original decision vector of item x, and P (ψ _n ) is the user's decision vector for the nth decision component of item a user preference-based distribution estimation probability model construction module 4 representing preference probabilities;
Using the user preference-based distribution estimation probability model P(x), a distribution estimation algorithm is adopted to generate N (N is the size of a given population) new individuals, each of which is one item. and a population generation module 5 that sets a category label vector for each new individual;
Category label vector of N new individuals in search space
a recommendation target item set construction module 6 that selects N items having the highest degree of similarity to constitute a recommendation target item set S ^u ;
an adaptation value calculation module 7 that calculates the adaptation value of each item in the recommendation target item set S ^u according to step 7;
A search result selection module 8 that selects TopN items with the highest adaptation value in ^Su as search results, and where TopN<N.

(Additional note)
(Additional note 1)
A personalized search method that combines an attention mechanism,
Collect and retrieve user u-generated content, including all items rated by user u, scores and text comments for each item, images of each item, and other users' usefulness evaluation scores for user u's ratings, and Step 1 of vectorizing comments, extracting features from item images, and obtaining eigenvectors;
Step 2 of configuring a dominant item group D including user preferences with items having a user score larger than a predetermined score threshold and a reliability larger than a predetermined confidence threshold, the items in D forming a set S; , S={(u, x _i , C _i , T _i , G _i )}, where x _i ∈D, C _i is the category label vector of item x _i , and T _i is the item is a vectorized representation of the user's text comment for x _i , G _i is a vectorized representation of the image features of item x _i , i = 1, 2, ..., |D|, and |D| Step 2 representing the number of items in D;
A user preference sensing model that combines attention mechanisms is constructed, and the model is composed of three layers of restricted Boltzmann machines based on a deep belief network, and the visible layer of the first layer of restricted Boltzmann machines is the first set of visible layers. It includes a unit v ₁ , a second set of visible units v ₂ and a third set of visible units v ₃ , the hidden layer is h ₁ , and h ₁ is the visible layer with the restriction of the second layer together with the hidden layer h ₂ A Boltzmann machine is constructed, _h2 is a visible layer, and a hidden layer _h3 constitutes a third layer restricted Boltzmann machine, and the parameters of the user preference sensing model that combines the above attention mechanism are θ={θ ₁ , θ ₂ , θ ₃ }={w ₁ , a ₁ , b ₁ , w ₂ , a ₂ , b ₂ , w ₃ , a ₃ , b ₃ },
Using the dominant item group D, train the first-layer restricted Boltzmann machine in the user preference sensing model that combines the attention mechanism with the contrastive divergence learning algorithm, and set the model parameters θ1={w ₁ , a ₁ , b ₁ },
After the training of the first layer RBM model is completed, given the state of hidden units, the activation state of each visible unit is conditionally independent, and the vector representation of some item xi [C _i , T _i , G _i ] is input to the visible layer, and the activation probabilities of the first, second, and third visible units are respectively
and
Here, a ₁ , _j , a ₁ , _k and a ₁ , _l represent the biases of the first set, second set and third set of visible units, respectively;
The information entropy of various multi-source heterogeneous data is calculated, and the information entropy of item category label is
and
The information entropy of the text comment vector is
and
The information entropy of the item image feature vector is
and
Here, c _ij represents the j-th element of the category label vector C _i of item x _i , and p _(cij) is the activation probability of the visible unit corresponding to the j-th element of the vector representation of the item category label in RBM1. represents,
t _ik represents the kth element of the vectorized representation T _i of user u's text comment for item x _i , and p(t _ik ) is the visible vector representation T i of the vectorized representation of user u's text comment in RBM1. represents the activation probability of the unit,
g _il represents the lth element of the vectorized representation G _i of the image features of item x _i , and p _(gil) is the activation of the visible unit corresponding to the lth element of the vectorized representation of the item image features in RBM1. represents the probability,
Next, calculate the ratio of various information entropies to the total information entropy as a weighting factor,
Here, H(x _i )=H(C _i )+H(T _i )+H(G _i ),
When vectors C _i , T _i , G _i are combined to form the decision vector Ψ _i of item x _i and inputted to each visible unit in v ₁ , v ₂ , v ₃ , the activation of each hidden unit in hidden layer h ₁ is The activation states are conditionally independent, and the activation probability of the m _- th hidden unit is
and
Here, m ₁ =1, 2,...,M1,
is the bias of the m _-th hidden unit in h ₁ , v _1j is the state of the j-th visible unit in the first set of visible units v ₁ of RMB1, and v _2k is the state of the j-th visible unit in the second set of visible units of RMB1. v ₂ is the state of the k-th visible unit, v _3l is the state of the l-th visible unit in the third set of visible units v ₃ of RMB1,
is the element value in w ₁ and represents the connection weight between the nth visible unit and m _1st hidden unit in RBM1, where n = 1, 2, ..., Φ,
represents the state of the m _- th hidden unit in the hidden layer _h1 ,
is the sigmoid activation function,
After the training of RBM1 is completed, the state of each hidden unit corresponding to item x _i is obtained according to equation (9), and the user's preference degree for each determining component of each item in dominant item group D, that is, the state of the visible layer Obtain the activation probability of the unit as an attention weighting coefficient at _n (x _i ),
here,
If Ψ _i is the state of each visible unit in the visible layer of RBM1, represents the state of the m _i -th hidden unit in the hidden layer h _i , and at _n (x _i ) is each determinant component ψ _in of item x _i represents the attention weight of
Encoding is performed on the item x _i in the dominant item group D based on the attention mechanism using the attention weight coefficient at _n (x _i ) as the weight coefficient of each determining component of the item x _i , and after encoding, it is expressed as x _ati ,
x _ati is input to RBM1 after pre-training to obtain the activation probability V _RBM1 (x _ati ) of the visible unit,
here,
is the n'th element of x _ati ,
Perform a self-attention mechanism calculation using the activation probability V _RBM1 (x _ati ) of the visible unit of RBM1, dynamically learn the user preference attention weight vector A (x _ati ) for each item,
Here, the softmax() function ensures that the sum of all weighting coefficients is 1, and the function a(V _RBM1 (x _ati ), w ₁ ) measures the attention weighting coefficient of item x _i with respect to user preference features. And the calculation is
and
Generating an item decision vector that combines the attention mechanism by combining the user preference attention weight vector A(x _ati ) and the original decision vectors C _i , T _i , G _i of item x _i ;
Item decision vector fused with attention mechanism
Construct a training set, train the RBM1, RBM2, and RBM3 models in DBN layer by layer, and after the training is completed, create a deep belief network-based user preference sensing model that integrates the attention mechanism and its optimized model parameters θ. Step 3 to obtain
Build a user preference-based distribution estimation probability model P(x) based on a deep belief network-based user preference sensing model that combines a trained attention mechanism and its model parameters,
P(x)=[P(ψ ₁ ), P(ψ ₂ ), ..., P(ψ _n ), ..., P(ψ _Φ )] (17)
Here, (ψ ₁ , ψ ₂ , ..., ψ _n , ..., ψ _Φ ) is the original decision vector of item x, and P (ψ _n ) is the user's decision vector for the nth decision component of item Step 4 representing the preference probability of
Set the population size N, and use the user preference-based distribution estimation probability model P(x) to generate N new individuals with the distribution estimation algorithm, each individual being one item, and the v-th Category label vector of new individual
The setting step for (v=1, 2,...,N) is
(5.1) Let v=1,
(5.2) Generate a random number z of [0, 1], and if z≦P (ψ _j =1), the category label vector of the vth new individual
The jth element of is 1, otherwise 0,
(5.3) Step 5, which is to add 1 to v and repeat step (5.2) until v>N;
Category label vector of N new individuals in search space
a step 6 of selecting N items having the highest degree of similarity to constitute a recommendation target item set S ^u ;
Adaptation value of each item in the recommendation target item set S ^u
Calculate,
here,
as well as
respectively represent the maximum value and minimum value of the energy function of the item in the recommendation target item set S ^u ,
is the energy function of item x ^* , x ^* ∈S ^u , and its calculation is
and
here,
is the nth determining component of item x ^* , step 7;
Selecting the top N items with the highest adaptation value in S ^u as the search results, TopN<N, step 8;
With the promotion of users' interactive search process and the dynamic evolution of user behavior, we will update the dominant item group D according to the latest evaluation data of current users, and retrain the user preference sensing model that combines the attention mechanism. , dynamically updating the extracted user preference features and simultaneously updating the user preference-based distribution estimation probability model P(x);
A personalized search method that combines an attention mechanism characterized by:

(Additional note 2)
The superior item group D further includes new items with a ratio of η, and the new items are obtained by randomly sampling in the search space.
A personalized search method that combines the attention mechanism described in Appendix 1, characterized in that:

(Appendix 3)
If the current user u has not rated the new item, the text comment made by the current user u's similar user u' to the new item is used as user u's rating for the new item, and If multiple similar users have all rated the new item, select the rating of the user with the highest degree of similarity to user u, and if none of the current user u's similar users have rated the new item. , user u's evaluation of the new item uses a method of randomly assigning values;
A personalized search method that combines the attention mechanism described in Appendix 2, characterized in that:

(Additional note 4)
A similar user to user u is a user who has a common score item with user u and whose degree of similarity is greater than a predetermined similarity threshold, and for user u' who has a common score item with user u, u'≠ u, and the similarity Sim(u, u') between u and u' is
and
here,
represents the item set that both users u and u' evaluated,
teeth
is user u's score for item x' in
is the score of user u' for x',
is the average score of all items rated by user u,
is the average score of all items rated by user u',
A personalized search method that combines the attention mechanism described in Appendix 3, characterized in that:

(Appendix 5)
Specifically, the step of training the RBM1, RBM2, and RBM3 models in the DBN layer by layer includes:
First, train RBM1, obtain the parameters {w ₁ , a ₁ , b ₁ }, transfer b ₁ to a ₂ in RBM2, train RBM2, and obtain the optimization parameters {w ₂ , a ₂ , b ₂ }, transmit b ₂ to a ₃ in RBM3, train RBM3, and obtain optimization parameters {w ₃ , a ₃ , b ₃ }.
A personalized search method that combines the attention mechanism described in Appendix 1, characterized in that:

(Appendix 6)
The calculation of the user's preference probability P(ψ _n ) for the nth determining component of the item is
First, a user preference-based probability distribution model p(x) is calculated based on the dominant item group D,
p(x) is a Φ-dimensional vector, its nth element P(ψ _n ) is the activation probability of the nth determining component of the user preference item, and a lower bound constraint is applied to P(ψ _n ). , the value after the constraint is the user's preference probability P(ψ _n ) for the nth determining component of the item, i.e.
and
ε is a predetermined lower bound threshold;
A personalized search method that combines the attention mechanism described in Appendix 1, characterized in that:

(Appendix 7)
In the three-layer restricted Boltzmann machine, the number of hidden units in the hidden layer in each layer of the restricted Boltzmann machine is 0.8 to 1.2 times the number of visible units in the visible layer.
A personalized search method that combines the attention mechanism described in Appendix 1, characterized in that:

(Appendix 8)
The ratio of new items to superior item group D is η<30%,
A personalized search method that combines the attention mechanism described in Appendix 2, characterized by the following.

(Appendix 9)
The step 6 calculates the similarity using Euclidean distance, that is, the smaller the Euclidean distance between two vectors, the higher the similarity between them.
A personalized search method that combines the attention mechanism described in Appendix 1, characterized in that:

(Appendix 10)
A personalized search system that combines an attention mechanism,
Collect and retrieve user u-generated content, including all items rated by user u, scores and text comments for each item, images of each item, and other users' usefulness evaluation scores for user u's ratings, and a user-generated content acquisition module that vectorizes comments, extracts features from item images, and acquires eigenvectors;
a superior item group construction module that configures a superior item group D including user preferences with items whose user score is greater than a predetermined score threshold and whose reliability is greater than a predetermined reliability threshold;
A user preference sensing model building and training module that builds and trains a user preference sensing model that integrates an attention mechanism, the model is composed of a three-layer restricted Boltzmann machine based on a deep belief network, and the first layer The visible layer of the restricted Boltzmann machine includes a first set of visible units v ₁ , a second set of visible units v ₂ and a third set of visible units v ₃ , the hidden layer is h ₁ , and h ₁ is the visible layer. , a second-layer restricted Boltzmann machine is constructed with the hidden layer _h2 , _h2 is the visible layer, and a third-layer restricted Boltzmann machine is constructed with the hidden layer _h3 . The parameters of the preference sensing model are θ={θ ₁ , θ ₂ , θ ₃ }={w ₁ , a ₁ , b ₁ , w 2 , a ₂ , b _{2 ,} w ₃ , a ₃ , b ₃ } A preference sensing model construction training module;
Build a user preference-based distribution estimation probability model P(x) based on a deep belief network-based user preference sensing model that combines a trained attention mechanism and its model parameters,
P(x)=[P(ψ ₁ ), P(ψ ₂ ), ..., P(ψ _n ), ..., P(ψ _Φ )] (17)
Here, (ψ ₁ , ψ ₂ , ..., ψ _n , ..., ψ _Φ ) is the original decision vector of item x, and P (ψ _n ) is the user's decision vector for the nth decision component of item a user preference-based distribution estimation probability model construction module representing preference probabilities;
Using the user preference-based distribution estimation probability model P(x), a distribution estimation algorithm is adopted to generate N (N is the size of a given population) new individuals, each of which is one item. and a population generation module that sets a category label vector for each new individual;
N new individual category label vectors in the search space
a recommendation target item set construction module that selects N items having the highest degree of similarity to constitute a recommendation target item set S ^u ;
an adaptation value calculation module that calculates the adaptation value of each item in the recommendation target item set S ^u ;
A personalized search system integrating an attention mechanism, comprising: a search result selection module that selects TopN items with the highest adaptation value in ^Su as search results, and where TopN<N.

Claims (10)

A personalized search method that combines an attention mechanism,
Collect and retrieve user u-generated content, including all items rated by user u, scores and text comments for each item, images of each item, and other users' usefulness evaluation scores for user u's ratings, and Step 1 of vectorizing comments, extracting features from item images, and obtaining eigenvectors;
Step 2 of configuring a dominant item group D including user preferences with items having a user score larger than a predetermined score threshold and a reliability larger than a predetermined confidence threshold, the items in D forming a set S; , S={(u, x _i , C _i , T _i , G _i )}, where x _i ∈D, C _i is the category label vector of item x _i , and T _i is the item is a vectorized representation of the user's text comment for x _i , G _i is a vectorized representation of the image features of item x _i , i = 1, 2, ..., |D|, and |D| Step 2 representing the number of items in D;
A user preference sensing model that combines attention mechanisms is constructed, and the model is composed of three layers of restricted Boltzmann machines based on a deep belief network, and the visible layer of the first layer of restricted Boltzmann machines is the first set of visible layers. It includes a unit v ₁ , a second set of visible units v ₂ and a third set of visible units v ₃ , the hidden layer is h ₁ , and h ₁ is the visible layer with the restriction of the second layer together with the hidden layer h ₂ A Boltzmann machine is constructed, _h2 is a visible layer, and a hidden layer _h3 constitutes a third layer restricted Boltzmann machine, and the parameters of the user preference sensing model that combines the above attention mechanism are θ={θ ₁ , θ ₂ , θ ₃ }={w ₁ , a ₁ , b ₁ , w ₂ , a ₂ , b ₂ , w ₃ , a ₃ , b ₃ },
Using the dominant item group D, train the first-layer restricted Boltzmann machine in the user preference sensing model that combines the attention mechanism with the contrastive divergence learning algorithm, and calculate the model parameters θ ₁ = {w ₁ , a ₁ , b ₁ },
After the training of the first layer RBM model is completed, given the state of the hidden unit, the activation state of each visible unit is conditionally independent, and the vector representation of some item x _i [C _i , T _i , G _i ] is input to the visible layer, and the activation probabilities of the first set, second set, and third set of visible units are respectively
and
Here, a _1,j , a _1,k and a _1,l represent the biases of the first set, second set and third set of visible units, respectively;
The information entropy of various multi-source heterogeneous data is calculated, and the information entropy of item category label is
and
The information entropy of the text comment vector is
and
The information entropy of the item image feature vector is
and
Here, c _ij represents the j-th element of the category label vector C _i of item x _i , and p ₍ c _ij) is the activation of the visible unit corresponding to the j-th element of the vector representation of the item category label in RBM1. represents the probability,
t _ik represents the kth element of the vectorized representation T _i of user u's text comment for item x _i , and p ₍ t _ik) is the visible vector representation T i of the vectorized representation of user u's text comment in RBM1. represents the activation probability of the unit,
g _il represents the lth element of the vectorized representation G _i of the image features of item x _i , and p _(gil) is the activation of the visible unit corresponding to the lth element of the vectorized representation of the item image features in RBM1. represents the probability,
Next, calculate the ratio of various information entropies to the total information entropy as a weighting factor,
Here, H(x _i )=H(C _i )+H(T _i )+H(G _i ),
When vectors C _i , T _i , G _i are combined to form the decision vector Ψ _i of item x _i and inputted to each visible unit in v ₁ , v ₂ , v ₃ , the activation of each hidden unit in hidden layer h ₁ is The activation states are conditionally independent, and the activation probability of the m _- th hidden unit is
and
Here, m ₁ =1, 2,..., M ₁ ,
is the bias of the m _-th hidden unit in h ₁ , v _1j is the state of the j-th visible unit in the first set of visible units v ₁ of RMB1, and v _2k is the state of the j-th visible unit in the second set of visible units of RMB1. v ₂ is the state of the k-th visible unit, v _3l is the state of the l-th visible unit in the third set of visible units v ₃ of RMB1,
is the element value in w ₁ and represents the connection weight between the nth visible unit and m _1st hidden unit in RBM1, where n = 1, 2, ..., Φ,
represents the state of the m _- th hidden unit in the hidden layer _h1 ,
is the sigmoid activation function,
After the training of RBM1 is completed, the state of each hidden unit corresponding to item x _i is obtained according to equation (9), and the user's preference degree for each determining component of each item in dominant item group D, that is, the state of the visible layer Obtain the activation probability of the unit as an attention weighting coefficient at _n (x _i ),
here,
If Ψ _i is the state of each visible unit in the visible layer of RBM1, represents the state of the m _i -th hidden unit in the hidden layer h _i , and at _n (x _i ) is each determinant component ψ _in of item x _i represents the attention weight of
Encoding is performed on the item x _i in the dominant item group D based on the attention mechanism using the attention weight coefficient at _n (x _i ) as the weight coefficient of each determining component of the item x _i , and after encoding, it is expressed as x _ati ,
x _ati is input to RBM1 after pre-training to obtain the activation probability V _RBM1 (x _ati ) of the visible unit,
here,
is the n'th element of x _ati ,
Perform a self-attention mechanism calculation using the activation probability V _RBM1 (x _ati ) of the visible unit of RBM1, dynamically learn the user preference attention weight vector A (x _ati ) for each item,
Here, the softmax() function ensures that the sum of all weighting coefficients is 1, and the function a(V _RBM1 (x _ati ), w ₁ ) measures the attention weighting coefficient of item x _i with respect to user preference features. And the calculation is
and
Generating an item decision vector that combines the attention mechanism by combining the user preference attention weight vector A(x _ati ) and the original decision vectors C _i , T _i , G _i of item x _i ;
Item decision vector fused with attention mechanism
Construct a training set, train the RBM1, RBM2, and RBM3 models in DBN layer by layer, and after the training is completed, create a deep belief network-based user preference sensing model that integrates the attention mechanism and its optimized model parameters θ. Step 3 to obtain
Build a user preference-based distribution estimation probability model P(x) based on a deep belief network-based user preference sensing model that combines a trained attention mechanism and its model parameters,
P(x)=[P(ψ ₁ ), P(ψ ₂ ), ..., P(ψ _n ), ..., P(ψ _Φ )] (17)
Here, (ψ ₁ , ψ ₂ , ..., ψ _n , ..., ψ _Φ ) is the original decision vector of item x, and P (ψ _n ) is the user's decision vector for the nth decision component of item Step 4 representing the preference probability of
Set the population size N, and use the user preference-based distribution estimation probability model P(x) to generate N new individuals with the distribution estimation algorithm, each individual being one item, and the v-th Category label vector of new individual
The setting step for (v=1, 2,...,N) is
(5.1) Let v=1,
(5.2) Generate a random number z of [0, 1], and if z≦P (ψ _j =1), the category label vector of the vth new individual
The jth element of is 1, otherwise 0,
(5.3) Step 5, which is to add 1 to v and repeat step (5.2) until v>N;
Category label vector of N new individuals in search space
a step 6 of selecting N items having the highest degree of similarity to constitute a recommendation target item set S ^u ;
Adaptation value of each item in the recommendation target item set S ^u
Calculate,
here,
as well as
respectively represent the maximum value and minimum value of the energy function of the item in the recommendation target item set S ^u ,
is the energy function of item x ^* , x ^* ∈S ^u , and its calculation is
and
here,
is the nth determining component of item x ^* , step 7;
Selecting the top N items with the highest adaptation value in S ^u as the search results, TopN<N, step 8;
With the promotion of users' interactive search process and the dynamic evolution of user behavior, we will update the dominant item group D according to the latest evaluation data of current users, and retrain the user preference sensing model that combines the attention mechanism. , dynamically updating the extracted user preference features and simultaneously updating the user preference-based distribution estimation probability model P(x);
A personalized search method that combines an attention mechanism characterized by: The superior item group D further includes new items with a ratio of η, and the new items are obtained by randomly sampling in the search space.
A personalized search method combining the attention mechanism according to claim 1. If the current user u has not rated the new item, the text comment made by the current user u's similar user u' to the new item is used as user u's rating for the new item, and If multiple similar users have all rated the new item, select the rating of the user with the highest degree of similarity to user u, and if none of the current user u's similar users have rated the new item. , user u's evaluation of the new item uses a method of randomly assigning values;
A personalized search method combining the attention mechanism according to claim 2. A similar user to user u is a user who has a common score item with user u and whose degree of similarity is greater than a predetermined similarity threshold, and for user u' who has a common score item with user u, u'≠ u, and the similarity Sim(u, u') between u and u' is
and
here,
represents the item set that both users u and u' evaluated,
teeth
is user u's score for item x' in
is the score of user u' for x',
is the average score of all items rated by user u,
is the average score of all items rated by user u',
A personalized search method combining the attention mechanism according to claim 3. Specifically, the step of training the RBM1, RBM2, and RBM3 models in the DBN layer by layer includes:
First, train RBM1, obtain the parameters {w ₁ , a ₁ , b ₁ }, transfer b ₁ to a ₂ in RBM2, train RBM2, and obtain the optimization parameters {w ₂ , a ₂ , b ₂ }, transmit b ₂ to a ₃ in RBM3, train RBM3, and obtain optimization parameters {w ₃ , a ₃ , b ₃ }.
A personalized search method combining the attention mechanism according to claim 1. The calculation of the user's preference probability P(ψ _n ) for the nth determining component of the item is
First, a user preference-based probability distribution model p(x) is calculated based on the dominant item group D,
p(x) is a Φ-dimensional vector, and its nth element P(ψ _n ) is the activation probability of the nth determining component of the user preference item, and a lower bound constraint is applied to P(ψ _n ). , the value after the constraint is the user's preference probability P(ψ _n ) for the nth determining component of the item, i.e.
and
ε is a predetermined lower bound threshold;
A personalized search method combining the attention mechanism according to claim 1. In the three-layer restricted Boltzmann machine, the number of hidden units in the hidden layer in each layer of the restricted Boltzmann machine is 0.8 to 1.2 times the number of visible units in the visible layer.
A personalized search method combining the attention mechanism according to claim 1. The ratio of new items to superior item group D is η<30%,
A personalized search method combining the attention mechanism according to claim 2. The step 6 calculates the similarity using Euclidean distance, that is, the smaller the Euclidean distance between two vectors, the higher the similarity between them.
A personalized search method combining the attention mechanism according to claim 1. A personalized search system that combines an attention mechanism,
Collect and retrieve user u-generated content, including all items rated by user u, scores and text comments for each item, images of each item, and other users' usefulness evaluation scores for user u's ratings, and a user-generated content acquisition module that vectorizes comments, extracts features from item images, and acquires eigenvectors;
a superior item group construction module that configures a superior item group D including user preferences with items whose user score is greater than a predetermined score threshold and whose reliability is greater than a predetermined reliability threshold;
A user preference sensing model building and training module that builds and trains a user preference sensing model that integrates an attention mechanism, the model is composed of a three-layer restricted Boltzmann machine based on a deep belief network, and the first layer The visible layer of the restricted Boltzmann machine includes a first set of visible units v ₁ , a second set of visible units v ₂ and a third set of visible units v ₃ , the hidden layer is h ₁ , and h ₁ is the visible layer. , a second-layer restricted Boltzmann machine is constructed with the hidden layer _h2 , _h2 is the visible layer, and a third-layer restricted Boltzmann machine is constructed with the hidden layer _h3 . The parameters of the preference sensing model are θ={θ ₁ , θ ₂ , θ ₃ }={w ₁ , a ₁ , b ₁ , w 2 , a ₂ , b _{2 ,} w ₃ , a ₃ , b ₃ } A preference sensing model construction training module;
Build a user preference-based distribution estimation probability model P(x) based on a deep belief network-based user preference sensing model that combines a trained attention mechanism and its model parameters,
P(x)=[P(ψ ₁ ), P(ψ ₂ ), ..., P(ψ _n ), ..., P(ψ _Φ )] (17)
Here, (ψ ₁ , ψ ₂ , ..., ψ _n , ..., ψ _Φ ) is the original decision vector of item x, and P (ψ _n ) is the user's decision vector for the nth decision component of item a user preference-based distribution estimation probability model construction module representing preference probabilities;
Using the user preference-based distribution estimation probability model P(x), a distribution estimation algorithm is adopted to generate N (N is the size of a given population) new individuals, each of which is one item. and a population generation module that sets a category label vector for each new individual;
N new individual category label vectors in the search space
a recommendation target item set construction module that selects N items having the highest degree of similarity to constitute a recommendation target item set S ^u ;
an adaptation value calculation module that calculates the adaptation value of each item in the recommendation target item set S ^u ;
A personalized search system integrating an attention mechanism, comprising: a search result selection module that selects TopN items with the highest adaptation value in ^Su as search results, and where TopN<N.

Images