CN114048386B - Deep learning-based personalized and diversified search method - Google Patents
Deep learning-based personalized and diversified search method Download PDFInfo
- Publication number
- CN114048386B CN114048386B CN202111391539.0A CN202111391539A CN114048386B CN 114048386 B CN114048386 B CN 114048386B CN 202111391539 A CN202111391539 A CN 202111391539A CN 114048386 B CN114048386 B CN 114048386B
- Authority
- CN
- China
- Prior art keywords
- query
- user
- document
- diversity
- personalized
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000013135 deep learning Methods 0.000 title claims abstract description 16
- 239000011159 matrix material Substances 0.000 claims abstract description 10
- 230000004931 aggregating effect Effects 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 26
- 238000012512 characterization method Methods 0.000 claims description 16
- 230000007774 longterm Effects 0.000 claims description 16
- 230000004913 activation Effects 0.000 claims description 8
- 238000013507 mapping Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- 238000012549 training Methods 0.000 claims description 4
- 230000004927 fusion Effects 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention realizes a search method based on deep learning and integrating individuation and diversification by a method in the intelligent search field. The method mainly comprises three steps of calculating weights of general diversity and personalized diversity, dynamically modeling diversity and aggregating score results, wherein a deep learning-based personalized and diversified search model (DFSPD model) is used as a core of the method, the diversity is divided into personalized diversity and general diversity, and influences of the personalization and the diversity on the results are considered at the same time, so that the results satisfying users are provided according to different conditions. The scheme has very high flexibility, the user description portrait generator, the word embedding matrix and the personalized weighting process are replaceable, and the accuracy is greatly improved.
Description
Technical Field
The invention relates to the technical field of network searching, in particular to a search method based on deep learning and integrating individuation and diversification.
Background
The ambiguity of search sentences is an important factor affecting the satisfaction degree of search engine results, and two main disambiguation methods are adopted, namely personalized search and diversification of search results. Personalized searches typically build a model that can learn information about the interests and preferences of a user from the user's search history to clarify the specific intent of the user in the current situation, thereby providing documents that meet the interests of the user. From early personalized learning based on artificial features, such as click features, word frequency features, etc., to learning related features of more complex abstraction based on various deep learning models, such as RNN, GAN, transformer, etc., personalized searching has been devoted to mining more accurate and fine-grained features of interest for users, however ignoring the diversity of results and the inherent diversity of user interests, which can lead to problems of redundancy of results, incomplete capture of user interests, etc., while not performing very well when users have no history data, so-called "cold start" problems. The diversity of search results is expected that the model can return a sufficiently diverse document set on the premise of ensuring the content correlation so as to meet different intentions of different users as much as possible, thereby achieving the purpose of disambiguation. Early diversification models typically designed a fixed formula that calculated diversity based on content similarity or sub-topic coverage, so that a locally optimal document was greedy selected to construct the returned results. The development of machine learning enabled diverse models to learn semantically based features to more rationally compute diversity. The limitation of diversification is that the specific intention of the user cannot be extracted accurately, so that an optimal document ordering result cannot be generated, and a large number of documents which are not interested by the current user are contained in the search result. The algorithm combining individuation and diversification can combine the advantages of the individuation and diversification to realize good and bad complementation, thereby providing more satisfactory search results for users. At present, research in the field is relatively deficient, most models are personalized and diversified by introducing personalized factors into a traditional diversification algorithm, and recently, static fusion work of personalization and diversification is also realized by a structured SVM model.
The existing personalized and diversified fusion work mainly has the following problems:
(1) In different situations, the emphasis of personalization and diversification should be different, for example, when a new user uses a search engine, or the user searches for a problem with great difference from history, so that the model cannot learn useful information from the search history, the emphasis should be more on diversification to improve the probability of user satisfaction, otherwise, the emphasis should be more on personalization, and most of the existing methods simply introduce a personalization factor of user information to improve the diversification model, or consider the integration of personalization and diversification statically, and do not dynamically consider the weight problem of the two.
(2) The interests of the users have inherent diversity, so that the individuation is focused, and meanwhile, the diversity of the user intention is also focused, so that incomplete capture of the user intention is prevented.
(3) The deep learning-based model can learn the characteristic information in high-order semantics, and most of the existing fusion models are based on manual characteristics or traditional machine learning methods, so that the accuracy is greatly improved.
Disclosure of Invention
For this purpose, the present inventionThe invention firstly provides a deep learning-based personalized and diversified search method, which inputs historical query data, current query and candidate document sets of a user, and is realized through three steps: step one: and based on HRNN model, according to long-term history H of user l Short term history H s Learning its initial long-term and short-term portrayal L 0 And S is 0 Wherein H is l ＝{{q 1 ,D 1 },…,{q n ,D n }, n represents the total number of queries H contained in the previous session s ＝{{q n+1 ,D n+1 },…,{q n+m ,D n+m }, m is the number of queries that have been searched in the current session, q n Representing the nth query presented by the user, D n For query q n The ad-hoc search engine returns an initial candidate document set, so that the matching degree of the query vector and the user description image is utilized, and div (d|q v ) Representing the general diversity of document d, div (d|S), div (d|L) representing the personalized diversity of the document, using r S ,r L As the weight of the personalized diversity, dynamically controlling the influence degree of the personalized diversity on the score result; step two: constructing RRNN model, selecting a local optimal document d of each step based on a current comprehensive score (d) for the documents in the residual candidate document set by using a greedy algorithm, then learning the influence of the document on the sub-topic representation of the long-short term description image and the query of the user according to the virtual sub-topic representation of the document, modeling the influence by using a reset gate structure, and updatingIs->Wherein->And after the sub topics covered by t selected documents are respectively forgotten, the vector characterization of the query statement, the short-term portrait of the user and the long-term portrait of the user is used for calculating the score of the next document selection.
Representing the originality of the query and user portrait vector characterizations. Finally, dynamic modeling of the diversity of the modeling documents is realized; and thirdly, aggregating each score result, and based on the score, namely the basis of reordering the candidate documents, thereby realizing a search result reordering algorithm integrating individuation and diversification. The document total score function is as follows:
score(d)＝P(d|q,D,S,u)＝P(d|q v ,S,L)＝
φ(div(d|q v ),r S div(d|S),r L div(d|L),rel(d,q))
wherein div (d|q) v ) Representing the general diversity of document d, div (d|S), div (d|L) representing the personalized diversity of the document, using r S ,r L As the weight of the personalized diversity to dynamically control the influence degree of the personalized diversity on the score, rel (d, q) represents the relevance score of the query and the document calculated based on the additional manual characteristics, phi is an aggregation function and is realized by using a multi-layer perceptron.
The specific calculation modes of the general diversity weight and the personalized diversity weight are as follows: learning word-embedding using word2vec techniques, and then weighting and summing resulting vector representations based on TF-IDF weights of the wordsLearning user-initiated long-short duration description image H 'using HRNN structure' 0 ,S' 0 ；
The HRNN structure firstly uses a first layer RNN to take all query records of users as input, and each session end is used for constructing interest characterization of the users in each session by a period node:
subscripts m and n denote the nth query record in the mth session, and subscript 1 denotes the first layer RNN structure, q m,n ,d m,n Representing the corresponding input query and average Guan Wendang, the short-term descriptive representation of the user may be represented as:where M represents the Mth current session, n M Representing a search that the user has conducted in the session;
the long-term descriptive portrait is constructed according to the whole history record which does not contain the current session record, firstly, each session characterization of the user history is taken as input, and a second-layer RNN structure is applied to learn the interest characterization of each period of the user:
where m represents the mth current session, n m Representing searches that have been conducted by the user in the session, and thereafter utilizing the current queryAnd user individual period interest characterization ∈ ->Similarity alpha of (a) m As the weight of each period interest, the weighted summation result is described as the long-term interest of the user:
introducing a convolutional neural network conv to learn a virtual sub-topic representation sequence of the conv from the original representation of the query:
wherein q is i Representing the ith sub-topic representation of the query, c represents the number of sub-topics the query contains. Similarly, sub-topic representations of documents and user long-short term descriptive images can be obtained: d, L 0 ,S 0 ：
d＝[d 1 ,d 2 …d c ]＝conv(d')
L 0 ＝[L 1 ,L 2 …L c ]＝conv(L' 0 )
S 0 ＝[S 1 ,S 2 …S c ]＝conv(S' 0 )
Furthermore, KNRM model was introduced and t was used q ＝[t q1 ,...,t qn ],t d ＝[t d1 ,...,t dm ]To represent word sequences of queries and documents, where t qi An i-th word embedding vector representing a query, t dj The j-th word embedded vector of the document is represented, n and M respectively represent the word numbers of the current query and the document, and KNRM constructs a transformation matrix M according to cosine similarity between each group of word pairs of the query and the document:
M ij ＝cos_similarity(t qi ,t dj )
and then the KNRM learns the multidimensional features on the transformation matrix by using k Gaussian kernels, and calculates the similarity of the query and the document by using a multi-layer perceptron, wherein the specific calculation steps are as follows:
F K (t q ,t d )＝φ(f 1 (M),...f o (M),...,f k (M))
wherein u is o ,σ o The mean value and standard deviation corresponding to the o-th Gaussian kernel are manually set hyper-parameters, and then the weight value can be expressed as follows:
the reset gate is constructed in the following manner:
wherein,the representation will->Spliced together, W q Is a learnable mapping parameter, will +.>Mapping to a real number, σ (·) represents the activation function.
The activation function is a tanh function.
The update function of the reset gate is:
the total score uses a learnable matrix W a The respective diversity and relevance scores are aggregated to obtain:
score(d)＝φ(s)＝tanh(s T W a )
where s represents a vector of sub-scores, phi represents a multi-layer perceptron using tanh as an activation function, s= [ div (d|q v ),r S div(d|S),r L div(d|L),rel(d,q)]Wherein q is v S, L isT superscript form omitting latest feature, d, q v S, L both represent the corresponding virtual sub-topic sequences, the scores of general diversity and personalized diversity are calculated along with KNRM structures:
div(d|q v )＝F K (d,q v )
div(d|S)＝F K (d,S)
div(d|L)＝F K (d,L)
rel(d,q)＝φ(f(d,q))
using f (d, q) to represent additional manual feature sequences for documents and queries, a multi-layer perceptron phi is used to fuse the individual manual features.
The loss function of the method is constructed using a binary class of loss functions:
where Q represents all queries in the training set, Q o Represent all pairs of samples corresponding to query q, y o Is a sample label whenRatio->Y is more suitable for the user demand o =1, otherwise 0, ++>Representation of model predicted +.>Ratio->Better probability, the pairs of samples are written in the form of a pair: (C, d) 1 ,d 2 ) When ∈0 is calculated by the following formula>
Finally, gradually optimizing the model by an AdamOptimezer optimizer.
The invention has the technical effects that:
a searching method is realized, and the method has the following characteristics:
(1) The personalized and diversified weights are dynamically learned according to the matching degree of the query proposed by the user and the user descriptive portrait.
(2) The diversity is divided into personalized diversity, which captures the inherent diversity of user interests, and general diversity, which deals with the problem that cold starts and history information are not useful.
(3) Virtual sub-topic characterizations of a user, a query sentence and a document which can be learned are constructed, so that the sub-topic characterizations with wider learning coverage are displayed in consideration of the diversity among the documents.
(4) The neural network is utilized to dynamically model the diversity of the documents, and personalized and diversified branches are constructed according to the long-short-term description portraits and query sentences of the users.
Drawings
FIG. 1 is a deep learning based fused personalized and diversified search model architecture;
FIG. 2 is a graphical representation of a reset gate structure
Detailed Description
The following is a preferred embodiment of the present invention and a technical solution of the present invention is further described with reference to the accompanying drawings, but the present invention is not limited to this embodiment.
The invention provides a deep learning-based personalized and diversified search method. The method is based on a deep learning-based search model integrating individuation and diversification, wherein a DFSPD (Dynamic Fusion Framework of Search Result Personalization and Diversification) model is taken as the core of the method, and the method divides the diversification into individuation diversification and general diversification to simultaneously consider the influence of individuation and diversification on the result. Specifically, long-term and short-term description images of a user are learned based on the HRNN model, personalized and diversified weights are learned by utilizing the matching degree of the query vector and the user description images, and the RRNN model is constructed to dynamically model the diversity of documents. Virtual sub-topics are introduced to display and consider sub-topic distribution, similarity of sub-topic sequences is calculated by using a KNRM-like structure, and finally each sub-score is fused by using a multi-layer perceptron.
Suppose that for a user u, its history data contains a long-term history H l And short term history H s The former includes the interaction behavior H in the previous session l ＝{{q 1 ,D 1 },…,{q n ,D n -where n represents the total number of queries contained in the previous session, the latter containing a series of queries and candidate documents H in the current session s ＝{{q n+1 ,D n+1 },…,{q n+m ,D n+m -m is the number of queries that have been searched in the current session. The DFSPD model is firstly based on the HRNN model according to the long-term history H of the user l Short term history H s Learning its initial long-term and short-term portrayal L 0 And S is 0 . When the user proposes the query q, the ad-hoc search engine returns an initial candidate document set D= { D 1 ,d 2 ,. } our model will select a locally optimal document d for each step based on the current composite score (d) greedy of the document * . After selecting a locally optimal document, a model learns the influence of the document on the sub-topic representation of the user long-short term description image and query according to the virtual sub-topic representation of the document, uses the reset gate structure designed by us to model the influence, and updatesIs->For convenience, we will sometimes follow +.>Reduced to q v S, L, we use +.>To emphasize the originality of the query and user portrait vector characterization, i.e., the state when no document is selected. The total score function for the document is as follows:
score(d)＝P(d|q,D,S,u)＝P(d|q v ,S,L)＝φ(div(d|q v ),r S div(d|S),r L div(d|L),rel(d,q))
wherein div (d|q) v ) Representing the general diversity of document d, div (d|S), div (d|L) representing the personalized diversity of the document, we use r S ,r L The degree of influence of the personalized diversity on the score is dynamically controlled as the weight of the personalized diversity. rel (d, q) represents the calculated relevance scores of queries and documents based on additional manual features. Phi is an aggregate function, we use a multi-layer perceptron (MLP) to implement. The model structure is shown in fig. 1.
The input of the model includes historical query data of the user, the current query and the candidate set of documents. The model mainly comprises the following steps:
1. weights for general diversity and personalized diversity are calculated.
2. Dynamic modeling of diversity.
3. The individual scoring results are aggregated.
Weights of general diversity and personalized diversity are calculated:
as we have previously said, personalization and diversification should be given different weights on a case-by-case basis, and more particularly, such weights are learned based on the similarity of the current query and the user descriptive representation. Meanwhile, a convolutional neural network is utilized to learn virtual sub-topic representation of user portraits, queries and documents, so that the matching degree between the user portraits, the queries and the documents is considered according to the sub-topics.
For modeling of documents and query sentences, we learn word embedding using word2vec techniques, then baseWeighted summation of the TF-IDF weights of words to obtain their vector representationsWe learn the user's initial long and short duration description image H ' using HRNN structure ' 0 ,S' 0 。
The idea of HRNN architecture is to construct a short-term descriptive image of a user from the user's search history using a hierarchical RNN model. Firstly, using a first layer RNN to take all query records of users as input, and constructing interest characterization of the users in each session by using each session end as a period node:
subscripts m and n denote the nth query record in the mth session, subscript 1 denotes the first tier RNN structure, q m,n ,d m,n Representing the corresponding input query and average documents. Let the current session be the Mth session in which the user has already performed n M The short-term descriptive representation of the user can be expressed as a sub-search:
the long-term descriptive representation is built from an overall history (excluding current session records), first, with each session representation of the user's history as input, a second-tier RNN structure is applied to learn the user's interest representations for each period:
let us say that the user has n in the mth session m And searching for a second time. Thereafter utilizing the current queryAnd user individual period interest characterization ∈ ->Similarity alpha of (a) m As the weight of each period interest, the weighted summation result is described as the long-term interest of the user:
to be able to obtain fine-grained sub-topics with wider coverage, we introduced a convolutional neural network conv to learn its virtual sub-topic representation sequence from the original representation of the query:
wherein q is i Representing the ith sub-topic representation of the query, c represents the number of sub-topics the query contains. Similarly, sub-topic representations of documents and user long-short term descriptive images can be obtained: d, L 0 ,S 0 。
To be able to learn finer grains from multiple dimensionsAnd S is 0 ,L 0 We introduced the KNRM model. KNRM is a common ad-hoc model, namely, document ordering is performed only according to content similarity, and individualization or diversification is not considered. The inputs of a traditional KNRM are queries and documents, both of which are represented by corresponding word embedded vector sequences. We use t q ＝[t q1 ,...,t qn ],t d ＝[t d1 ,...,t dm ]To represent word sequences and documents of queriesWherein t is qi An i-th word embedding vector representing a query, t dj The j-th word embedding vector representing the document, n and m representing the number of words of the current query and document, respectively. KNRM builds a transformation matrix M according to cosine similarity between each group of word pairs of the query and the document:
M ij ＝cos_similarity(t qi ,t dj )
the KNRM learns multidimensional features on a transformation matrix by using k Gaussian kernels, and then calculates similarity between query and document by using a multi-layer perceptron (MLP), wherein the method comprises the following steps of:
F K (t q ,t d )＝φ(f 1 (M),...f o (M),...,f k (M))
wherein u is o ,σ o The mean value and standard deviation corresponding to the o-th Gaussian kernel are manually set hyper-parameters. The idea similar to KNRM is applied to calculate the similarity of the sub-topic sequences, and the method can be used for obtaining:
dynamic modeling of diversity:
when the current local optimum document d is selected * Later, we hope that the query and user portrayal can forget about d * Related aspects, thereby focusing on d in the next document selection * The angle that has not been covered yet. Based on this assumption, we construct a Reset gate (Reset gate):
wherein,the representation will->Spliced together, W q Is a learnable mapping parameter, will +.>Mapped to a real number. Sigma (·) represents the activation function, here using the tanh function.
A graphical representation of this structure is shown in fig. 2.
The mere use of reset gates has several drawbacks: 1. the candidate documents usually have dozens of tanh functions, which are functions of mapping real numbers to (-1, 1), and the multiplication result of a plurality of numbers with absolute values smaller than 1 is a value with small absolute values, so that the vector loses expression capacity at a later stage, and gradient vanishing and other problems are caused. 2. After browsing a certain number of documents, the user may forget what was previously seen and may therefore want to click on a document similar to the previously browsed document. Inspired by resnet, our update function is as follows:
the structure not only can well solve the problem of gradient disappearance, but also gives the model the opportunity to memorize the prior characteristic distribution. S, S t ,L t The same updating method is also adopted. We call this RNN structure, which is a unit of reset gates, RRNN.
Aggregating individual scoring results:
MMR (Maximal Marginal Relevance) is a classical structure that balances the relevance score and the diversity score. He combines the relevance score and the diversity score linearly by introducing a balancing factor λ. However lambda isA manually set superparameter is not the optimal choice in most cases. To find a more suitable balance factor, we use a learnable matrix W a To aggregate the various diversity and relevance scores into a final score:
score(d)＝φ(s)＝tanh(s T W a )
where s represents the vector of the sub-scores and phi represents the multi-layer perceptron with tanh as the activation function. s= [ div (d|q) v ),r S div(d|S),r L div(d|L),rel(d,q)]Here we omit the t superscript indicating the latest feature, here d, q, for simplicity v S, L each represent a corresponding virtual sub-topic sequence. Wherein the scores of general diversity and personalized diversity are also calculated along with KNRM structures:
div(d|q v )＝F K (d,q v )
div(d|S)＝F K (d,S)
div(d|L)＝F K (d,L)
rel(d,q)＝φ(f(d,q))
we use f (d, q) to represent additional manual feature sequences for documents and queries, use a multi-layer perceptron phi to fuse the individual manual features, and calculate a relevance score.
Training and optimizing:
in previous personalisation work, researchers thought that the clicked document only reflected the relevance of the document to the user and the query, however the user's behaviour was complex, and a redundant relevant document might not be clicked on by the user, as the user's information needs in this respect have been met. That is, the behavior of the user reflects not only document relevance but also document diversity. So we consider the document clicked by each step of user as a positive sample, and vice versa, from which the relevance and diversity both meet the user's needs. One sample pair of our model can be expressed as (r 1 ,r 2 )，r 1 ,r 2 Representing two document rankings, it is apparent that our model is the List-paper model, however the difference between these two rankings is only reflected in the last documentOn, therefore, the sample pair can also be written as (C, d 1 ,d 2 ) Where C represents the selected set of documents, d 1 ,d 2 Representing the last sample in the rank, let d 1 Is a positive sample, d 2 Is a negative example, whereby our model can be reduced to the C-based pariwise method and a two-class loss function is used to construct the loss function of our model:
where Q represents all queries in the training set, Q o Representing all pairs of samples corresponding to query q. y is o Is a sample label whenRatio->Y is more suitable for the user demand o =1, whereas 0. />Representation of model predicted +.>Ratio->Better probability. Because our sample pairs can be written in the form of a pair: (C, d) 1 ,d 2 ) We can calculate by the following formula
Finally, gradually optimizing the model by an AdamOptimezer optimizer.
Claims (6)
1. A deep learning-based personalized and diversified search method is characterized in that: the historical query data, the current query and the candidate document set of the user are input, and the method is realized through three steps: step one: and based on HRNN model, according to long-term history H of user l Short term history H s Learning its initial long-term and short-term portrayal L 0 And S is 0 Wherein H is l ＝{{q 1 ,D 1 },…,{q n ,D n }, n represents the total number of queries H contained in the previous session s ＝{{q n+1 ,D n+1 },…,{q n+m ,D n+m }, m is the number of queries that have been searched in the current session, q n Representing the nth query presented by the user, D n For query q n The ad-hoc search engine returns an initial candidate document set, so that the matching degree of the query vector and the user description image is utilized, and div (d|q v ) Representing the general diversity of document d, div (d|S), div (d|L) representing the personalized diversity of the document, using r S ,r L As the weight of the personalized diversity, dynamically controlling the influence degree of the personalized diversity on the score; step two: constructing an RRNN model, and selecting a local optimal document d of each step for candidate documents in the rest candidate document sets according to the current comprehensive score (d) by using a greedy algorithm * The influence of the document on the sub-topic representation of the user long-short term description image and the query is learned according to the virtual sub-topic representation of the document, and the influence is modeled by using a reset gate structure to updateS t-1 ,L t-1 Is->S t ,L t Wherein->S t ,L t Respectively representAfter forgetting the sub-topics covered by t selected documents, query sentences, vector characterization of the user short-term portraits and long-term portraits for score calculation of next document selection,/a user selects a new document, and a new document is selected>L 0 ,S 0 Representing the originality of queries and user portrait vector representations; finally, dynamic modeling of the diversity of the modeling documents is realized; and thirdly, aggregating each score result, wherein the score is the basis for reordering the candidate documents, and outputting the reordered search results.
2. The deep learning-based fused personalized and diversified search method of claim 1 wherein: the specific calculation modes of the general diversity weight and the personalized diversity weight are as follows: learning word-embedding using word2vec techniques, and then weighting and summing resulting vector representations based on TF-IDF weights of the wordsd ', learning the user's initial long and short period descriptive picture H 'using HRNN structure' 0 ,S' 0 ；
The HRNN structure firstly uses a first layer RNN to take all query records of users as input, and each session end is used for constructing interest characterization of the users in each session by a period node:
subscripts m and n denote the nth query record in the mth session, subscript 1 denotes the first tier RNN structure, q m,n ,d m,n Representing the corresponding input query and average Guan Wendang, the short-term descriptive representation of the user may be represented as:where M represents the Mth current session, n M Indicating that the user has been engaged in the sessionSearching;
the long-term descriptive portrait is constructed according to the whole history record which does not contain the current session record, firstly, each session characterization of the user history is taken as input, and a second-layer RNN structure is applied to learn the interest characterization of each period of the user:
where m represents the mth current session, n m Representing searches that have been conducted by the user in the session, and thereafter utilizing the current queryAnd user individual period interest characterization ∈ ->Similarity alpha of (a) m As the weight of each period interest, the weighted summation result is described as the long-term interest of the user:
introducing a convolutional neural network conv to learn a virtual sub-topic representation sequence of the conv from the original representation of the query:
wherein q is i Representing the ith sub-topic representation of the query, c represents the number of sub-topics contained in the query and is derived therefromSub-topic representation to document and user long-short term description portraits: d, L 0 ,S 0 :
d＝[d 1 ,d 2 …d c ]＝conv(d')
L 0 ＝[L 1 ,L 2 …L c ]＝conv(L' 0 )
S 0 ＝[S 1 ,S 2 …S c ]＝conv(S' 0 )
Furthermore, KNRM model was introduced and t was used q ＝[t q1 ,...,t qn ],t d ＝[t d1 ,...,t dm ]To represent word sequences of queries and documents, where t qi An i-th word embedding vector representing a query, t dj The j-th word embedded vector of the document is represented, n and M respectively represent the word numbers of the current query and the document, and KNRM constructs a transformation matrix M according to cosine similarity between each group of word pairs of the query and the document:
M ij ＝cos_similarity(t qi ,t dj )
and then the KNRM learns the multidimensional features on the transformation matrix by using k Gaussian kernels, and calculates the similarity of the query and the document by using a multi-layer perceptron, wherein the specific calculation steps are as follows:
F k (t q ,t d )＝φ(f 1 (M),...f o (M),...,f k (M))
wherein u is o ,σ o The mean value and standard deviation corresponding to the o-th Gaussian kernel are manually set hyper-parameters, and then the weight value can be expressed as follows:
3. the deep learning-based fused personalized and diversified search method of claim 2, wherein: the reset gate is constructed in the following manner:
wherein,the representation will->d * Spliced together, W q Is a learnable mapping parameter, will +.>Mapping to a real number, σ (·) represents the activation function.
4. A deep learning based fused personalized and diversified search method of claim 3 wherein: the activation function is a tanh function.
5. The deep learning-based converged personalized and diversified search method of claim 4, wherein: the update function of the reset gate is:
the total score uses a learnable matrix W a The respective diversity and relevance scores are aggregated to obtain:
score(d)＝φ(s)＝tanh(s T W a )
where s represents a vector of sub-scores, phi represents a multi-layer perceptron using tanh as an activation function, s= [ div (d|q v ),r S div(d|S),r L div(d|L),rel(d,q)]Wherein q is v S, L isS t ,L t T superscript form omitting latest feature, d, q v S, L both represent the corresponding virtual sub-topic sequences, the scores of general diversity and personalized diversity are calculated along with KNRM structures:
div(d|q v )＝F K (d,q v )
div(d|S)＝F K (d,S)
div(d|L)＝F K (d,L)
rel(d,q)＝φ(f(d,q))
using f (d, q) to represent additional manual feature sequences for the document and the query, using a multi-layer perceptron phi to fuse the individual manual features, rel (d, q) to represent the relevance scores of the query and the document calculated based on the additional manual features.
6. The deep learning-based converged personalized and diversified search method of claim 5, wherein: the loss function of the method is constructed using a binary class of loss functions:
where Q represents all queries in the training set, Q o Represent all pairs of samples corresponding to query q, y o Is a sample label whenRatio->Y is more suitable for the user demand o =1, otherwise 0, ++>Representation of model predicted +.>Ratio->Better probability, the pairs of samples are written in the form of a pair: (C, d) 1 ,d 2 ) When ∈0 is calculated by the following formula>
Finally, gradually optimizing the model by an AdamOptimezer optimizer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111391539.0A CN114048386B (en) | 2021-11-23 | 2021-11-23 | Deep learning-based personalized and diversified search method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111391539.0A CN114048386B (en) | 2021-11-23 | 2021-11-23 | Deep learning-based personalized and diversified search method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114048386A CN114048386A (en) | 2022-02-15 |
| CN114048386B true CN114048386B (en) | 2024-04-09 |
Family
ID=80211164
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111391539.0A Active CN114048386B (en) | 2021-11-23 | 2021-11-23 | Deep learning-based personalized and diversified search method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114048386B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP4293662A1 (en) * | 2022-06-17 | 2023-12-20 | Samsung Electronics Co., Ltd. | Method and system for personalising machine learning models |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20110023304A (en) * | 2009-08-31 | 2011-03-08 | 서울시립대학교 산학협력단 | Method and system of configuring user profile based on a concept network and personalized query expansion system using the same |
| CN105068661A (en) * | 2015-09-07 | 2015-11-18 | 百度在线网络技术（北京）有限公司 | Man-machine interaction method and system based on artificial intelligence |
| CN111241407A (en) * | 2020-01-21 | 2020-06-05 | 中国人民大学 | Personalized search method based on reinforcement learning |
| CN112182387A (en) * | 2020-09-29 | 2021-01-05 | 中国人民大学 | Personalized search algorithm for enhancing time information |
| CN112182373A (en) * | 2020-09-25 | 2021-01-05 | 中国人民大学 | Context expression learning-based personalized search method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10803055B2 (en) * | 2017-12-15 | 2020-10-13 | Accenture Global Solutions Limited | Cognitive searches based on deep-learning neural networks |
-
2021
- 2021-11-23 CN CN202111391539.0A patent/CN114048386B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20110023304A (en) * | 2009-08-31 | 2011-03-08 | 서울시립대학교 산학협력단 | Method and system of configuring user profile based on a concept network and personalized query expansion system using the same |
| CN105068661A (en) * | 2015-09-07 | 2015-11-18 | 百度在线网络技术（北京）有限公司 | Man-machine interaction method and system based on artificial intelligence |
| CN111241407A (en) * | 2020-01-21 | 2020-06-05 | 中国人民大学 | Personalized search method based on reinforcement learning |
| CN112182373A (en) * | 2020-09-25 | 2021-01-05 | 中国人民大学 | Context expression learning-based personalized search method |
| CN112182387A (en) * | 2020-09-29 | 2021-01-05 | 中国人民大学 | Personalized search algorithm for enhancing time information |
Non-Patent Citations (1)
| Title |
|---|
| 基于强化学习的多样性文档排序算法;官蕊;丁家满;贾连印;游进国;姜瑛;;计算机工程与科学;20200915(第09期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114048386A (en) | 2022-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109299396B (en) | Convolutional neural network collaborative filtering recommendation method and system fusing attention model | |
| CN110555112A (en) | interest point recommendation method based on user positive and negative preference learning | |
| Liu et al. | Shifu2: A network representation learning based model for advisor-advisee relationship mining | |
| Mohammadi et al. | Ensemble deep learning for aspect-based sentiment analysis | |
| WO2022095573A1 (en) | Community question answering website answer sorting method and system combined with active learning | |
| US20210256367A1 (en) | Scoring for search retrieval and ranking alignment | |
| CN112182373B (en) | Sexualization search method based on context representation learning | |
| Chen et al. | Rlpath: a knowledge graph link prediction method using reinforcement learning based attentive relation path searching and representation learning | |
| CN114358657B (en) | Post recommendation method and device based on model fusion | |
| CN115186097A (en) | Knowledge graph and reinforcement learning based interactive recommendation method | |
| CN112182387A (en) | Personalized search algorithm for enhancing time information | |
| CN113987155B (en) | Conversational retrieval method integrating knowledge graph and large-scale user log | |
| Huang et al. | Neural embedding collaborative filtering for recommender systems | |
| CN111125538A (en) | Searching method for enhancing personalized retrieval effect by using entity information | |
| CN114048386B (en) | Deep learning-based personalized and diversified search method | |
| CN115588122A (en) | News classification method based on multi-mode feature fusion | |
| CN116108267A (en) | Recommendation method and related equipment | |
| Yu et al. | Memory-augmented meta-learning framework for session-based target behavior recommendation | |
| CN114840747A (en) | News recommendation method based on comparative learning | |
| Qiao et al. | A joint neural network model for combining heterogeneous user data sources: An example of at‐risk student prediction | |
| Yao et al. | Scalable algorithms for CQA post voting prediction | |
| Wang et al. | MCCP: multi-modal fashion compatibility and conditional preference model for personalized clothing recommendation | |
| CN111435378B (en) | Query result ordering method and device, electronic equipment and storage medium | |
| JP2023550510A (en) | Recommendation methods, devices, electronic devices and storage media | |
| Xuan et al. | Cooperative hierarchical Dirichlet processes: Superposition vs. maximization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |