CN104615767B - Training method, search processing method and the device of searching order model - Google Patents

Training method, search processing method and the device of searching order model Download PDF

Info

Publication number
CN104615767B
CN104615767B CN201510082145.5A CN201510082145A CN104615767B CN 104615767 B CN104615767 B CN 104615767B CN 201510082145 A CN201510082145 A CN 201510082145A CN 104615767 B CN104615767 B CN 104615767B
Authority
CN
China
Prior art keywords
search result
search
term
vector
order model
Prior art date
Application number
CN201510082145.5A
Other languages
Chinese (zh)
Other versions
CN104615767A (en
Inventor
张军
Original Assignee
百度在线网络技术(北京)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 百度在线网络技术(北京)有限公司 filed Critical 百度在线网络技术(北京)有限公司
Priority to CN201510082145.5A priority Critical patent/CN104615767B/en
Publication of CN104615767A publication Critical patent/CN104615767A/en
Application granted granted Critical
Publication of CN104615767B publication Critical patent/CN104615767B/en

Links

Abstract

The embodiments of the invention provide a kind of training method, search processing method and the device of searching order model.The training method of the searching order model includes:The sample data of multigroup mark is obtained, sample data described in every group includes search term and its corresponding multiple search result entries for being noted as positive example or negative example;Input layer, term vector layer, hidden layer and the output layer of the searching order model of search term and its generation of corresponding search result entry based on Gated RNN in multigroup sample data, the searching order model is trained, to learn the parameter of the searching order model.Training, search processing method and the device of the searching order model of the embodiment of the present invention, it is possible to increase the degree of accuracy for the search result entry ranking score being calculated, and provide the user more accurately searching order result.

Description

Training method, search processing method and the device of searching order model

Technical field

The present invention relates to natural language processing technique field, more particularly to a kind of training method of searching order model, search Rope processing method and processing device.

Background technology

With the development of the Internet, applications, search process technology is also increasingly ripe.The search term that user inputs is expressed as Specific form, the calculating of fraction is ranked up (by taking the title of webpage as an example) with search result entry to be sorted, so as to More accurately searching order result is returned to according to ranking score, is the key problem of search engine system.

The method of traditional calculating ranking score is:By calculating search term with search result entry (with the title of webpage Exemplified by) included in the degree that matches completely of word, to calculate the similarity between the two text strings as ranking score. For example, search term is " apple new product release ", entitled " Apple Inc. issues new mobile phone " of webpage, word " apple " is arrived in investigation The two words match completely with " issue ", the ranking score come between estimating searching word and the title of webpage.But the party Method only accounts for word in literal matching degree, investigates polysemy (for example, apple also has meaning as fruit), Approximate word matching (for example, new product with new) is not investigated, and therefore, the ranking score degree of accuracy obtained based on this method is not high yet.

In recent years, with the development of depth learning technology, occur using deep-neural-network technology come learn word to Amount represents (vector for vocabulary being shown as a real number composition), by calculating by the search term that term vector forms and search result bar Similarity between mesh calculates the method for the ranking score between search term and search result entry.In the method, utilize Feedforward neural network, the word in search term and search result entry is mapped among the vector space of a low-dimensional, simply The term vector all words in search term be added to obtain the vector representation of search term, search result entry also does same processing Its vector representation is obtained, then calculates the similarity between the two vector representations, as final ranking score.Though this method from In a way, solves the problems such as conventional method does not consider polysemy, near synonym, however, being only simply added term vector The vector representation of the sentence of formation, fail the dependence investigated between word and word, for example, the search term " work which gains a reputation for sb. of Cheng Long The simple addition of term vector in product ", it is all that one kind of " works " one word is modified just not investigate " Cheng Long " and " becomeing famous ".Cause This, the ranking score degree of accuracy obtained based on this method is not also high.

The content of the invention

The purpose of the embodiment of the present invention is, there is provided a kind of training method of searching order model, search processing method and Device, to improve the accuracy that the ranking score of search term and search result entry calculates, and provide the user and more accurately search Rope ranking results.

In order to realize foregoing invention purpose, the embodiment provides one kind to be based on gating cycle neutral net The training method of the searching order model of (Gated RNN), including:Obtain the sample data of multigroup mark, sample described in every group Data include search term and its corresponding multiple search result entries for being noted as positive example or negative example;According to multigroup sample The input layer of the searching order model of search term and its generation of corresponding search result entry based on Gated RNN in data, Term vector layer, hidden layer and output layer;The searching order model is trained, to learn the ginseng of the searching order model Number.

The embodiment of the present invention additionally provides a kind of trainer of the searching order model based on gating cycle neutral net, Including:Sample data acquisition module, for obtaining the sample data of multigroup mark, sample data described in every group include search term and It is noted as multiple search result entries of positive example or negative example accordingly;Searching order model generation module, for according to institute State the searching order model of search term and its generation of corresponding search result entry based on Gated RNN in multigroup sample data Input layer, term vector layer, hidden layer and output layer;Parameter learning module, for being trained to the searching order model, To learn the parameter of the searching order model.

The embodiment of the present invention additionally provides a kind of search processing method, including:Receive the search term of user;Searched according to described Rope word obtains multiple search result entries;Using the search term and the multiple search result entry as input, from trained The searching order model based on Gated RNN in obtain the ranking score of each search result entry respectively;According to institute Ranking score is stated to be ranked up the multiple search result entry;Send ranked search result entry.

The embodiment of the present invention additionally provides a kind of search process device, including:Search term receiving module, for receiving user Search term;Search result entry acquisition module, for obtaining multiple search result entries according to the search term;Ranking score Acquisition module, for using the search term and the multiple search result entry as input, Gated to be based on from housebroken The ranking score of each search result entry is obtained in RNN searching order model respectively;Search result entry sequence mould Block, for being ranked up according to the ranking score to the multiple search result entry;Search result entry sending module, use In the ranked search result entry of transmission.

Training method, search processing method and the device of searching order model provided in an embodiment of the present invention, pass through combination Search term, corresponding search result entry and the searching order model based on Gated RNN trained using sample data, The Similarity value between search term and search result entry is calculated as ranking score, and according to ranking score to search result bar Mesh is ranked up, and so as to improve the accuracy that ranking score calculates between search term and search result entry, and can be to use Family provides more accurately searching order result.

Brief description of the drawings

Fig. 1 is the general principle block diagram for showing the embodiment of the present invention;

Fig. 2 is the flow of the training method for the searching order model based on Gated RNN for showing the embodiment of the present invention one Figure;

Fig. 3 is the illustrative diagram for the searching order model based on Gated RNN for showing the embodiment of the present invention one;

Fig. 4 is the original of the hidden layer generation for the searching order model based on Gated RNN for showing the embodiment of the present invention one Manage schematic diagram;

Fig. 5 is the original of the output layer generation for the searching order model based on Gated RNN for showing the embodiment of the present invention one Manage schematic diagram;

Fig. 6 is the flow chart for the search processing method for showing the embodiment of the present invention two;

Fig. 7 is the logic of the trainer for the searching order model based on Gated RNN for showing the embodiment of the present invention three Block diagram;

Fig. 8 is the logic diagram for the search process device for showing the embodiment of the present invention four.

Embodiment

The basic conception of the present invention is the sample data for obtaining multigroup mark, searching in multigroup sample data It is rope word and its corresponding search result entry generation input layer of the searching order model based on Gated RNN, term vector layer, hidden Layer and output layer are hidden, the searching order model is trained, to learn the parameter of the searching order model.Using described The search term of user and the multiple search result entries got are expressed as vector by parameter, calculate the phase between two vectors Like degree be used as ranking score, multiple search result entries are ranked up further according to ranking score, so as to improve search term with The accuracy that ranking score calculates between search result entry, and more accurately searching order result can be provided the user.

Fig. 1 is the general principle block diagram for showing the embodiment of the present invention.Reference picture 1, in of the invention, need to obtain training sample first This, specifically, sample data can be obtained from user's inquiry log as training sample;Secondly, using the training sample to base It is trained in Gated RNN searching order model, to learn the parameter of the model, that is, utilizes designed training Algorithm is trained to the searching order model based on Gated RNN of foundation, obtains the searching order mould based on Gated RNN The optimal parameter of type.Finally, the search term of user and corresponding multiple search result entries are obtained, will be searched using these parameters Rope word and multiple search result entries are expressed as vector, by the calculating of the similarity between two vectors, obtain search term Ranking score between each search result entry, multiple search result entries are ranked up further according to ranking score, most Ranked search result entry is obtained eventually.

Semantic similarity calculation method of the embodiment of the present invention, method for processing search results and device are entered below in conjunction with the accompanying drawings Row is described in detail.

Embodiment one

Fig. 2 is the flow of the training method for the searching order model based on Gated RNN for showing the embodiment of the present invention one Figure.Reference picture 2, the training method of the searching order model based on Gated RNN comprise the following steps:

In step S110, the sample data of multigroup mark is obtained, sample data described in every group is including search term and its accordingly The multiple search result entries for being noted as positive example or negative example.

According to the design of the present invention, the search result entry for being noted as positive example is the search result bar being clicked Mesh, the search result entry for being noted as negative example is the search result entry being not clicked on.Specifically, when user inputs After one search term, multiple search result entries can be obtained, it is further clear that user have selected certain search result progress therein Look at, this selected search result entry is the search result entry that was clicked, conversely, being then searching of being not clicked on Hitch really bar mesh.

Sample data described in the present embodiment is by M groups<Q,T+,T->To the sample of composition.M value is typically enough Greatly, it will usually more than 100,000,000 magnitudes.<Q,T+,T->To being got among user's inquiry log.Table 1 is one group<Q,T+,T- >To example, wherein Q represents the search term that user is inquired about, T+Represent Title positive examples, be user after the Query is searched for, Title corresponding to the search result entry being clicked, T-Represent Title and bear example, be then the search result entry being not clicked on Corresponding title, as shown in table 1:

Table 1

In step S120, search term and its corresponding search result entry generation base in multigroup sample data In the input layer of Gated RNN searching order model, term vector layer, hidden layer and output layer.

According to an alternative embodiment of the invention, step S120 may include, to the search term, corresponding with the search term Search result entry segmented respectively, input layer is generated by word segmentation result, found respectively point from pre-defined vocabulary Term vector corresponding to each participle obtained, term vector layer is generated by the term vector.

Show specifically, Fig. 3 is the exemplary of searching order model based on Gated RNN for showing the embodiment of the present invention one It is intended to.Reference picture 3, the search term in training sample, the search result entry corresponding with search term are segmented respectively, example Such as, it is assumed that a search term is made up of T participle, then is designated as:Query=(w1,…,wT), similarly, it is noted as positive example Search result entry is made up of M word, is designated as:Title+=(w1,…,wM), the search result entry of negative example is noted as by L Individual word composition, is designated as:Title-=(w1,…,wL), each participle that word segmentation processing is obtained inputs, you can generation input layer; Each participle w in text stringiBelong to a word in pre-defined vocabulary, the size of vocabulary is | V | (including with To identify the special word of the OOV not among dictionary<OOV>);Each participle can be found corresponding by way of looking up the dictionary Term vector, the vector layer is referred to as term vector layer.Need exist for illustrating, output layer not shown in Fig. 3, in subsequent content Generation output layer can be described in detail.

Explanation is needed exist for, the term vector is a kind of mode for the word in language to be carried out to mathematicization, is cared for Name Si Yi, term vector are exactly that a vocabulary is shown as a vector, and simplest term vector mode is with a very long vector To represent a word, vectorial length is the size of vocabulary, and vectorial component only has one " 1 ", and other are all " 0 ", the position of " 1 " Put to should position of the word in vocabulary, for example, " microphone " is expressed as [0 001 00 000000000 0...], but this mode can not portray the similitude between word and word well, herein on basis, occur a kind of word again Vector representation, overcome aforesaid drawbacks.Its general principle is direct common one word of vector representation, such as [0.792,0.177,0.107,0.109,0.542 ...], that is, common vector representation form.In actual applications, net The term vector of network represents each input word wiCorresponding term vector, it is the column vector that a length is EMBEDDING_SIZE C(wi)。

According to another alternative embodiment of the present invention, step S120 may also include, and the term vector layer be carried out non-linear Transformation calculations obtain hidden layer.Specifically, to any term vector of the term vector layer, all it is handled as follows, until obtaining Whole vector in the hidden layer:Current term vector is obtained, is searched according to the current term vector, based on Gated RNN The hidden layer transformation matrix parameter of the search term of rope order models and the hidden layer transformation matrix parameter of search result entry, are obtained To updating the data and resetting data, according to the previous word updated the data with the replacement data to the current term vector The vector of the corresponding hidden layer of vector is handled, and obtains the vector of hidden layer corresponding to the current term vector.It is namely logical Cross below equation and perform the processing for nonlinear transformation being carried out to the term vector layer hidden layer being calculated:

zj=sigmoid [Wze]j+[Uzh<t-1>]j),

rj=sigmoid [Wre]j+[Urh<t-1>]j)

Wherein,For j-th of element in t-th of vector of the hidden layer,For the t- of the hidden layer J-th of element in 1 vector,For the dependent coefficient between two vectors of the hidden layer, zjFor from the hidden layer The t-1 vector updates the data, rjTo reset data from the t-1 of hidden layer vector, e is the of the term vector layer T term vector, W, Wz、WrIt is the hidden layer transformation matrix parameter of the search term of the Gated RNN searching order models, U, Uz、UrIt is the hidden layer transformation matrix parameter of the search result entry of the Gated RNN searching order models.Here, W, Wz、WrIt is that three line numbers are HIDDEN_SIZE, columns is EMBEDDING_SIZE matrix, U, Uz、UrIt is then that three line numbers are HIDDEN_SIZE, columns are also HIDDEN_SIZE matrix.Tanh and sigmoid is two different nonlinear transformation letters Number.

Specifically, Fig. 4 is the hidden layer life for the searching order model based on Gated RNN for showing the embodiment of the present invention one Into principle schematic.Reference picture 4, in embodiments of the present invention, gating cycle neutral net unit is referred to as using one kind The nonlinear transformation unit of (Gated Recurrent UNIT) generates the vector of hidden layer, nonlinear transformation study The characteristics of unit, is that it is possible to by Reset Gate (" r " in Fig. 4) and Update Gate (" z " in Fig. 4) come automatic The dependence for learning between word and word, reset gate are for learning the how many letters of needs band from information above Current nonlinear transformation unit is ceased, and Update gate are for learning among the information of needs renewal above How much information to current nonlinear transformation unit.By Reset Gate and Update Gate combined use, because This, the searching order model based on Gated RNN that the present invention is generated can automatically learn the dependence between word and word Relation, formula as previously shown, zjIt is the specific mathematical formulae for realizing Update gate, rjIt is to realize that Reset Gate's is specific Mathematical formulae.

Need exist for explanation, in actual applications, the hidden layer of network represent to be generated based on Gated RNN's State of the searching order model in each time point i, it is the column vector h that a length is HIDDEN_SIZEi, EMBEDDING_SIZE common span is that 50 to 1000, HIDDEN_SIZE common value is EMBEDDING_SIZE 1 to 4 times.

According to another alternative embodiment of the present invention, step S120 may also include, according to calculating obtained hidden layer Search term in the sample data similarity with the corresponding multiple search result entries for being noted as positive example or negative example respectively, will Output layer of the value for each similarity being calculated as the searching order model.

Further, step S120 may particularly include:Respectively positive example or negative example are noted as by the search term, accordingly Multiple search result entries word segmentation result in last corresponding vector in the hidden layer of participle as the search Word, the corresponding multiple search result bar object vectors for being noted as positive example or negative example, using described in the vector calculating The search term similarity with the corresponding multiple search result entries for being noted as positive example or negative example respectively, it is each by what is be calculated Output layer of the value of individual similarity as the searching order model.

It is noted as respectively with corresponding specifically, being performed by below equation and calculating the search term using the vector The similarity of multiple search result entries of positive example or negative example, arranged the value for each similarity being calculated as the search The processing of the output layer of sequence model:

Wherein, Q be search term vector representation, T be search result entry vector representation, m be vector dimension, QiFor Vectorial Q i-th of element, TiFor i-th of element of vector T.

Specifically, Fig. 5 is the output layer life for the searching order model based on Gated RNN for showing the embodiment of the present invention one Into principle schematic, reference picture 5, using last corresponding vector in hidden layer of participle as final vector representation, example Such as, the search term " Beijing Administration for Industry and Commerce " in foregoing table 1, " office " are last participles, then " office " is corresponding in hidden layer Vector (" h in Fig. 5T") vector representation as search term " Beijing Administration for Industry and Commerce ", it can similarly obtain, " encyclopaedia " is corresponding to be hidden The vector (" h in Fig. 5 of layerM") as the search result entry " Baidu of Administration for Industry and Commerce of Beijing hundred for being noted as positive example The vector representation of section ", " general bureau " corresponding vector (" h in Fig. 5 in hidden layerL") as the search result bar for being noted as negative example The vector representation of mesh " State Administration for Industry and Commerce of the People's Republic of China ", then by search term and it is labeled as positive example or negative example Search result entry do vector representation after, it is possible to Similarity value between two vectors is obtained (such as by above-mentioned formula cosine+、cosine-) output layer as the searching order model.

In step S130, the searching order model is trained, to learn the parameter of the searching order model.

According to the exemplary embodiment of the present invention, step S120 may include, according to the search term respectively with corresponding quilt The similarity for being labeled as multiple search result entries of positive example or negative example establishes loss function, using the sample data to described Loss function is trained, and acquisition causes the minimum searching order model based on Gated RNN of the loss function Parameter sets.The searching order model is trained specifically, being performed by below equation, to learn the searching order The processing of the parameter of model:

Wherein, own<Q,T+,T->To being to cause the described of J (θ) minimums to be based on Gated RNN for all sample datas, θ Searching order model parameter sets,For the search term and the search result bar for being noted as positive example Similarity value between mesh,For the search term and it is noted as between the search result entry of negative example Similarity value.

Explanation is needed exist for, above-mentioned formula is loss function, and the search row is trained with stochastic gradient descent method Sequence model, specifically, being exactly with reversely passing using stochastic gradient descent method (Stochastic Gradient Descen, SGD) Algorithm (Back Propagation Through Time, BPTT) is broadcast, optimal parameter θ can be obtained.The thought of SGD algorithms It is by calculating the gradient of a certain group of training sample (partial derivative of parameter), carrying out the parameter that iteration renewal random initializtion is crossed, more New method is to allow parameter to subtract a set learning rate (learning rate) every time to be multiplied by the gradient calculated, from And the value that the searching order model based on Gated RNN can be allowed to be calculated according to parameter after many iterations, with reality Difference between value minimizes on defined loss functions.In addition, BPTT algorithms are the effective meters of one kind in RNN networks The method for calculating the gradient of parameter.

, can be according to multigroup sample data of acquisition by the training method of the searching order model based on Gated RNN In search term and its corresponding search result entry generation input layer of the searching order model based on Gated RNN, word to Layer, hidden layer and output layer are measured, and the searching order model is trained, to learn the ginseng of the searching order model Number, the searching order model can learn to the dependence between word and word so that using the parameter by search term and Multiple search result entries are expressed as vector, the ranking score degree of accuracy obtained by the Similarity Measure between two vectors It is higher, and more accurately searching order result can be provided the user using the ranking score.

Embodiment two

Fig. 6 is the flow chart for the search processing method for showing the embodiment of the present invention two.Reference picture 6, it can draw in such as search Hold up and methods described is performed on server.The search processing method comprises the following steps:

In step S210, the search term of user is received.

The search term can be the search term sent from client.For example, user is in browser searches engine interface Input " vehicle driving against traffic regulations inquiry " is scanned for, and the search term is sent to search engine server by browser application.

In step S220, multiple search result entries are obtained according to the search term.

Search engine server can be used search term using existing search technique (for example, from webpage rope prepared in advance Draw) get multiple search result entries.

In step S230, using the search term and the multiple search result entry as input, it is based on from housebroken The ranking score of each search result entry is obtained in Gated RNN searching order model respectively.

According to an alternative embodiment of the invention, step S230 may include, obtain described housebroken based on Gated RNN's The parameter of searching order model, according to the parameter by the search term and the multiple search result entry be converted into respectively to Amount represented, the search term and every is calculated respectively according to the search term by vector representation and the multiple search result entry Similarity value between the individual search result entry, and will Similarity value conduct corresponding with each search result entry The ranking score of each search result entry.

In step 240, the multiple search result entry is ranked up according to the ranking score.

In step 250, ranked search result entry is sent.

By the search processing method, multiple search results corresponding to search term based on user and the search term got Entry, obtain the row of each search result entry respectively from the housebroken searching order model based on Gated RNN Sequence fraction, the multiple search result entry is ranked up further according to the ranking score, it is ranked so as to send Search result entry, compared with prior art, improve ranking score between search term and search result entry calculate it is accurate Property, and more accurately searching order result can be provided the user.

Embodiment three

Fig. 7 is the logic of the trainer for the searching order model based on Gated RNN for showing the embodiment of the present invention three Block diagram.Reference picture 7, the trainer of the searching order model based on Gated RNN include sample data acquisition module 310th, searching order model generation module 320 and parameter learning module 330.

Sample data acquisition module 310 is used for the sample data for obtaining multigroup mark, and sample data described in every group includes searching Rope word and its corresponding multiple search result entries for being noted as positive example or negative example.

Preferably, the search result entry for being noted as positive example is the search result entry being clicked, the quilt The search result entry for being labeled as negative example is the search result entry being not clicked on.

Searching order model generation module 320 is used for the search term in multigroup sample data and its searched accordingly Hitch really bar mesh generates input layer, term vector layer, hidden layer and the output layer of the searching order model based on Gated RNN.

Further, the searching order model generation module 320 is used for the search term, relative with the search term The search result entry answered is segmented respectively, and input layer is generated by word segmentation result, is found respectively from pre-defined vocabulary Term vector corresponding to each participle got, term vector layer is generated by the term vector.

Preferably, the searching order model generation module 320 is additionally operable to carry out nonlinear transformation to the term vector layer Hidden layer is calculated.

Alternatively, the searching order model generation module 320 is additionally operable to any term vector to the term vector layer, all It is handled as follows, until obtaining vector whole in the hidden layer:Current term vector is obtained, according to the current word Vector, the hidden layer transformation matrix parameter of search term based on Gated RNN searching order models and search result entry Hidden layer transformation matrix parameter, is updated the data and is reset data, is updated the data according to described with the replacement data to institute The previous term vector for stating current term vector corresponds to the vector of hidden layer and handled, and obtains corresponding to the current term vector The vector of hidden layer.Specifically, by below equation perform it is described to the term vector layer carry out nonlinear transformation be calculated The processing of hidden layer:

zj=sigmoid ([Wze]j+[Uzh<t-1>]j),

rj=sigmoid ([Wre]j+[Urh<t-1>]j)

Wherein,For j-th of element in t-th of vector of the hidden layer,For the of the hidden layer J-th of element in t-1 vector,For the dependent coefficient between two vectors of the hidden layer, zjFor from the hidden layer The t-1 vector update the data, rjTo reset data from the t-1 vector of the hidden layer, e is the term vector layer T-th of term vector, W, Wz、WrIt is the hidden layer transformation matrix parameter of the search term of the Gated RNN searching order models, U、Uz、UrIt is the hidden layer transformation matrix parameter of the search result entry of the Gated RNN searching order models.

Further, the searching order model generation module 320 is additionally operable to calculate the sample according to obtained hidden layer Search term in the notebook data similarity with the corresponding multiple search result entries for being noted as positive example or negative example respectively, will be counted Output layer of the value of obtained each similarity as the searching order model.

Alternatively, the searching order model generation module 320 is additionally operable to be marked by the search term, accordingly respectively For last corresponding vector in the hidden layer of participle in the word segmentation result of positive example or multiple search result entries of negative example As the search term, the corresponding multiple search result bar object vectors for being noted as positive example or negative example, using described Vector calculates the search term similarity with the corresponding multiple search result entries for being noted as positive example or negative example respectively, will Output layer of the value for each similarity being calculated as the searching order model.

It is noted as respectively with corresponding specifically, being performed by below equation and calculating the search term using the vector The similarity of multiple search result entries of positive example or negative example, arranged the value for each similarity being calculated as the search The processing of the output layer of sequence model:

Wherein, Q be search term vector representation, T be search result entry vector representation, m be vector dimension, QiFor Vectorial Q i-th of element, TiFor i-th of element of vector T.

Parameter learning module 330, for being trained to the searching order model, to learn the searching order model Parameter.

Further, the parameter learning module 330 is used to be noted as just with corresponding respectively according to the search term The similarity of multiple search result entries of example or negative example establishes loss function, using the sample data to the loss function It is trained, obtains the parameter sets for the searching order model based on Gated RNN for causing the loss function minimum.

Specifically, by below equation perform it is described the searching order model is trained, to learn the search The processing of the parameter of order models:

Wherein, own<Q,T+,T->To being to cause the described of J (θ) minimums to be based on Gated RNN for all sample datas, θ Searching order model parameter sets,For the search term and the search result bar for being noted as positive example Similarity value between mesh,For the search term and it is noted as between the search result entry of negative example Similarity value.

, can be according to multigroup sample data of acquisition by the trainer of the searching order model based on Gated RNN In search term and its corresponding search result entry generation input layer of the searching order model based on Gated RNN, word to Layer, hidden layer and output layer are measured, and the searching order model is trained, to learn the ginseng of the searching order model Number, the searching order model can learn to the dependence between word and word so that using the parameter by search term and Multiple search result entries are expressed as vector, the ranking score degree of accuracy obtained by the Similarity Measure between two vectors It is higher, and more accurately searching order result can be provided the user using the ranking score.

Example IV

Fig. 8 is the logic diagram for the search process device for showing the embodiment of the present invention four.Reference picture 8, the search process Device includes search term receiving module 410, search result entry acquisition module 420, ranking score acquisition module 430, search knot Really bar mesh order module 440 and search result entry sending module 450.

Search term receiving module 410 is used for the search term for receiving user.

Search result entry acquisition module 420 is used to obtain multiple search result entries according to the search term.

Ranking score acquisition module 430 is used for using the search term and the multiple search result entry as inputting, from The ranking score of each search result entry is obtained in the housebroken searching order model based on Gated RNN respectively.

Further, the ranking score acquisition module 430 can include:

Parameter acquiring unit, for obtaining the parameter of the housebroken searching order model based on Gated RNN,

Vector representation unit, for being turned the search term and the multiple search result entry respectively according to the parameter Change vector representation into,

Ranking score computing unit, for according to the search term by vector representation and the multiple search result entry The Similarity value between the search term and each search result entry is calculated respectively, and will be with each search result Ranking score of the Similarity value corresponding to entry as each search result entry.

Search result entry order module 440 is used to carry out the multiple search result entry according to the ranking score Sequence.

Search result entry sending module 450 is used to send ranked search result entry.

By the search process device, multiple search results corresponding to search term based on user and the search term got Entry, obtain the row of each search result entry respectively from the housebroken searching order model based on Gated RNN Sequence fraction, the multiple search result entry is ranked up further according to the ranking score, it is ranked so as to send Search result entry, compared with prior art, improve ranking score between search term and search result entry calculate it is accurate Property, and more accurately searching order result can be provided the user.

In several embodiments provided by the present invention, it should be understood that disclosed apparatus and method, it can be passed through Its mode is realized.For example, device embodiment described above is only schematical, for example, the division of the module, only Only a kind of division of logic function, can there is other dividing mode when actually realizing.

In addition, each functional module in each embodiment of the present invention can be integrated in a processing module, can also That modules are individually physically present, can also two or more modules be integrated in a module.Above-mentioned integrated mould Block can both be realized in the form of hardware, can also be realized in the form of hardware adds software function module.

The above-mentioned integrated module realized in the form of software function module, can be stored in one and computer-readable deposit In storage media.Above-mentioned software function module is stored in a storage medium, including some instructions are causing a computer It is each that equipment (can be personal computer, server, or network equipment etc.) or processor (processor) perform the present invention The part steps of embodiment methods described.And foregoing storage medium includes:USB flash disk, mobile hard disk, read-only storage (Read- Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disc or CD etc. it is various Can be with the medium of store program codes.

The foregoing is only a specific embodiment of the invention, but protection scope of the present invention is not limited thereto, any Those familiar with the art the invention discloses technical scope in, change or replacement can be readily occurred in, should all be contained Cover within protection scope of the present invention.Therefore, protection scope of the present invention should be based on the protection scope of the described claims.

Claims (20)

1. the training method of searching order model of the one kind based on gating cycle neutral net (Gated RNN), it is characterised in that Methods described includes:
Obtain the sample data of multigroup mark, sample data described in every group include search term and its it is corresponding be noted as positive example or Multiple search result entries of negative example;
Search term and its corresponding search result entry in multigroup sample data generate the search row based on Gated RNN Input layer, term vector layer, hidden layer and the output layer of sequence model;
The searching order model is trained, to learn the parameter of the searching order model.
2. training method according to claim 1, it is characterised in that it is described be noted as positive example search result entry be The search result entry being clicked, the search result entry for being noted as negative example is the search result bar being not clicked on Mesh.
3. training method according to claim 2, it is characterised in that the search in multigroup sample data The input layer of the searching order model of word and its generation of corresponding search result entry based on Gated RNN and the place of term vector layer Reason includes:
The search term, the search result entry corresponding with the search term are segmented respectively, generated by word segmentation result Input layer,
Find term vector corresponding to each participle got respectively from pre-defined vocabulary, from the term vector generate word to Measure layer.
4. training method according to claim 3, it is characterised in that the search in multigroup sample data The processing of the hidden layer of the searching order model of word and its generation of corresponding search result entry based on Gated RNN includes:It is right The term vector layer carries out nonlinear transformation and hidden layer is calculated.
5. training method according to claim 4, it is characterised in that the search in multigroup sample data The processing of the output layer of the searching order model of word and its generation of corresponding search result entry based on Gated RNN includes:
Search term in the sample data is calculated according to obtained hidden layer and is noted as positive example or negative example with corresponding respectively Multiple search result entries similarity, using the value for each similarity being calculated as the defeated of the searching order model Go out layer.
6. training method according to claim 4, it is characterised in that described that nonlinear transformation is carried out to the term vector layer The processing of hidden layer, which is calculated, to be included:
To any term vector of the term vector layer, all it is handled as follows, until obtaining vector whole in the hidden layer:
Current term vector is obtained,
Joined according to the hidden layer transformation matrix of the current term vector, search term based on Gated RNN searching order models The hidden layer transformation matrix parameter of number and search result entry, is updated the data and is reset data,
Hidden layer is corresponded to the previous term vector of the current term vector with the replacement data according to described update the data Vector is handled, and obtains the vector of hidden layer corresponding to the current term vector.
7. training method according to claim 5, it is characterised in that the hidden layer that the basis obtains calculates the sample Search term in the data similarity with the corresponding multiple search result entries for being noted as positive example or negative example respectively, will be calculated The value of obtained each similarity includes as the processing of the output layer of the searching order model:
In the word segmentation result for the multiple search result entries for being noted as positive example or negative example by the search term, accordingly respectively most The corresponding vector in the hidden layer of the latter participle is as the search term, corresponding positive example or the negative example of being noted as Multiple search result bar object vectors,
Using the vector calculate the search term respectively with the corresponding multiple search result bars for being noted as positive example or negative example Purpose similarity, the output layer using the value for each similarity being calculated as the searching order model.
8. training method according to claim 1, it is characterised in that it is described that the searching order model is trained, Included with learning the processing of the parameter of the searching order model:
Built according to similarity of the search term respectively with the corresponding multiple search result entries for being noted as positive example or negative example Vertical loss function,
The loss function is trained using the sample data, obtains described being based on for causing the loss function minimum The parameter sets of Gated RNN searching order model.
9. a kind of search processing method, it is characterised in that methods described includes:
Receive the search term of user;
Multiple search result entries are obtained according to the search term;
Using the search term and the multiple search result entry as input, from the housebroken search based on Gated RNN The ranking score of each search result entry is obtained in order models respectively;
The multiple search result entry is ranked up according to the ranking score;
Send ranked search result entry.
10. according to the method for claim 9, it is characterised in that described with the search term and the multiple search result Entry obtains each search result respectively as input from the housebroken searching order model based on Gated RNN The processing of the ranking score of entry includes:
The parameter of the housebroken searching order model based on Gated RNN is obtained,
The search term and the multiple search result entry are converted into by vector representation according to the parameter respectively,
The search term and each is calculated according to the search term by vector representation and the multiple search result entry respectively Similarity value between the search result entry, and will Similarity value corresponding with each search result entry as every The ranking score of the individual search result entry.
11. the trainer of searching order model of the one kind based on gating cycle neutral net (Gated RNN), its feature exist In described device includes:
Sample data acquisition module, for obtaining the sample data of multigroup mark, sample data described in every group include search term and It is noted as multiple search result entries of positive example or negative example accordingly;
Searching order model generation module, for the search term in multigroup sample data and its corresponding search result entry Generate input layer, term vector layer, hidden layer and the output layer of the searching order model based on Gated RNN;
Parameter learning module, for being trained to the searching order model, to learn the parameter of the searching order model.
12. trainer according to claim 11, it is characterised in that the search result entry for being noted as positive example It is the search result entry being clicked, the search result entry for being noted as negative example is the search result being not clicked on Entry.
13. trainer according to claim 12, it is characterised in that the searching order model generation module be used for pair The search term, the search result entry corresponding with the search term are segmented respectively, and input layer is generated by word segmentation result, Find term vector corresponding to each participle got respectively from pre-defined vocabulary, term vector is generated by the term vector Layer.
14. trainer according to claim 13, it is characterised in that the searching order model generation module is additionally operable to Nonlinear transformation is carried out to the term vector layer hidden layer is calculated.
15. trainer according to claim 14, it is characterised in that the searching order model generation module is additionally operable to Search term in the sample data is calculated according to obtained hidden layer and is noted as the more of positive example or negative example with corresponding respectively The similarity of individual search result entry, the output using the value for each similarity being calculated as the searching order model Layer.
16. trainer according to claim 14, it is characterised in that the searching order model generation module is additionally operable to To any term vector of the term vector layer, all it is handled as follows, until obtaining vector whole in the hidden layer:Obtain Current term vector, become according to the hidden layer of the current term vector, search term based on Gated RNN searching order models The hidden layer transformation matrix parameter of matrix parameter and search result entry is changed, data are updated the data and reset, according to institute State to update the data and the previous term vector of the current term vector is corresponded at the vector of hidden layer with the replacement data Reason, obtains the vector of hidden layer corresponding to the current term vector.
17. trainer according to claim 15, it is characterised in that the searching order model generation module is additionally operable to Last in the word segmentation result for the multiple search result entries for being noted as positive example or negative example by the search term, accordingly respectively The individual corresponding vector in the hidden layer of participle as the search term, corresponding be noted as the multiple of positive example or negative example Search result bar object vector, using the vector calculate the search term respectively with corresponding positive example or the negative example of being noted as The similarity of multiple search result entries, the output using the value for each similarity being calculated as the searching order model Layer.
18. trainer according to claim 11, it is characterised in that the parameter learning module is used to search according to Similarity of the rope word respectively with the corresponding multiple search result entries for being noted as positive example or negative example establishes loss function, utilizes The sample data is trained to the loss function, is obtained and is caused the described of loss function minimum to be based on Gated The parameter sets of RNN searching order model.
19. a kind of search process device, it is characterised in that described device includes:
Search term receiving module, for receiving the search term of user;
Search result entry acquisition module, for obtaining multiple search result entries according to the search term;
Ranking score acquisition module, for using the search term and the multiple search result entry as input, from trained The searching order model based on Gated RNN in obtain the ranking score of each search result entry respectively;
Search result entry order module, for being ranked up according to the ranking score to the multiple search result entry;
Search result entry sending module, for sending ranked search result entry.
20. device according to claim 19, it is characterised in that the ranking score acquisition module includes:
Parameter acquiring unit, for obtaining the parameter of the housebroken searching order model based on Gated RNN,
Vector representation unit, for being converted into the search term and the multiple search result entry respectively according to the parameter Vector representation,
Ranking score computing unit, for distinguishing according to by the search term of vector representation and the multiple search result entry The Similarity value between the search term and each search result entry is calculated, and will be with each search result entry Ranking score of the corresponding Similarity value as each search result entry.
CN201510082145.5A 2015-02-15 2015-02-15 Training method, search processing method and the device of searching order model CN104615767B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510082145.5A CN104615767B (en) 2015-02-15 2015-02-15 Training method, search processing method and the device of searching order model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510082145.5A CN104615767B (en) 2015-02-15 2015-02-15 Training method, search processing method and the device of searching order model

Publications (2)

Publication Number Publication Date
CN104615767A CN104615767A (en) 2015-05-13
CN104615767B true CN104615767B (en) 2017-12-29

Family

ID=53150209

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510082145.5A CN104615767B (en) 2015-02-15 2015-02-15 Training method, search processing method and the device of searching order model

Country Status (1)

Country Link
CN (1) CN104615767B (en)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104915448B (en) * 2015-06-30 2018-03-27 中国科学院自动化研究所 A kind of entity based on level convolutional network and paragraph link method
CN105809473A (en) * 2016-02-29 2016-07-27 北京百度网讯科技有限公司 Training method, service recommending method for coupling model parameters and associated apparatus
CN105930413A (en) * 2016-04-18 2016-09-07 北京百度网讯科技有限公司 Training method for similarity model parameters, search processing method and corresponding apparatuses
CN106095966B (en) * 2016-06-15 2019-12-27 成都品果科技有限公司 User extensible label labeling method and system
CN106250464A (en) * 2016-07-28 2016-12-21 北京百度网讯科技有限公司 The training method of order models and device
CN106294584B (en) * 2016-07-28 2019-11-05 北京百度网讯科技有限公司 The training method and device of order models
CN106294635B (en) * 2016-08-02 2019-09-03 北京百度网讯科技有限公司 Application program searching method, the training method of deep neural network model and device
CN106326338A (en) * 2016-08-03 2017-01-11 北京百度网讯科技有限公司 Service providing method and device based on search engine
CN106202574A (en) * 2016-08-19 2016-12-07 清华大学 The appraisal procedure recommended towards microblog topic and device
CN106354852A (en) * 2016-09-02 2017-01-25 北京百度网讯科技有限公司 Search method and device based on artificial intelligence
CN106354856A (en) * 2016-09-05 2017-01-25 北京百度网讯科技有限公司 Enhanced deep neural network search method and device based on artificial intelligence
CN106557554B (en) * 2016-11-04 2019-11-08 北京百度网讯科技有限公司 The display methods and device of search result based on artificial intelligence
CN106777088A (en) * 2016-12-13 2017-05-31 飞狐信息技术(天津)有限公司 The method for sequencing search engines and system of iteratively faster
CN108256583A (en) * 2018-01-25 2018-07-06 北京东方科诺科技发展有限公司 A kind of multi-tag classification learning method based on coupling learning

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101211339A (en) * 2006-12-29 2008-07-02 上海芯盛电子科技有限公司 Intelligent web page classifier based on user behaviors
CN101566998A (en) * 2009-05-26 2009-10-28 华中师范大学 Chinese question-answering system based on neural network
CN103678599A (en) * 2013-12-13 2014-03-26 北京奇虎科技有限公司 Method and device for judging document correlation based on PLSA algorithm
CN103744951A (en) * 2014-01-02 2014-04-23 上海大学 Method for ordering significance of keywords in text
CN104049755A (en) * 2014-06-18 2014-09-17 中国科学院自动化研究所 Information processing method and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9519858B2 (en) * 2013-02-10 2016-12-13 Microsoft Technology Licensing, Llc Feature-augmented neural networks and applications of same

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101211339A (en) * 2006-12-29 2008-07-02 上海芯盛电子科技有限公司 Intelligent web page classifier based on user behaviors
CN101566998A (en) * 2009-05-26 2009-10-28 华中师范大学 Chinese question-answering system based on neural network
CN103678599A (en) * 2013-12-13 2014-03-26 北京奇虎科技有限公司 Method and device for judging document correlation based on PLSA algorithm
CN103744951A (en) * 2014-01-02 2014-04-23 上海大学 Method for ordering significance of keywords in text
CN104049755A (en) * 2014-06-18 2014-09-17 中国科学院自动化研究所 Information processing method and device

Also Published As

Publication number Publication date
CN104615767A (en) 2015-05-13

Similar Documents

Publication Publication Date Title
Ristoski et al. Rdf2vec: Rdf graph embeddings for data mining
US8386240B2 (en) Domain dictionary creation by detection of new topic words using divergence value comparison
US20160026696A1 (en) Identifying query aspects
US9679558B2 (en) Language modeling for conversational understanding domains using semantic web resources
CN104834747B (en) Short text classification method based on convolutional neural networks
US9147154B2 (en) Classifying resources using a deep network
US20130339344A1 (en) Web-scale entity relationship extraction
CN101223525B (en) Relationship networks
US20120191745A1 (en) Synthesized Suggestions for Web-Search Queries
US8380489B1 (en) System, methods, and data structure for quantitative assessment of symbolic associations in natural language
CN104573028A (en) Intelligent question-answer implementing method and system
US9141906B2 (en) Scoring concept terms using a deep network
CN104615589A (en) Named-entity recognition model training method and named-entity recognition method and device
CN103679462A (en) Comment data processing method and device and searching method and system
CN101944099B (en) Method for automatically classifying text documents by utilizing body
CN104834748B (en) It is a kind of to utilize the image search method based on deep semantic sequence Hash coding
WO2013192101A1 (en) Ranking search results based on click through rates
US20150220833A1 (en) Generating vector representations of documents
US20110191336A1 (en) Contextual image search
TW201617941A (en) Generating and using a knowledge-enhanced model
CN100595759C (en) Method and device for enquire enquiry extending as well as related searching word stock
CN103678576B (en) The text retrieval system analyzed based on dynamic semantics
CN103744981B (en) System for automatic classification analysis for website based on website content
CN104462593B (en) A kind of method and apparatus that the push of user individual message related to resources is provided
CN102902821B (en) The image high-level semantics mark of much-talked-about topic Network Based, search method and device

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
GR01 Patent grant
GR01 Patent grant