CN109086417A - Search for evaluation method and device - Google Patents
Search for evaluation method and device Download PDFInfo
- Publication number
- CN109086417A CN109086417A CN201810885991.4A CN201810885991A CN109086417A CN 109086417 A CN109086417 A CN 109086417A CN 201810885991 A CN201810885991 A CN 201810885991A CN 109086417 A CN109086417 A CN 109086417A
- Authority
- CN
- China
- Prior art keywords
- search result
- depth
- search
- browsing
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This disclosure relates to a kind of search evaluation method and device, which comprises according to the relevance score of search result and click necessity scoring, determine the probability that search result is clicked；According to the depth of the abstract of search result, the depth of target pages and the probability being clicked, the expectation browsing depth of search result is determined；Depth is browsed according to the expectation of search result, determines the expectation browsing initial depth of search result；Initial depth, the depth of abstract, the depth of target pages, relevance score and browsing depth distribution function are browsed according to the expectation of search result, determines the income of search result；According to the income of search result, the scoring of searched page is determined.Search evaluation method according to an embodiment of the present disclosure and device, in the scoring for determining searched page, by the depth of the abstract of search result and the depth of target pages in view of can preferably reflect the usage experience of user in the design of index.
Description
Technical field
This disclosure relates to computer field more particularly to a kind of search evaluation method and device.
Background technique
In the mobile interchange epoch, the searching request from mobile terminal that search engine receives constantly increases.Current movement
Search technique is mainly migrated from traditional desktop search, also mainly uses traditional desktop to the evaluation method of mobile search
The evaluation index of search, such as nDCG, RBP, ERR etc..Evaluation to search is the pith of information retrieval field, it can be helped
Researcher measures the quality of search result, and the search experience of reflection search user, is the performance boost direction of search system.
Current widely used evaluation index is all based on the correlation mark of search result, consider search result income with
The decaying of search results ranking, using the resultful integral benefit of institute on result of page searching as evaluation score come to search page
It is evaluated in face.Since most of search result is homogeneity in conventional desktop search, existing evaluation index is considering income
Decaying when be mainly based upon the sequence of result, by taking nDCG as an example, the attenuation function of the income of k-th of result is 1/log2 (1+
k)。
However, mobile search is at many aspects, all there is significant differences with conventional desktop search: information requirement is distributed not
Together, on the mobile apparatus, people can more search for content relevant to information such as geographical location, amusements；Interactive mode is different,
In traditional desktop search, people are mainly interacted with mouse-keyboard, and use smaller szie on the mobile apparatus more
Multi-point touch screen etc..On the other hand, the form of search result is also varied widely than before, with traditional with text
Result appearance form based on this information is compared, and there are a large amount of results comprising rich interactive information in current mobile search.
The evaluation of these search results being distinguished as under mobile environment brings new problem: search result is no longer homogeneity
, but tend to heterogeneousization.This to consider that the mode of income decaying is no longer fully effective based on sort result.Search result is in
Existing information is more abundant.This makes user not need to click these results and can obtain useful letter from result of page searching
Breath, needs to be taken into account in evaluation for the click probability of search result.Scouting screen is smaller, and resultant content is more.This
So that user can not obtain whole useful informations of search result quickly, and may need through interactive modes such as sliding screens
All information is obtained, the difference in size of search result is also required to be taken into account in evaluation.
Summary of the invention
In view of this, the present disclosure proposes a kind of search evaluation method and devices.
According to the one side of the disclosure, a kind of search evaluation method is proposed, comprising:
According to the relevance score of multiple search results in searched page and necessity scoring is clicked, determines each search
As a result the probability being clicked, wherein the relevance score is used to characterize the phase between search key and described search result
Guan Xing, the degree of perfection clicking necessity and scoring for characterizing the abstract of search result；
According to the depth of the abstract of each search result, the depth of the target pages of each search result and described each
The probability that search result is clicked determines the expectation browsing depth of each search result, wherein the target pages are search knot
The page that fruit jumps after being clicked；
Depth is browsed according to the expectation of each search result, determines the expectation browsing initial depth of each search result；
Initial depth, the depth of the abstract of each search result, each search are browsed according to the expectation of each search result
As a result the depth of target pages, the relevance score of each search result and browsing depth distribution function, determination are each searched
The income of hitch fruit；
According to the income of multiple search results, the scoring of the described search page is determined.
In one possible implementation, the method also includes:
In the training search result of training searched page in training set, determining has various relevance scores and click
The probability that the training search result of necessity scoring is clicked, wherein the trained searched page and training search key phase
It is corresponding.
In one possible implementation, it in the training search result of the training searched page in training set, determines
The probability that training search result with various relevance scores and click necessity scoring is clicked, comprising:
In the training search result of the trained searched page, determining has the first relevance score and the first click must
First quantity of the training search result of the property wanted scoring；
In the training search result for clicking necessity scoring with the first relevance score and first, determination is clicked
Training search result the second quantity；
The ratio of second quantity and first quantity being determined as to, there is the first relevance score and first to click
The probability that the training search result of necessity scoring is clicked.
In one possible implementation, the method also includes:
According to the browsing depth of the training searched page in training set, the browsing depth distribution function is determined, wherein institute
It states browsing depth and is used to characterize depth of the shown most deep position of the trained searched page in training searched page.
In one possible implementation, according to the browsing depth of the training searched page in training set, determine described in
Browse depth distribution function, comprising:
Determine the relation curve between the quantity of the training searched page of the browsing depth and each browsing depth；
The relation curve is fitted, determines the browsing depth distribution function.
In one possible implementation, the relation curve is fitted, determines the browsing depth distribution letter
Number, comprising:
According to formulaIt is bent to the relationship
Line is fitted, and determines the browsing depth distribution function, wherein D (h) is the browsing depth distribution function, and h is described clear
Look at depth, Φ is the cumulative distribution function of standard gaussian distribution, and λ and μ are fitting parameter.
In one possible implementation, according to the depth of the abstract of each search result, the mesh of each search result
The probability that the depth and each search result for marking the page are clicked determines the expectation browsing depth of each search result,
Include:
According to formulaDetermine the expectation browsing depth of each search result,
In, evhkDepth is browsed for the expectation of k-th search result, and P (C | Rk,Nk) it is the probability that k-th of search result is clicked, Rk
For the relevance score of k-th of search result, NkIt scores for the click necessity of k-th of search result,It is searched for for k-th
As a result the depth of target pages,For the depth of the abstract of k-th of search result.
In one possible implementation, depth is browsed according to the expectation of each search result, determines each search knot
The expectation of fruit browses initial depth, comprising:
According to formulaDetermine the expectation browsing initial depth of each search result, wherein
startkInitial depth, evh are browsed for the expectation of k-th of search resultiDepth is browsed for the expectation of i-th of search result.
In one possible implementation, initial depth is browsed according to the expectation of each search result, each search is tied
The depth of the abstract of fruit, the depth of the target pages of each search result, the relevance score and browsing of each search result
Depth distribution function determines the income of each search result, comprising:
According to formulaDetermine the income of each search result, wherein
dgkFor the income of k-th of search result, startkInitial depth is browsed for the expectation of k-th of search result,It is searched for k-th
The depth of the abstract of hitch fruit,For the depth of the target pages of k-th of search result, RkFor the correlation of k-th of search result
Property scoring, D (h) be browsing depth distribution function.
According to another aspect of the present disclosure, a kind of search evaluating apparatus is proposed, comprising:
Probability determination module, for the relevance score and click necessity according to multiple search results in searched page
Scoring, determines the probability that each search result is clicked, wherein the relevance score is for characterizing search key and described
Correlation between search result, the degree of perfection clicking necessity and scoring for characterizing the abstract of search result；
It is expected that browse depth determining module, for according to the depth of the abstract of each search result, each search result
The probability that the depth of target pages and each search result are clicked determines that the expectation browsing of each search result is deep
Degree, wherein the target pages are the page jumped after search result is clicked；
It is expected that browsing initial depth determining module, for browsing depth according to the expectation of each search result, determine each
The expectation of search result browses initial depth；
Income determining module, for the expectation according to each search result to browse initial depth, each search result is plucked
The depth of the target pages of the depth, each search result wanted, the relevance score of each search result and browsing depth point
Cloth function determines the income of each search result；
The determining module that scores determines the scoring of the described search page for the income according to multiple search results.
In one possible implementation, described device further include:
Training probabilistic module, in the training search result for the training searched page in training set, determine have it is each
The probability that kind relevance score and the training search result for clicking necessity scoring are clicked, wherein the trained searched page
It is corresponding with training search key.
In one possible implementation, the trained probabilistic module is used for:
In the training search result of the trained searched page, determining has the first relevance score and the first click must
First quantity of the training search result of the property wanted scoring；
In the training search result for clicking necessity scoring with the first relevance score and first, determination is clicked
Search result the second quantity；
The ratio of second quantity and first quantity being determined as to, there is the first relevance score and first to click
The probability that the training search result of necessity scoring is clicked.
In one possible implementation, described device further include:
Depth distribution function determination module is browsed, for the browsing depth according to the training searched page in training set, really
The fixed browsing depth distribution function, wherein the browsing depth for characterize the trained searched page it is shown most
Depth of the deep position in training searched page.
In one possible implementation, the browsing depth distribution function determination module is used for:
Determine the relation curve between the quantity of the training searched page of the browsing depth and each browsing depth；
The relation curve is fitted, determines the browsing depth distribution function.
In one possible implementation, the relation curve is fitted, determines the browsing depth distribution letter
Number, comprising:
According to formulaIt is bent to the relationship
Line is fitted, and determines the browsing depth distribution function, wherein D (h) is the browsing depth distribution function, and h is described clear
Look at depth, Φ is the cumulative distribution function of standard gaussian distribution, and λ and μ are fitting parameter.
In one possible implementation, the expectation browsing depth determining module is used for:
According to formulaDetermine the expectation browsing depth of each search result, wherein
evhkDepth is browsed for the expectation of k-th search result, and P (C | Rk,Nk) it is the probability that k-th of search result is clicked, RkIt is
The relevance score of k search result, NkIt scores for the click necessity of k-th of search result,For k-th of search result
Target pages depth,For the depth of the abstract of k-th of search result.
In one possible implementation, the expectation browsing initial depth determining module is used for:
According to formulaDetermine the expectation browsing initial depth of each search result, wherein
startkInitial depth, evh are browsed for the expectation of k-th of search resultiDepth is browsed for the expectation of i-th of search result.
In one possible implementation, the income determining module is used for:
According to formulaDetermine the income of each search result, wherein
dgkFor the income of k-th of search result, startkInitial depth is browsed for the expectation of k-th of search result,It is searched for k-th
The depth of the abstract of hitch fruit,For the depth of the target pages of k-th of search result, RkFor the correlation of k-th of search result
Property scoring, D (h) be browsing depth distribution function.
Search evaluation method according to an embodiment of the present disclosure and device, the probability being clicked by each search result,
The depth of the target pages of the depth and search result of the abstract of each search result can determine that the expectation of each search result is clear
It lookes at depth, and then determines that the expectation of each search result browses initial depth, further, it may be determined that the receipts of each search result
Benefit, and then determine the scoring of searched page.In the scoring for determining searched page, by the depth and target of the abstract of search result
The depth of the page is in view of can preferably reflect the usage experience of user in the design of index.
According to below with reference to the accompanying drawings to detailed description of illustrative embodiments, the other feature and aspect of the disclosure will become
It is clear.
Detailed description of the invention
Comprising in the description and constituting the attached drawing of part of specification and specification together illustrates the disclosure
Exemplary embodiment, feature and aspect, and for explaining the principles of this disclosure.
Fig. 1 is the flow chart of search evaluation method shown according to an exemplary embodiment；
Fig. 2 is the flow chart of search evaluation method shown according to an exemplary embodiment；
Fig. 3 is the flow chart of search evaluation method shown according to an exemplary embodiment；
Fig. 4 shows the block diagram of search evaluating apparatus shown according to an exemplary embodiment；
Fig. 5 shows the block diagram of search evaluating apparatus shown according to an exemplary embodiment；
Fig. 6 shows the block diagram of search evaluating apparatus shown according to an exemplary embodiment；
Fig. 7 shows the block diagram of search evaluating apparatus shown according to an exemplary embodiment.
Specific embodiment
Various exemplary embodiments, feature and the aspect of the disclosure are described in detail below with reference to attached drawing.It is identical in attached drawing
Appended drawing reference indicate element functionally identical or similar.Although the various aspects of embodiment are shown in the attached drawings, remove
It non-specifically points out, it is not necessary to attached drawing drawn to scale.
Dedicated word " exemplary " means " being used as example, embodiment or illustrative " herein.Here as " exemplary "
Illustrated any embodiment should not necessarily be construed as preferred or advantageous over other embodiments.
In addition, giving numerous details in specific embodiment below to better illustrate the disclosure.
It will be appreciated by those skilled in the art that without certain details, the disclosure equally be can be implemented.In some instances, for
Method, means, element and circuit well known to those skilled in the art are not described in detail, in order to highlight the purport of the disclosure.
Fig. 1 is the flow chart of search evaluation method shown according to an exemplary embodiment.As shown in Figure 1, the method
It can be used for server or terminal, which comprises
In step s 11, it is scored according to the relevance score of multiple search results in searched page and click necessity,
Determine the probability that each search result is clicked, wherein the relevance score is for characterizing search key and described search
As a result the correlation between, the degree of perfection clicking necessity and scoring for characterizing the abstract of search result；
In step s 12, according to the depth of the abstract of each search result, the depth of the target pages of each search result
And the probability that each search result is clicked, determine the expectation browsing depth of each search result, wherein the target
The page is the page jumped after search result is clicked；
In step s 13, depth is browsed according to the expectation of each search result, determines the expectation browsing of each search result
Initial depth；
In step S14, according to the depth of the expectation browsing initial depth of each search result, the abstract of each search result
Degree, the depth of target pages of each search result, the relevance score of each search result and browsing depth distribution function,
Determine the income of each search result；
In step S15, according to the income of multiple search results, the scoring of the described search page is determined.
Search evaluation method according to an embodiment of the present disclosure, the probability being clicked by each search result are each searched
The depth of the target pages of the depth and search result of the abstract of hitch fruit can determine the expectation browsing depth of each search result,
And then determine that the expectation of each search result browses initial depth, and further, it may be determined that the income of each search result, in turn
Determine the scoring of searched page.In the scoring for determining searched page, by the depth of the abstract of search result and target pages
Depth is in view of can preferably reflect the usage experience of user in the design of index.
It in one possible implementation, in step s 11, can must according to the relevance score and click of search result
The property wanted scores, and inquires in training set relevance score having the same and clicks necessity and scores what training search result was clicked
Probability, and be the probability that search result is clicked by the determine the probability.
Fig. 2 is the flow chart of search evaluation method shown according to an exemplary embodiment.As shown in Fig. 2, the method
Further include:
In step s 16, in the training search result of the training searched page in training set, determining has various correlations
Property scoring and click the probability that is clicked of training search result of necessity scoring, wherein the trained searched page and training
Search key is corresponding.
It in this example, may include multiple trained searched pages in training set, each trained searched page is instructed with specific
It is corresponding to practice search key.Multiple trained search results, each trained search result are all had on each trained searched page
There can be relevance score and click necessity scoring, the relevance score of training search result is crucial for characterizing training search
The click necessity of correlation between word and training search result, training search result scores for characterizing trained search result
Abstract degree of perfection, for example, if training search result abstract sophistication it is higher, included rich in abstract
Rich content, then necessarily click the training search result, also can get the content of the training search result, therefore,
When the sophistication of the abstract of training search result is higher, it is lower to click necessity scoring, conversely, training the abstract of search result
Sophistication it is lower when, then click necessity scoring it is higher.
In this example, the relevance score of each trained search result and click necessity scoring can beating according to labeler
It separately wins, or is obtained according to expertise Evaluation Method.In this example, labeler is according to the content and training for training search result
Correlation between search key, the degree of correlation marking between the training search result and training search key, or
Person is obtained according to expertise for the degree of correlation marking between the content and training search key of training search result
Score, the as relevance score of the training search result.For example, the relevance score of training search result is 1 point, 2
Point ... 5 points etc., score is higher, indicates that the degree of correlation between training search result and training search key is higher, the disclosure
Method and score value to relevance score are with no restrictions.
In this example, labeler is carried out according to the degree of perfection of the abstract of training search result for the training search result
Marking, or according to expertise, the degree of perfection for the abstract of the training search result is given a mark, score obtained,
The as click necessity scoring of the training search result.For example, the click necessity scoring of training search result is 1 point, 2
Point, 3 points etc., score is higher, indicates that the degree of perfection of the abstract of training search result is lower.The disclosure scores necessity is clicked
Method and score value with no restrictions.
In one possible implementation, it in the training search result of the training searched page in training set, determines
The probability that training search result with various relevance scores and click necessity scoring is clicked, comprising: in the training
In the training search result of searched page, determining, there is the first relevance score and first to click the training search of necessity scoring
As a result the first quantity；In the training search result for clicking necessity scoring with the first relevance score and first, determine
The second quantity by the training search result clicked；The ratio of second quantity and first quantity is determined as having
The probability that first relevance score and the training search result of the first click necessity scoring are clicked.
In one possible implementation, first relevance score can be arbitrary relevance score, and first
Click necessity scoring can be arbitrary by clicking necessity scoring.It can be in the training search result of the trained searched page
In, determining, there is the first relevance score and first to click the first quantity of the training search result that necessity scores, in example
In, the first relevance score is 5 points, and it is 3 points that first, which clicks necessity, which scores, can be in all trained searched pages of training set
All trained search results in, determine that relevance score is 5 points, click necessity scoring as 3 points of training search result
Quantity.
It in one possible implementation, can be in the instruction for clicking necessity scoring with the first relevance score and first
Practice in search result, determines the second quantity of the training search result being clicked.In this example, the first relevance score is 5 points,
First click necessity scoring be 3 points, can all relevance scores of training set be 5 points and click necessity scoring be 3 points
Training search result in, determine the second quantity of training search result being clicked.
In one possible implementation, the second quantity can be determined as to the ratio of the first quantity related with first
Property scoring and the first click necessity scoring the probability that is clicked of training search result.In this example, the first relevance score
It is 5 points, it is 3 points that first, which clicks necessity, which scores, in training set, there is the first relevance score and the first click necessity to comment
The search result divided has 10000, and in this 10000 search results, the search result being clicked has 2000, then the first number
Amount is 10000, and the second quantity is 2000, and the training search result of necessity scoring is clicked with the first relevance score and first
The probability being clicked is 0.2.
In this example, in the above manner, can determine with various relevance scores and click the training of necessity scoring
The probability that search result is clicked.For example, relevance score is 5 points and clicks necessity to score the search result for being 2 points by point
The probability hit is 0.25, and the probability that the search result that relevance score is 5 points and click necessity scoring is 1 point is clicked is
0.3, relevance score is 4 points and clicks the probability that the search result that necessity scoring is 3 points is clicked to be 0.2 etc..In step
In S11, according to the relevance score of search result and necessity scoring can be clicked, come inquire relevance score having the same and
Click the probability that is clicked of training search result of necessity scoring, and it is general by the determine the probability to be that the search result is clicked
Rate.
It in one possible implementation, in step s 12, can be according to the depth of the abstract of each search result, every
The probability that the depth of the target pages of a search result and each search result are clicked, determines each search result
It is expected that browsing depth.In this example, the depth of abstract be search result abstract in searched page occupied depth.Showing
In example, the depth that the depth of the abstract of search result can be the bottom end position of the abstract of this search result subtracts top
The depth of position, for example, the bottom end institute that the depth of the abstract of search result can be the abstract of this search result is in place
Set what the top position that the abstract of this search result is subtracted at a distance from page top obtained at a distance from page top
Difference.
In one possible implementation, the target pages of search result are the page jumped after search result is clicked
Face, the depth of target pages can be the total height of target pages, the i.e. distance on the top of target pages to bottom end.
In one possible implementation, according to the depth of the abstract of each search result, the mesh of each search result
The probability that the depth and each search result for marking the page are clicked determines the expectation browsing depth of each search result,
May include according to the following formula (1) come determine each search result expectation browse depth:
Wherein, evhkDepth is browsed for the expectation of k-th search result, and P (C | Rk,Nk) it is that k-th of search result is clicked
Probability, RkFor the relevance score of k-th of search result, NkIt scores for the click necessity of k-th of search result,It is
The depth of the target pages of k search result,For the depth of the abstract of k-th of search result, k is just more than or equal to 1
Integer.
In one possible implementation, in step s 13, depth can be browsed according to the expectation of each search result,
Determine the expectation browsing initial depth of each search result.In this example, it is expected that browser is that initial depth can indicate in order
The depth of browsing, for example, in the abstract for having browsed first on searched page search result and first search result
After target pages, then the abstract of second search result and the target pages of second browsing result are browsed, browses the again later
The target pages etc. of abstract and third the browsing result of three search results, the expectation of some search result browse initial depth
For the summation of the depth of the abstract and target pages of all browsing results before browsing the search result.
In one possible implementation, depth is browsed according to the expectation of each search result, determines each search knot
The expectation of fruit browses initial depth, it may include (2) determine the expectation browsing initial depth of each search result according to the following formula:
Wherein, startkInitial depth, evh are browsed for the expectation of k-th of search resultiFor the expectation of i-th of search result
Browse depth, 1≤i≤k.
In one possible implementation, in step S14, it is deep starting can be browsed according to the expectation of each search result
It spends, depth, the correlation of each search result of the target pages of the depth of the abstract of each search result, each search result
Scoring and browsing depth distribution function, determine the income of each search result.
Fig. 3 is the flow chart of search evaluation method shown according to an exemplary embodiment.As shown in figure 3, the method
Further include:
In step S17, according to the browsing depth of the training searched page in training set, the browsing depth distribution is determined
Function, wherein the shown most deep position that the browsing depth is used to characterize the trained searched page is searched in training
Depth in the page.
In this example, can indicate train searched page shown most deep position corresponding to scroll bar rolling away from
From i.e. distance of the scroll bar apart from page top.The browsing depth distribution function can indicate the training of different browsing depth
Probability distribution of the searched page in all browsing pages of training set.
In one possible implementation, according to the browsing depth of the training searched page in training set, determine described in
Browse depth distribution function can include: between the quantity for determining the training searched page of the browsing depth and each browsing depth
Relation curve；The relation curve is fitted, determines the browsing depth distribution function.
In one possible implementation, browsing depth can be divided into multiple sections, and determine each training search
Section belonging to the browsing depth of the page, and the quantity of the training searched page in each section is counted, browsing depth can be drawn
Relation curve between the quantity of the training searched page of each browsing depth.
In one possible implementation, the relation curve is fitted, determines the browsing depth distribution letter
Number, it may include (3) are fitted relation curve according to the following formula, browse depth distribution function to determine:
Wherein, D (h) is the browsing depth distribution function, and h is the browsing depth, and Φ is the tired of standard gaussian distribution
Product distribution function, λ and μ are fitting parameter.
In one possible implementation, initial depth is browsed according to the expectation of each search result, each search is tied
The depth of the abstract of fruit, the depth of the target pages of each search result, the relevance score and browsing of each search result
Depth distribution function determines the income of each search result, and the receipts of each search result are determined including (4) according to the following formula
Benefit:
Wherein, dgkFor the income of k-th of search result, startkInitial depth is browsed for the expectation of k-th of search result,For the depth of the abstract of k-th of search result,For the depth of the target pages of k-th of search result, RkIt is searched for k-th
The relevance score of hitch fruit, D (h) are browsing depth distribution function.
In one possible implementation, each search result on searched page can be determined according to the above aspect
Income.
It in one possible implementation, can be according to the income of multiple search results, described in determination in step S15
The scoring of searched page.In this example, the income of all search results on searched page can be summed, and summation is tied
Fruit is determined as the scoring of searched page.
Search evaluation method according to an embodiment of the present disclosure, the probability being clicked by each search result are each searched
The depth of the target pages of the depth and search result of the abstract of hitch fruit can determine the expectation browsing depth of each search result,
And then determine that the expectation of each search result browses initial depth, further, it can be determined based on browsing depth distribution function every
The income of a search result, and then determine the scoring of searched page.In the scoring for determining searched page, by plucking for search result
The depth of the depth, target pages wanted and the browsing depth of training searched page, can preferably instead in view of in the design of index
Reflect the usage experience of user.
In this example, 50 search keys can be randomly selected, and this 50 search are closed using 4 search engines respectively
Keyword is inquired, and each search key obtains 4 searched pages, obtains 200 searched pages altogether.Each search key
4 searched pages constitute 3 searched page groups, each searched page group is made of two searched pages, in each search
In page group, a searched page comes from other 3 search engines from a fixed search engine, another searched page
In one, 50 search keys obtain 150 searched page groups.User can be obtained respectively to two in searched page group
The scoring of searched page, that is, in searched page group, the higher searched page of user's scoring is better user experience
Searched page.Further, method described in step S11- step S15 can be used to obtain to each search in searched page group
The scoring of the page, and choose the higher searched page that scores in searched page group.In this example, in 150 searched page groups,
According to the consistent of the higher page of scoring selected of method described in step S11- step S15 and the page of better user experience
Rate reaches 85.33%, and therefore, search evaluation method according to an embodiment of the present disclosure preferably reflects the usage experience of user.
Fig. 4 shows the block diagram of search evaluating apparatus shown according to an exemplary embodiment.As shown in figure 4, described device
Include:
Probability determination module 11, for necessary according to the relevance score of multiple search results in searched page and click
Property scoring, determine the probability that each search result is clicked, wherein the relevance score is for characterizing search key and institute
State the correlation between search result, the degree of perfection clicking necessity and scoring for characterizing the abstract of search result；
It is expected that depth determining module 12 is browsed, for the depth according to the abstract of each search result, each search result
Target pages depth and the probability that is clicked of each search result, determine that the expectation browsing of each search result is deep
Degree, wherein the target pages are the page jumped after search result is clicked；
It is expected that browsing initial depth determining module 13, for browsing depth according to the expectation of each search result, determine every
The expectation of a search result browses initial depth；
Income determining module 14, for browsing initial depth, each search result according to the expectation of each search result
The depth of abstract, the depth of the target pages of each search result, the relevance score of each search result and browsing depth
Distribution function determines the income of each search result；
The determining module 15 that scores determines the scoring of the described search page for the income according to multiple search results.
Fig. 5 shows the block diagram of search evaluating apparatus shown according to an exemplary embodiment.As shown in figure 5, described device
Further include:
Training probabilistic module 16, in the training search result for the training searched page in training set, determination has
The probability that various relevance scores and the training search result for clicking necessity scoring are clicked, wherein the trained search page
Face is corresponding with training search key.
In one possible implementation, the trained probabilistic module 16 is used for:
In the training search result of the trained searched page, determining has the first relevance score and the first click must
First quantity of the training search result of the property wanted scoring；
In the training search result for clicking necessity scoring with the first relevance score and first, determination is clicked
Search result the second quantity；
The ratio of second quantity and first quantity being determined as to, there is the first relevance score and first to click
The probability that the training search result of necessity scoring is clicked.
In one possible implementation, the expectation browsing depth determining module 12 is used for:
According to formulaDetermine the expectation browsing depth of each search result,
In, evhkDepth is browsed for the expectation of k-th search result, and P (C | Rk,Nk) it is the probability that k-th of search result is clicked, Rk
For the relevance score of k-th of search result, NkIt scores for the click necessity of k-th of search result,It is searched for for k-th
As a result the depth of target pages,For the depth of the abstract of k-th of search result.
In one possible implementation, the expectation browsing initial depth determining module 13 is used for:
According to formulaDetermine the expectation browsing initial depth of each search result, wherein
startkInitial depth, evh are browsed for the expectation of k-th of search resultiDepth is browsed for the expectation of i-th of search result.
In one possible implementation, in one possible implementation, described device further include:
Depth distribution function determination module 17 is browsed, for the browsing depth according to the training searched page in training set,
Determine the browsing depth distribution function, wherein the browsing depth is for characterizing the shown of the trained searched page
Depth of the most deep position in training searched page.
In one possible implementation, the browsing depth distribution function determination module 17 is used for:
Determine the relation curve between the quantity of the training searched page of the browsing depth and each browsing depth；
The relation curve is fitted, determines the browsing depth distribution function.
In one possible implementation, the relation curve is fitted, determines the browsing depth distribution letter
Number, comprising:
According to formulaIt is bent to the relationship
Line is fitted, and determines the browsing depth distribution function, wherein D (h) is the browsing depth distribution function, and h is described clear
Look at depth, Φ is the cumulative distribution function of standard gaussian distribution, and λ and μ are fitting parameter.
In one possible implementation, the income determining module 14 is used for:
According to formulaDetermine the income of each search result, wherein
dgkFor the income of k-th of search result, startkInitial depth is browsed for the expectation of k-th of search result,It is searched for k-th
The depth of the abstract of hitch fruit,For the depth of the target pages of k-th of search result, RkFor the correlation of k-th of search result
Property scoring, D (h) be browsing depth distribution function.
Fig. 6 shows the block diagram of the search evaluating apparatus 800 according to one embodiment of the disclosure.For example, device 800 can be shifting
Mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, Medical Devices, body-building are set
It is standby, personal digital assistant etc..
Referring to Fig. 6, device 800 may include following one or more components: processing component 802, memory 804, power supply
Component 806, multimedia component 808, audio component 810, the interface 812 of input/output (I/O), sensor module 814, and
Communication component 816.
The integrated operation of the usual control device 800 of processing component 802, such as with display, telephone call, data communication, phase
Machine operation and record operate associated operation.Processing component 802 may include that one or more processors 820 refer to execute
It enables, to perform all or part of the steps of the methods described above.In addition, processing component 802 may include one or more modules, just
Interaction between processing component 802 and other assemblies.For example, processing component 802 may include multi-media module, it is more to facilitate
Interaction between media component 808 and processing component 802.
Memory 804 is configured as storing various types of data to support the operation in device 800.These data are shown
Example includes the instruction of any application or method for operating on device 800, contact data, and telephone book data disappears
Breath, picture, video etc..Memory 804 can be by any kind of volatibility or non-volatile memory device or their group
It closes and realizes, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM) is erasable to compile
Journey read-only memory (EPROM), programmable read only memory (PROM), read-only memory (ROM), magnetic memory, flash
Device, disk or CD.
Power supply module 806 provides electric power for the various assemblies of device 800.Power supply module 806 may include power management system
System, one or more power supplys and other with for device 800 generate, manage, and distribute the associated component of electric power.
Multimedia component 808 includes the screen of one output interface of offer between described device 800 and user.One
In a little embodiments, screen may include liquid crystal display (LCD) and touch panel (TP).If screen includes touch panel, screen
Curtain may be implemented as touch screen, to receive input signal from the user.Touch panel includes one or more touch sensings
Device is to sense the gesture on touch, slide, and touch panel.The touch sensor can not only sense touch or sliding action
Boundary, but also detect duration and pressure associated with the touch or slide operation.In some embodiments, more matchmakers
Body component 808 includes a front camera and/or rear camera.When device 800 is in operation mode, such as screening-mode or
When video mode, front camera and/or rear camera can receive external multi-medium data.Each front camera and
Rear camera can be a fixed optical lens system or have focusing and optical zoom capabilities.
Audio component 810 is configured as output and/or input audio signal.For example, audio component 810 includes a Mike
Wind (MIC), when device 800 is in operation mode, when such as call mode, recording mode, and voice recognition mode, microphone is matched
It is set to reception external audio signal.The received audio signal can be further stored in memory 804 or via communication set
Part 816 is sent.In some embodiments, audio component 810 further includes a loudspeaker, is used for output audio signal.
I/O interface 812 provides interface between processing component 802 and peripheral interface module, and above-mentioned peripheral interface module can
To be keyboard, click wheel, button etc..These buttons may include, but are not limited to: home button, volume button, start button and lock
Determine button.
Sensor module 814 includes one or more sensors, and the state for providing various aspects for device 800 is commented
Estimate.For example, sensor module 814 can detecte the state that opens/closes of device 800, and the relative positioning of component, for example, it is described
Component is the display and keypad of device 800, and sensor module 814 can be with 800 1 components of detection device 800 or device
Position change, the existence or non-existence that user contacts with device 800,800 orientation of device or acceleration/deceleration and device 800
Temperature change.Sensor module 814 may include proximity sensor, be configured to detect without any physical contact
Presence of nearby objects.Sensor module 814 can also include optical sensor, such as CMOS or ccd image sensor, at
As being used in application.In some embodiments, which can also include acceleration transducer, gyro sensors
Device, Magnetic Sensor, pressure sensor or temperature sensor.
Communication component 816 is configured to facilitate the communication of wired or wireless way between device 800 and other equipment.Device
800 can access the wireless network based on communication standard, such as WiFi, 2G or 3G or their combination.In an exemplary implementation
In example, communication component 816 receives broadcast singal or broadcast related information from external broadcasting management system via broadcast channel.
In one exemplary embodiment, the communication component 816 further includes near-field communication (NFC) module, to promote short range communication.Example
Such as, NFC module can be based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra wide band (UWB) technology,
Bluetooth (BT) technology and other technologies are realized.
In the exemplary embodiment, device 800 can be believed by one or more application specific integrated circuit (ASIC), number
Number processor (DSP), digital signal processing appts (DSPD), programmable logic device (PLD), field programmable gate array
(FPGA), controller, microcontroller, microprocessor or other electronic components are realized, for executing the above method.
In the exemplary embodiment, a kind of non-volatile computer readable storage medium storing program for executing is additionally provided, for example including calculating
The memory 804 of machine program instruction, above-mentioned computer program instructions can be executed above-mentioned to complete by the processor 820 of device 800
Method.
Fig. 7 shows the block diagram of the search evaluating apparatus 1900 according to one embodiment of the disclosure.For example, device 1900 can be by
It is provided as a server.Referring to Fig. 7, it further comprises one or more processors that device 1900, which includes processing component 1922,
And memory resource represented by a memory 1932, for store can by the instruction of the execution of processing component 1922, such as
Application program.The application program stored in memory 1932 may include it is one or more each correspond to one group refer to
The module of order.In addition, processing component 1922 is configured as executing instruction, to execute the above method.
Device 1900 can also include that a power supply module 1926 be configured as the power management of executive device 1900, and one
Wired or wireless network interface 1950 is configured as device 1900 being connected to network and input and output (I/O) interface
1958.Device 1900 can be operated based on the operating system for being stored in memory 1932, such as Windows ServerTM, Mac
OS XTM, UnixTM, LinuxTM, FreeBSDTM or similar.
In the exemplary embodiment, a kind of non-volatile computer readable storage medium storing program for executing is additionally provided, for example including calculating
The memory 1932 of machine program instruction, above-mentioned computer program instructions can be executed by the processing component 1922 of device 1900 to complete
The above method.
The disclosure can be system, method and/or computer program product.Computer program product may include computer
Readable storage medium storing program for executing, containing for making processor realize the computer-readable program instructions of various aspects of the disclosure.
Computer readable storage medium, which can be, can keep and store the tangible of the instruction used by instruction execution equipment
Equipment.Computer readable storage medium for example can be-but be not limited to-storage device electric, magnetic storage apparatus, optical storage set
Standby, electric magnetic storage apparatus, semiconductor memory apparatus or above-mentioned any appropriate combination.Computer readable storage medium is more
Specific example (non exhaustive list) includes: portable computer diskette, hard disk, random access memory (RAM), read-only storage
Device (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable pressure
Contracting disk read-only memory (CD-ROM), memory stick, floppy disk, mechanical coding equipment, is for example deposited digital versatile disc (DVD) thereon
Contain punch card or groove internal projection structure and the above-mentioned any appropriate combination of instruction.Computer used herein above
Readable storage medium storing program for executing is not interpreted that instantaneous signal itself, the electromagnetic wave of such as radio wave or other Free propagations pass through
The electromagnetic wave (for example, the light pulse for passing through fiber optic cables) or pass through electric wire transmission that waveguide or other transmission mediums are propagated
Electric signal.
Computer-readable program instructions as described herein can be downloaded to from computer readable storage medium it is each calculate/
Processing equipment, or outer computer or outer is downloaded to by network, such as internet, local area network, wide area network and/or wireless network
Portion stores equipment.Network may include copper transmission cable, optical fiber transmission, wireless transmission, router, firewall, interchanger, gateway
Computer and/or Edge Server.Adapter or network interface in each calculating/processing equipment are received from network to be counted
Calculation machine readable program instructions, and the computer-readable program instructions are forwarded, for the meter being stored in each calculating/processing equipment
In calculation machine readable storage medium storing program for executing.
Computer program instructions for executing disclosure operation can be assembly instruction, instruction set architecture (ISA) instructs,
Machine instruction, machine-dependent instructions, microcode, firmware instructions, condition setup data or with one or more programming languages
The source code or object code that any combination is write, the programming language include the programming language-of object-oriented such as
Smalltalk, C++ etc., and conventional procedural programming languages-such as " C " language or similar programming language.Computer
Readable program instructions can be executed fully on the user computer, partly execute on the user computer, be only as one
Vertical software package executes, part executes on the remote computer or completely in remote computer on the user computer for part
Or it is executed on server.In situations involving remote computers, remote computer can pass through network-packet of any kind
It includes local area network (LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as benefit
It is connected with ISP by internet).In some embodiments, by utilizing computer-readable program instructions
Status information carry out personalized customization electronic circuit, such as programmable logic circuit, field programmable gate array (FPGA) or can
Programmed logic array (PLA) (PLA), the electronic circuit can execute computer-readable program instructions, to realize each side of the disclosure
Face.
Referring herein to according to the flow chart of the method, apparatus (system) of the embodiment of the present disclosure and computer program product and/
Or block diagram describes various aspects of the disclosure.It should be appreciated that flowchart and or block diagram each box and flow chart and/
Or in block diagram each box combination, can be realized by computer-readable program instructions.
These computer-readable program instructions can be supplied to general purpose computer, special purpose computer or other programmable datas
The processor of processing unit, so that a kind of machine is produced, so that these instructions are passing through computer or other programmable datas
When the processor of processing unit executes, function specified in one or more boxes in implementation flow chart and/or block diagram is produced
The device of energy/movement.These computer-readable program instructions can also be stored in a computer-readable storage medium, these refer to
It enables so that computer, programmable data processing unit and/or other equipment work in a specific way, thus, it is stored with instruction
Computer-readable medium then includes a manufacture comprising in one or more boxes in implementation flow chart and/or block diagram
The instruction of the various aspects of defined function action.
Computer-readable program instructions can also be loaded into computer, other programmable data processing units or other
In equipment, so that series of operation steps are executed in computer, other programmable data processing units or other equipment, to produce
Raw computer implemented process, so that executed in computer, other programmable data processing units or other equipment
Instruct function action specified in one or more boxes in implementation flow chart and/or block diagram.
The flow chart and block diagram in the drawings show system, method and the computer journeys according to multiple embodiments of the disclosure
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
One module of table, program segment or a part of instruction, the module, program segment or a part of instruction include one or more use
The executable instruction of the logic function as defined in realizing.In some implementations as replacements, function marked in the box
It can occur in a different order than that indicated in the drawings.For example, two continuous boxes can actually be held substantially in parallel
Row, they can also be executed in the opposite order sometimes, and this depends on the function involved.It is also noted that block diagram and/or
The combination of each box in flow chart and the box in block diagram and or flow chart, can the function as defined in executing or dynamic
The dedicated hardware based system made is realized, or can be realized using a combination of dedicated hardware and computer instructions.
The presently disclosed embodiments is described above, above description is exemplary, and non-exclusive, and
It is not limited to disclosed each embodiment.Without departing from the scope and spirit of illustrated each embodiment, for this skill
Many modifications and changes are obvious for the those of ordinary skill in art field.The selection of term used herein, purport
In the principle, practical application or technological improvement to the technology in market for best explaining each embodiment, or lead this technology
Other those of ordinary skill in domain can understand each embodiment disclosed herein.
Claims (10)
1. a kind of search evaluation method characterized by comprising
According to the relevance score of multiple search results in searched page and necessity scoring is clicked, determines each search result
The probability being clicked, wherein the relevance score is used to characterize the correlation between search key and described search result,
The degree of perfection clicking necessity and scoring for characterizing the abstract of search result；
According to the depth of the abstract of each search result, the depth of the target pages of each search result and each search
As a result the probability being clicked, determine each search result expectation browsing depth, wherein the target pages be search result by
The page jumped after to click；
Depth is browsed according to the expectation of each search result, determines the expectation browsing initial depth of each search result；
Initial depth, the depth of the abstract of each search result, each search result are browsed according to the expectation of each search result
The depth of target pages, the relevance score of each search result and browsing depth distribution function, determine that each search is tied
The income of fruit；
According to the income of multiple search results, the scoring of the described search page is determined.
2. the method according to claim 1, wherein the method also includes:
In the training search result of training searched page in training set, determining has various relevance scores and clicks necessary
Property scoring the probability that is clicked of training search result, wherein the trained searched page is corresponding with trained search key.
3. according to the method described in claim 2, it is characterized in that, the training search knot of the training searched page in training set
In fruit, the probability that there are various relevance scores and the training search result of click necessity scoring to be clicked is determined, comprising:
In the training search result of the trained searched page, determining, there is the first relevance score and first to click necessity
First quantity of the training search result of scoring；
In the training search result for clicking necessity scoring with the first relevance score and first, determine by the instruction clicked
Practice the second quantity of search result；
The ratio of second quantity and first quantity being determined as to, there is the first relevance score and first to click necessity
Property scoring the probability that is clicked of training search result.
4. according to the method described in claim 2, it is characterized in that, the method also includes:
According to the browsing depth of the training searched page in training set, the browsing depth distribution function is determined, wherein described clear
Depth of looking at is used to characterize depth of the shown most deep position of the trained searched page in training searched page.
5. according to the method described in claim 4, it is characterized in that, deep according to the browsing of the training searched page in training set
Degree, determines the browsing depth distribution function, comprising:
Determine the relation curve between the quantity of the training searched page of the browsing depth and each browsing depth；
The relation curve is fitted, determines the browsing depth distribution function.
6. according to the method described in claim 5, determining the browsing it is characterized in that, be fitted to the relation curve
Depth distribution function, comprising:
According to formulaTo the relation curve into
Row fitting, determines the browsing depth distribution function, wherein D (h) is the browsing depth distribution function, and h is that the browsing is deep
Degree, Φ are the cumulative distribution function of standard gaussian distribution, and λ and μ are fitting parameter.
7. the method according to claim 1, wherein according to the depth of the abstract of each search result, each searching
The probability that the depth of the target pages of hitch fruit and each search result are clicked, determines the expectation of each search result
Browse depth, comprising:
According to formulaDetermine the expectation browsing depth of each search result, wherein evhk
Depth is browsed for the expectation of k-th search result, and P (C | Rk,Nk) it is the probability that k-th of search result is clicked, RkIt is k-th
The relevance score of search result, NkIt scores for the click necessity of k-th of search result,For the mesh of k-th of search result
The depth of the page is marked,For the depth of the abstract of k-th of search result.
8. being determined the method according to claim 1, wherein browsing depth according to the expectation of each search result
The expectation of each search result browses initial depth, comprising:
According to formulaDetermine the expectation browsing initial depth of each search result, wherein startkFor
The expectation of k-th of search result browses initial depth, evhiDepth is browsed for the expectation of i-th of search result.
9. the method according to claim 1, wherein according to the expectation of each search result browse initial depth,
The depth of the abstract of each search result, the depth of the target pages of each search result, the correlation of each search result are commented
Divide and browse depth distribution function, determines the income of each search result, comprising:
According to formulaDetermine the income of each search result, wherein dgkFor
The income of k-th of search result, startkInitial depth is browsed for the expectation of k-th of search result,For k-th of search knot
The depth of the abstract of fruit,For the depth of the target pages of k-th of search result, RkCorrelation for k-th of search result is commented
Point, D (h) is browsing depth distribution function.
10. a kind of search evaluating apparatus characterized by comprising
Probability determination module, for being commented according to the relevance score and click necessity of multiple search results in searched page
Point, determine the probability that each search result is clicked, wherein the relevance score is for characterizing search key and described searching
Correlation between hitch fruit, the degree of perfection clicking necessity and scoring for characterizing the abstract of search result；
It is expected that depth determining module is browsed, for depth, the target of each search result according to the abstract of each search result
The probability that the depth of the page and each search result are clicked determines the expectation browsing depth of each search result,
In, the target pages are the page jumped after search result is clicked；
It is expected that browsing initial depth determining module, for browsing depth according to the expectation of each search result, each search is determined
As a result expectation browses initial depth；
Income determining module browses the abstract of initial depth, each search result for the expectation according to each search result
Depth, the depth of the target pages of each search result, the relevance score of each search result and browsing depth distribution letter
Number, determines the income of each search result；
The determining module that scores determines the scoring of the described search page for the income according to multiple search results.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810885991.4A CN109086417A (en) | 2018-08-06 | 2018-08-06 | Search for evaluation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810885991.4A CN109086417A (en) | 2018-08-06 | 2018-08-06 | Search for evaluation method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109086417A true CN109086417A (en) | 2018-12-25 |
Family
ID=64834063
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810885991.4A Pending CN109086417A (en) | 2018-08-06 | 2018-08-06 | Search for evaluation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109086417A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112100528A (en) * | 2020-09-09 | 2020-12-18 | 北京三快在线科技有限公司 | Method, device, equipment and medium for training search result scoring model |
| CN114020957A (en) * | 2021-11-08 | 2022-02-08 | 杭州网易云音乐科技有限公司 | Search algorithm evaluation method and device, computing equipment and medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060069982A1 (en) * | 2004-09-30 | 2006-03-30 | Microsoft Corporation | Click distance determination |
| CN107679082A (en) * | 2017-08-31 | 2018-02-09 | 阿里巴巴集团控股有限公司 | Question and answer searching method, device and electronic equipment |
-
2018
- 2018-08-06 CN CN201810885991.4A patent/CN109086417A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060069982A1 (en) * | 2004-09-30 | 2006-03-30 | Microsoft Corporation | Click distance determination |
| CN107679082A (en) * | 2017-08-31 | 2018-02-09 | 阿里巴巴集团控股有限公司 | Question and answer searching method, device and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| CHENG LUO,YIQUN LIU 等: ""Evaluating Mobile Search with Height-Biased Gain"", 《PROCEEDING OF THE 40TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112100528A (en) * | 2020-09-09 | 2020-12-18 | 北京三快在线科技有限公司 | Method, device, equipment and medium for training search result scoring model |
| CN114020957A (en) * | 2021-11-08 | 2022-02-08 | 杭州网易云音乐科技有限公司 | Search algorithm evaluation method and device, computing equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109189987A (en) | Video searching method and device | |
| US11520824B2 (en) | Method for displaying information, electronic device and system | |
| CN104572905B (en) | Print reference creation method, photo searching method and device | |
| CN106708282B (en) | A kind of recommended method and device, a kind of device for recommendation | |
| CN105787102B (en) | Searching method, device and the device for search | |
| CN109800325A (en) | Video recommendation method, device and computer readable storage medium | |
| CN108932253A (en) | Multimedia search result methods of exhibiting and device | |
| CN108780374A (en) | User interface for multivariable search | |
| CN109241301A (en) | Resource recommendation method and device | |
| CN109151548A (en) | Interface alternation method and device | |
| CN108227950A (en) | A kind of input method and device | |
| CN108959320A (en) | The method and apparatus of preview video search result | |
| CN108038102A (en) | Recommendation method, apparatus, terminal and the storage medium of facial expression image | |
| CN110019675A (en) | A kind of method and device of keyword extraction | |
| CN107423296A (en) | Searching method, device and the device for search | |
| CN105607757A (en) | Input method and device and device used for input | |
| CN109783656A (en) | Recommended method, system and the server and storage medium of audio, video data | |
| CN108717403B (en) | Processing method and device for processing | |
| CN108320208A (en) | Vehicle recommends method and device | |
| CN110209877A (en) | Video analysis method and device | |
| CN110020106A (en) | A kind of recommended method, recommendation apparatus and the device for recommendation | |
| CN106815291A (en) | Search result items exhibiting method, device and the device represented for search result items | |
| CN104572848B (en) | Searching method based on browser and device | |
| CN111752436A (en) | Recommendation method and device and recommendation device | |
| CN109086417A (en) | Search for evaluation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181225 |