CN107704467A - Search quality appraisal procedure and device - Google Patents
Search quality appraisal procedure and device Download PDFInfo
- Publication number
- CN107704467A CN107704467A CN201610645103.2A CN201610645103A CN107704467A CN 107704467 A CN107704467 A CN 107704467A CN 201610645103 A CN201610645103 A CN 201610645103A CN 107704467 A CN107704467 A CN 107704467A
- Authority
- CN
- China
- Prior art keywords
- search
- search result
- user
- assessed
- ranking results
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
Abstract
The embodiment of the invention discloses a kind of search quality appraisal procedure and device, methods described to include：Database is assessed based on user's history search record data construction search quality, the search quality assesses database and includes at least one searching keyword, corresponding at least one search result items and user's operating characteristics data for the search result items；For each searching keyword at least one searching keyword, based on baseline search strategy and search strategy to be assessed, the corresponding search result items in database are assessed to the search quality respectively and are ranked up to obtain the first ranking results and the second ranking results；The search quality of the search strategy to be assessed is assessed based on the user's history operating characteristics data, first ranking results and second ranking results.The embodiment of the present invention makes whole evaluation process full automation, to reduce cost of labor and Human disturbance degree, and improves search quality and assesses the degree of accuracy.
Description
Technical field
The present embodiments relate to information search technique field, more particularly to a kind of search quality appraisal procedure and device.
Background technology
In current Internet era, information content rapid expansion on network increases, user want to directly find needed for substantially
It is to look for a needle in a haystack；The appearance of search engine, people can be accurately and rapidly helped to obtain required letter from mass network data
Breath.The sequence effect of Optimizing Search engine, allow user fast find needed for be search engine mission.Therefore how quickly to searching
Indexing the search quality progress offline evaluation held up becomes a popular research field, plays in search technique research and development important
Role.
Search-engine results it is whether fine or not, embody it is in the industry cycle alleged in correlation (Relevance).The solution of narrow sense
Releasing is：Retrieval result and the degree of correlation of user's inquiry.At present, the conventional offline evaluation of industry has artificial mark calculating to recall
Rate, accuracy rate is recalled, search engine quality index (Discounted Cumulative Gain, DCG) is weighed, weighs sequence matter
The indexs such as the index (Normalized Discounted Cumulative Gain, NDCG) of amount.Because these indexs are all bases
Evaluate in artificial labeled data, influenceed by subjective factor, data volume, cause actual effect on evaluation result and line to differ very
Far, and cost of labor is higher.
The content of the invention
The embodiment of the present invention provides a kind of search quality appraisal procedure and device, makes whole evaluation process full automation,
To reduce cost of labor and Human disturbance degree, and improve search quality and assess the degree of accuracy.
In a first aspect, the embodiments of the invention provide a kind of search quality appraisal procedure, including：
Database is assessed based on user's history search record data construction search quality, the search quality assesses database
Include at least one searching keyword, corresponding at least one search result items and the user behaviour for the search result items
Make characteristic；
For each searching keyword at least one searching keyword, based on baseline search strategy and to be assessed
Search strategy, the corresponding search result items in database are assessed to the search quality respectively and are ranked up to obtain the first sequence knot
Fruit and the second ranking results；
Based on the user's history operating characteristics data, first ranking results and second ranking results to described
The search quality of search strategy to be assessed is assessed.
Second aspect, the embodiment of the present invention additionally provide a kind of search quality apparatus for evaluating, including：
Database construction module is assessed, for assessing data based on user's history search record data construction search quality
Storehouse, the search quality assess database include at least one searching keyword, corresponding at least one search result items and
For user's operating characteristics data of the search result items；
Order module, for for each searching keyword at least one searching keyword, being searched based on baseline
Rope strategy and search strategy to be assessed, the corresponding search result items in database are assessed to the search quality respectively and are ranked up
Obtain the first ranking results and the second ranking results；
Evaluation module, for based on the user's history operating characteristics data, first ranking results and described second
Ranking results are assessed the search quality of the search strategy to be assessed.
The embodiment of the present invention includes at least one searching keyword, right by searching for record data construction based on user's history
The search quality assessment of at least one search result items answered and user's operating characteristics data for the search result items
Database, and for each searching keyword at least one searching keyword, based on baseline search strategy and to be evaluated
Estimate search strategy, assessing the corresponding search result items in database to the search quality respectively is ranked up to obtain the first sequence
As a result with the second ranking results, based on the user's history operating characteristics data, first ranking results and the second row
Sequence result is assessed the search quality of the search strategy to be assessed, makes whole evaluation process full automation, to reduce
Cost of labor and Human disturbance degree, and improve search quality and assess the degree of accuracy.
Brief description of the drawings
Fig. 1 is a kind of flow chart of search quality appraisal procedure in the embodiment of the present invention one；
Fig. 2 is a kind of flow chart of search quality appraisal procedure in the embodiment of the present invention two；
Fig. 3 is a kind of flow chart of search quality appraisal procedure in the embodiment of the present invention three；
Fig. 4 is a kind of flow chart of search quality appraisal procedure in the embodiment of the present invention four；
Fig. 5 is a kind of structure chart of search quality apparatus for evaluating in the embodiment of the present invention five.
Embodiment
The present invention is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining the present invention, rather than limitation of the invention.It also should be noted that in order to just
Part related to the present invention rather than entire infrastructure are illustrate only in description, accompanying drawing.
Embodiment one
Fig. 1 is a kind of flow chart for search quality appraisal procedure that the embodiment of the present invention one provides, and the present embodiment is applicable
Situation about being assessed when search engine is being developed the search quality of search strategy to be assessed, this method can be by the present invention
The search quality apparatus for evaluating that embodiment provides performs, and the device can be integrated in fixed terminal (for example, desktop computer or notes
This computer) or server in, as shown in figure 1, specifically including：
S101, search quality assessment database, the search quality assessment are constructed based on user's history search record data
Database includes at least one searching keyword, corresponding at least one search result items and for the search result items
User's operating characteristics data.
Wherein, user's history search record data includes but is not limited to user's click logs.In user's click logs
It has recorded click situation of the user for the search result items under each searching keyword, including but not limited to number of clicks.Institute
State user's operating characteristics data and include but is not limited to click data of the user for the search result items (for example, clicking on secondary
Number).The search result items can be but be not limited to it is following any one：The link address URL that is extracted from webpage, browse net
Page or the other web page characteristics extracted from webpage.
Specifically, webpage can be extracted using crawler technology from internet, and user's history search behavior is based on, obtains and use
From user's click logs are randomly selected to the history click logs of the webpage of extraction by preset flow (such as 2%) for family
Searching keyword, user's operating characteristics data of the search result items included in webpage corresponding to the searching keyword are recalled,
And based on the characteristic information of webpage extraction search result items.Searching keyword, search result items based on extraction, search result items
Characteristic information and user operating characteristics data three between corresponding relation, construction search quality assess database.Wherein, institute
Stating the characteristic information of search result items can obtain beforehand through machine learning, reference when being sorted as the search result items because
One of element.
S102, for each searching keyword at least one searching keyword, based on baseline search strategy and
Search strategy to be assessed, the corresponding search result items in database are assessed to the search quality respectively and are ranked up to obtain first
Ranking results and the second ranking results.
Wherein, the baseline search strategy is the search strategy run on existing line, and the search strategy to be assessed is
Search strategy to be assessed on the basis of the search strategy run on existing line.
Specifically, each searching keyword being primarily based at least one searching keyword enters in a search engine
Row search, assessed from the search quality and search result items corresponding with each searching keyword and search knot are obtained in database
The characteristic information of fruit item, the characteristic information and the baseline search strategy for being then based on search result items are carried out to search result items
Sequence then obtains the first ranking results, and the characteristic information based on search result items and the search strategy to be assessed are tied to search
Fruit item is ranked up, and obtains the second ranking results.
S103, based on the user's history operating characteristics data, first ranking results and second ranking results
The search quality of the search strategy to be assessed is assessed.
Specifically, when first ranking results are consistent with second ranking results, then illustrate described to be assessed search
The search quality of rope strategy does not get a promotion.When first ranking results and inconsistent second ranking results,
Then further the search quality of the search strategy to be assessed is assessed according to the user's history operating characteristics data.
The present embodiment includes at least one searching keyword, corresponding by searching for record data construction based on user's history
At least one search result items and for the search result items user's operating characteristics data search quality assess data
Storehouse, and for each searching keyword at least one searching keyword, based on baseline search strategy and to be assessed search
Rope strategy, the corresponding search result items in database are assessed to the search quality respectively and are ranked up to obtain the first ranking results
With the second ranking results, tied based on the user's history operating characteristics data, first ranking results and second sequence
Fruit is assessed the search quality of the search strategy to be assessed, makes whole evaluation process full automation, artificial to reduce
Cost and Human disturbance degree, and improve search quality and assess the degree of accuracy.
On the basis of above-described embodiment, database bag is assessed based on user's history search record data construction search quality
Include：
Searched for from user's history in record data and extract at least one searching keyword；
Searched for from user's history in record data, extract the use of search result items corresponding with the searching keyword extracted
Family operating characteristics data；
The user of at least one searching keyword, corresponding search result items and the search result items is operated
Characteristic assesses data as the search quality.
Wherein, the user's history search record data includes but is not limited to user's click logs, and the user operates special
Sign data include but is not limited to user's history click data.
Specifically, searching for record data by excavating user's history, user browsing behavior construction search matter is therefrom extracted
Amount assesses database.For example, randomly selecting the searching keyword of 2% flow, user corresponding to the searching keyword is filtered out
Click logs, recall user's operating characteristics number of at least one search result items and search result items under each searching keyword
According to using user's operating characteristics data of the searching keyword and its corresponding search result items and search result items as searching
Rope quality assessment data storehouse.When scanning for quality evaluation, it is primarily based on searching keyword and scans in a search engine,
And assessed from the search quality in database and obtain at least one search result items corresponding with the searching keyword, then
Based on baseline search strategy and search strategy to be assessed, the corresponding search result in database is assessed to the search quality respectively
Item is ranked up to obtain the first ranking results and the second ranking results.
On the basis of above-described embodiment, searched for from user's history in record data, extraction is closed with the inquiry extracted
After user's operating characteristics data of search result items corresponding to keyword, methods described also includes：
Based on predetermined search result items screening strategy, Screening Treatment is carried out to the search result items, wherein described search
Rope result items screening strategy is associated with user's operating characteristics data.
Wherein, the predetermined search result items screening strategy is preferably to recall strategy.
Specifically, the present embodiment is said so that user's operating characteristics data are user's history click data as an example
It is bright.The searching keyword for extracting preset ratio (such as 2%) is searched in record data in user's history, user is obtained and clicks on day
Will, for example, it is as follows that user's history click behavior is have recorded in the click logs：
A little whether searching keyword query- search result items url-
query-url1-click
query-url2-no_click
query-url3-click
query-url4-click
query-url5-no_click
query-url6-no_click
Behavior is clicked on according to the user of above-mentioned record and recalls strategy, statistics obtains the history of position difference search result pair
Click data.Wherein, the strategy of recalling includes but is not limited to following at least one：(1) when merely clicking on first, u1 is recalled>
U2, u1>u3；(2) nth bar a little, (n+1)th no point when, recall un>un+1；(3) it is better than skipping to click on, nth bar a little, the
When n-1 bars do not have point, 1 un is recalled>un-1；A little, (n+1)th does not have a little nth bar, and the n-th+2 a little, recalls 1 un+2>un+
1；(4) adopting consecutive click chemical reaction or to skip to tie, a little, (n-1)th does not have a little nth bar, and the n-th -2 do not have a little, recall un-2=un+1, and
A little, (n+1)th does not have a little n bars, and the n-th+2 do not have a little, recall un=un+1, un+1=un+2.Obtained according to above-mentioned strategy of recalling
At least one search result items corresponding to searching keyword, the click behavior to each search result items and number of clicks are entered successively
Row statistics.For example, above-mentioned url either with or without being clicked, be clicked it is how many inferior.According to the click of above-mentioned each search result items time
Number construction location difference search result pair.For example, search result items 1 are clicked 10 times, search result items 2 are clicked 20 times, then
The position difference search result of construction to for：Search result items 2 are clicked on 10 times than search result items more than 1.
Or construction location difference search result pair, for each described position difference search result centering, unite respectively
Count total time that the situation of the user's history number of clicks more than another search result items of one of search result items occurs
Number awin；The situation for counting the user's history number of clicks less than another search result items of one of search result items goes out
Existing total degree bwin；Count the user's history number of clicks equal to another search result items of one of search result items
Situation occur total degree sum.
For example, ultimately constructed search quality assesses database as shown in following table one：
Table one
| Searching keyword | Position difference search result pair | [awinbwin sum] |
| Searching keyword 1 | Search result items 1- search result items 2 | [001] |
| Searching keyword 1 | Search result items 2- search result items 3 | [100] |
| Searching keyword 1 | Search result items 3- search result items 4 | [010] |
| Searching keyword 1 | Search result items 4- search result items 5 | [001] |
| …… | …… | …… |
On the basis of above-described embodiment, for each searching keyword at least one searching keyword, base
In baseline search strategy and search strategy to be assessed, the corresponding search result items in database are assessed to the search quality respectively
Being ranked up to obtain the first ranking results and the second ranking results includes：
For each searching keyword at least one searching keyword, the page for corresponding to search result items is extracted
Feature；
From the page feature extracted, baseline policy characteristics and policy characteristics to be assessed are obtained；
Based on the baseline policy characteristics and policy characteristics to be assessed and corresponding baseline Policy model and plan to be assessed
The search result items are ranked up to obtain first ranking results and second ranking results by slightly model.
On the basis of above-described embodiment, based on the user's history operating characteristics data, first ranking results and
Second ranking results carry out assessment to the search quality of the search strategy to be assessed to be included：
Based on first ranking results and the second ranking results, determine that same sorting position be present corresponds to different search
The sorting position of result items situation；
Different search result items on the sorting position are formed into position difference search result pair；
Based on the position difference search result pair and corresponding user's operating characteristics data, to the search to be assessed
Strategy is assessed.
If specifically, in first ranking results and second ranking results, at all identical sorting positions
Search result items all same, it is determined that the search quality of the search strategy to be assessed does not get a promotion.If described first
In ranking results and second ranking results, corresponding search result items difference at least one identical sorting position be present,
The different search result items on the sorting position are then formed into position difference search result pair, searched for based on the position difference
As a result pair and corresponding user's operating characteristics data, the search strategy to be assessed is assessed.
If for example, were search result items 1 at a certain sorting position in first ranking results, second sequence
As a result at same sorting position it is search result items 2 in, then the search result items 1 and the search result items 2 is formed into position
Difference search result pair is put, counting user clicks on behavior to the search result items 1 and the history of search result items 2 respectively, if
The history of search result items 1 clicks on behavior and is more than search result items 2, then illustrates that the search quality of the search strategy to be assessed obtains
To lifting, otherwise illustrate that the search quality of the search strategy to be assessed does not get a promotion.
On the basis of above-described embodiment, based on the difference search result pair and corresponding user's operating characteristics number
According to carrying out assessment to the search strategy to be assessed includes：
Based on each position difference search result pair and corresponding user's operating characteristics data, simulation search environment is determined
In each click when the search result that is hit of the difference search result centering；
The click result of each search result of each position difference search result centering under statistical simulation search environment；
Based on the click result counted, the search strategy to be assessed is assessed.
A kind of implementation therein of the present embodiment is each position difference search result under statistical simulation search environment
The number of clicks of each search result of centering, under identical sorting position, if the search knot in first ranking results
The number of clicks of fruit item is less than the number of clicks of the search result items in second ranking results, then illustrates described to be assessed search
The search quality of rope strategy gets a promotion, and otherwise illustrates that the search quality of the search strategy to be assessed is not lifted.
Another implementation of the present embodiment is that the search result items in statistics the first ranking results of click are often
In the total degree G for the number for clicking on the search result items in the second ranking results；Statistics clicks on the search in the first ranking results
Total degree B of the number of result items less than the number for clicking on the search result items in the second ranking results；Statistics clicks on first row
The number of search result items in sequence result is equal to the total degree S for the number for clicking on the search result items in the second ranking results.
Based on number G, B and the S counted, the metewand of the calculating search strategy to be assessed.
Specifically, metewand P can be calculated by formula P=0.5* (G-B)/(G+S+B), if P is positive number,
Illustrate that the search quality of the search strategy to be assessed has been lifted, otherwise do not lifted.
When the searching keyword is multiple, by same position difference search result under all searching keywords under
Total degree G, total degree S and total degree B are summed up respectively, based on add and after total degree G, plus and after total degree S and add
Total degree B with after obtains metewand P, the search quality based on obtained metewand P to the search strategy to be assessed
Assessed.
When the sorting position comprising different search result items is multiple, the position difference on all sorting positions is searched for
As a result the total degree G under, total degree S and total degree B are summed up respectively, based on add and after total degree G, plus and after it is total
Number S and plus and after total degree B obtain metewand P, based on obtained metewand P to the search strategy to be assessed
Search quality is assessed.
On the basis of above-described embodiment, if based on each position difference search result pair and corresponding user operation
Characteristic, determine the search result bag that the difference search result centering is hit during each click in simulation search environment
Include：
Based on user's operating characteristics data, described one search result items of position difference search result centering are counted respectively
User's history number of clicks is more than, is less than and occurred equal to the situation of the user's history number of clicks of another search result items
Total degree awin, bwin and sum；
Calculated based on described awin, bwin and sum the probability of clicking on the search result in first ranking results and
Click on the probability of the search result in the second ranking results；
Based on the probability for clicking on the search result in first ranking results and click on the search in the second ranking results
As a result probability, using random digit generation method, position difference search during each click in simulation search environment is determined
As a result the search result that centering is hit.
It is special being operated based on each position difference search result pair and corresponding user on the basis of above-described embodiment
Data are levied, determine the search result that the position difference search result centering is hit during each click in simulation search environment
Before, methods described also includes：
Probability is clicked on based on predeterminated position, using random digit generation method, determines the position difference search result to being
No generation click behavior, and when click behavior occurs, based on each position difference search result pair and corresponding user
Operating characteristics data, determine that what position difference search result centering during each click in simulation search environment hit searches
Hitch fruit.
If specifically, in the first ranking results positioned at second row tagmeme put for search result items 1, in the second sequence
As a result in positioned at second row tagmeme put for search result items 2, then form position based on search result items 1 and search result items 2
Difference search result pair, based on above-mentioned search quality assess data count respectively the position difference search result pair awin,
Bwin and sum.If awin, bwin and sum are respectively 50,150 and 200, then the click probability for obtaining search result items 1 is
50/ (50+150+200)=12.5%, the click probability of search result items 2 is 150/ (50+150+200)=37.5%, then gives birth to
Into one group of random number, if, the random number of generation is 1,2,3,4,5,6,7,8,9,10；Then according to click probability 12.5% and point
Hit probability 37.5% to be grouped random number, because 12.5% and 37.5% ratio is 1:3, then 1,2,3 is classified as searching for
The corresponding random number of result items 1,4,5,6,7,8,9,10 are classified as the corresponding random number of search result items 2, then at random from
A random number is extracted in this group of random number 1,2,3,4,5,6,7,8,9,10, if the random number is pair of search result items 1
The random number answered, then the search result items in the first ranking results for clicking on result to click on are search result items 1, if should be with
Machine number is the corresponding random number of search result items 2, then the search result items in the second ranking results for clicking on result to click on
That is search result items 2.The total number of clicks of the first ranking results and the second ranking results is finally counted, according to total number of clicks to described
The search quality of search strategy to be assessed is assessed.For example, if the total number of clicks of the second ranking results is more than the first sequence
As a result, it is determined that the search quality of the search strategy to be assessed has been lifted, and is not otherwise got a promotion.
Above-described embodiment is constructed comprising at least one searching keyword, correspondingly by searching for record data based on user's history
At least one search result items and for the search result items user's operating characteristics data search quality assess number
According to storehouse, and for each searching keyword at least one searching keyword, based on baseline search strategy and to be assessed
Search strategy, the corresponding search result items in database are assessed to the search quality respectively and are ranked up to obtain the first sequence knot
Fruit and the second ranking results, based on the user's history operating characteristics data, first ranking results and second sequence
As a result the search quality of the search strategy to be assessed is assessed, makes whole evaluation process full automation, to reduce people
Work cost and Human disturbance degree, and improve search quality and assess the degree of accuracy.
Embodiment two
Fig. 2 is a kind of flow chart for search quality appraisal procedure that the embodiment of the present invention two provides, and the present embodiment is above-mentioned
On the basis of embodiment, it is preferably to be gone through from user to assess database based on user's history search record data construction search quality
At least one searching keyword is extracted in history search record data；Searched for from user's history in record data, extraction and institute
User's operating characteristics data of search result items corresponding to the searching keyword of extraction；By at least one searching keyword,
User's operating characteristics data of corresponding search result items and the search result items assess data as the search quality.
Searched for from user's history in record data, the user's operation for extracting search result items corresponding with the searching keyword extracted
After characteristic, methods described further preferably includes：Based on predetermined search result items screening strategy, the search result items are entered
Row Screening Treatment, wherein the search result items screening strategy is associated with user's operating characteristics data.It is as shown in Fig. 2 specific
Including：
S201, searched for from user's history in record data and extract at least one searching keyword, and extracted with being taken out
User's operating characteristics data of search result items corresponding to the searching keyword taken.
Wherein, the user's history search record data includes but is not limited to user's click logs, and the user operates special
Sign data include but is not limited to user's history click data.
S202, based on predetermined search result items screening strategy, Screening Treatment is carried out to the search result items, will described in
User's operating characteristics data of at least one searching keyword, corresponding search result items and the search result items are as institute
State search quality and assess data.
Wherein, the search result items screening strategy is associated with user's operating characteristics data.
S203, for each searching keyword at least one searching keyword, based on baseline search strategy and
Search strategy to be assessed, the corresponding search result items in database are assessed to the search quality respectively and are ranked up to obtain first
Ranking results and the second ranking results.
S204, based on the user's history operating characteristics data, first ranking results and second ranking results
The search quality of the search strategy to be assessed is assessed.
On the present embodiment each step detailed description referring to above-described embodiment, repeat no more here.
The present embodiment constructs search quality assessment database by searching for record data based on user's history, and based on described
Search quality assesses database, baseline search strategy and search strategy to be assessed, assesses database to the search quality respectively
In corresponding search result items be ranked up to obtain the first ranking results and the second ranking results, and grasped based on the user's history
Make characteristic, first ranking results and second ranking results to enter the search quality of the search strategy to be assessed
Row is assessed, and is made whole evaluation process full automation, to reduce cost of labor and Human disturbance degree, and is improved search quality and comment
Estimate the degree of accuracy.
Embodiment three
Fig. 3 is a kind of flow chart for search quality appraisal procedure that the embodiment of the present invention three provides, and the present embodiment is above-mentioned
On the basis of embodiment, by for each searching keyword at least one searching keyword, based on baseline search plan
Slightly with search strategy to be assessed, the corresponding search result items in database are assessed to the search quality respectively and are ranked up to obtain
First ranking results and the second ranking results are preferably：It is crucial for each inquiry at least one searching keyword
Word, extract the page feature of corresponding search result items；From the page feature extracted, baseline policy characteristics and to be assessed are obtained
Policy characteristics；Based on the baseline policy characteristics and policy characteristics to be assessed and corresponding baseline Policy model and plan to be assessed
The search result items are ranked up to obtain first ranking results and second ranking results by slightly model.Such as Fig. 3
It is shown, specifically include：
S301, search quality assessment database, the search quality assessment are constructed based on user's history search record data
Database includes at least one searching keyword, corresponding at least one search result items and for the search result items
User's operating characteristics data.
S302, for each searching keyword at least one searching keyword, extract corresponding search result items
Page feature, from the page feature extracted, obtain baseline policy characteristics and policy characteristics to be assessed.
S303, based on the baseline policy characteristics and policy characteristics to be assessed and corresponding baseline Policy model and to be evaluated
Estimate Policy model, the search result items are ranked up to obtain first ranking results and second ranking results.
S304, based on the user's history operating characteristics data, first ranking results and second ranking results
The search quality of the search strategy to be assessed is assessed.
The present embodiment assesses database for each inquiry at least one searching keyword based on the search quality
Keyword, extract the page feature of corresponding search result items, and from the page feature extracted, obtain baseline policy characteristics and
Policy characteristics to be assessed, and based on the baseline policy characteristics and policy characteristics to be assessed and corresponding baseline Policy model and
Policy model to be assessed, the search result items are ranked up to obtain first ranking results and the second sequence knot
Fruit, make whole evaluation process full automation, to reduce cost of labor and Human disturbance degree, and improve search quality and assess standard
Exactness.
Example IV
Fig. 4 is a kind of flow chart for search quality appraisal procedure that the embodiment of the present invention four provides, and the present embodiment is above-mentioned
On the basis of embodiment, the user's history operating characteristics data, first ranking results and second sequence will be based on
As a result the search quality of the search strategy to be assessed is assessed, is preferably based on first ranking results and second row
Sequence result, determine the sorting position that same sorting position corresponds to different search result items situations be present；By the sorting position
On different search result items composition position difference search result pair；Based on the position difference search result pair and corresponding
User's operating characteristics data, the search strategy to be assessed is assessed.Further, the difference search result will be based on
Pair and corresponding user's operating characteristics data, the search strategy to be assessed is assessed preferably：Based on each position
Difference search result pair and corresponding user's operating characteristics data, determine difference during each click in simulation search environment
The search result that different search result centering is hit；Each position difference search result centering is each under statistical simulation search environment
The click result of individual search result；Based on the click result counted, the search strategy to be assessed is assessed.Such as Fig. 4
It is shown, specifically include：
S401, search quality assessment database, the search quality assessment are constructed based on user's history search record data
Database includes at least one searching keyword, corresponding at least one search result items and for the search result items
User's operating characteristics data.
S402, for each searching keyword at least one searching keyword, based on baseline search strategy and
Search strategy to be assessed, the corresponding search result items in database are assessed to the search quality respectively and are ranked up to obtain first
Ranking results and the second ranking results.
S403, based on first ranking results and the second ranking results, determine to have same sorting position corresponding not
With the sorting position of search result items situation, by the different search result items composition position difference search knot on the sorting position
Fruit pair.
S404, based on each position difference search result pair and corresponding user's operating characteristics data, it is determined that simulation is searched
The search result that the difference search result centering is hit during each click in grommet border.
Specifically, being based on user's operating characteristics data, described one search of position difference search result centering is counted respectively
The user's history number of clicks of result items is more than, is less than and the user's history number of clicks equal to another search result items
Total degree awin, bwin and sum that situation occurs；Calculated based on described awin, bwin and sum and click on the first sequence knot
The probability of search result in fruit and the probability for clicking on the search result in the second ranking results；Sorted based on clicking on described first
As a result the probability of the search result in and the probability for clicking on the search result in the second ranking results, utilize generating random number side
Method, determine the search result that the position difference search result centering is hit during each click in simulation search environment.
The click knot of each search result of each position difference search result centering under S405, statistical simulation search environment
Fruit.
S406, based on the click result counted, the search strategy to be assessed is assessed.
Specifically, the number for clicking on the search result items in the first ranking results can be counted more than the second ranking results of click
In search result items number total degree G；The number that statistics clicks on the search result items in the first ranking results is less than point
The total degree B of the number for the search result items hit in the second ranking results；Statistics clicks on the search result in the first ranking results
The number of item is equal to the total degree S for the number for clicking on the search result items in the second ranking results.Based on number G, the B counted
And S, the metewand of the calculating search strategy to be assessed.The search strategy to be assessed is carried out according to the metewand
Assess.
The present embodiment is by simulation search environment, and each position difference search result centering under statistical simulation search environment
Each search result click result.Based on the click result counted, the search strategy to be assessed is assessed.No
Only make whole evaluation process full automation, to reduce cost of labor and Human disturbance degree, and improve search quality and comment
Estimate the degree of accuracy.
Embodiment five
Fig. 5 show a kind of structural representation of search quality apparatus for evaluating of the offer of the embodiment of the present invention five, the device
It can be realized by the way of software or hardware, the device can be integrated in fixed terminal or server, as shown in figure 5, the device
Concrete structure it is as follows：Assess database construction module 51, order module 52, evaluation module 53；
The assessment database construction module 51 is used to assess based on user's history search record data construction search quality
Database, the search quality assess database and include at least one searching keyword, corresponding at least one search result items
And user's operating characteristics data for the search result items；
The order module 52 is used for each searching keyword being directed at least one searching keyword, based on base
Line search strategy and search strategy to be assessed, the corresponding search result items assessed respectively the search quality in database are carried out
Sequence obtains the first ranking results and the second ranking results；
The evaluation module 53 is used for based on the user's history operating characteristics data, first ranking results and described
Second ranking results are assessed the search quality of the search strategy to be assessed.
The search quality that search quality apparatus for evaluating described in the present embodiment is used to perform described in the various embodiments described above is assessed
Method, its technical principle is similar with caused technique effect, repeats no more here.
On the basis of above-described embodiment, the assessment database construction module 51 is specifically used for, and is searched for from user's history
At least one searching keyword is extracted in record data；Searched for from user's history in record data, extract and extracted
User's operating characteristics data of search result items corresponding to searching keyword；By at least one searching keyword, corresponding
User's operating characteristics data of search result items and the search result items assess data as the search quality.
On the basis of above-described embodiment, described device also includes：Screening module 54；
The screening module 54 is used to search in record data from user's history in the assessment database construction module 51,
After user's operating characteristics data of extraction search result items corresponding with the searching keyword extracted, based on predetermined search knot
Fruit item screening strategy, Screening Treatment is carried out to the search result items, wherein the search result items screening strategy is grasped with user
It is associated as characteristic.
On the basis of above-described embodiment, the user's history search record data includes user's click logs, Yi Jisuo
Stating user's operating characteristics data includes user's history click data.
On the basis of above-described embodiment, the order module 52 is specifically used for, crucial at least one inquiry
Each searching keyword in word, extract the page feature of corresponding search result items；From the page feature extracted, base is obtained
Line policy characteristics and policy characteristics to be assessed；Based on the baseline policy characteristics and policy characteristics to be assessed and corresponding baseline
Policy model and Policy model to be assessed, the search result items are ranked up to obtain first ranking results and described
Second ranking results.
On the basis of above-described embodiment, the evaluation module 53 includes：Position determination unit 531, the and of pairing unit 532
Assessment unit 533；
The position determination unit 531 is used to be based on first ranking results and the second ranking results, determines exist
Same sorting position corresponds to the sorting position of different search result items situations；
The pairing unit 532 is used for the different search result items composition position difference search knot on the sorting position
Fruit pair；
The assessment unit 533 is used to be based on the position difference search result pair and corresponding user's operating characteristics number
According to assessing the search strategy to be assessed.
On the basis of above-described embodiment, the assessment unit 533 includes：It is single to hit determination subelement 5331, statistics
Member 5332 and assessment subelement 5333；
The determination subelement 5331 of hitting is used to be based on each position difference search result pair and corresponding user behaviour
Make characteristic, determine the search knot that the difference search result centering is hit during each click in simulation search environment
Fruit；
The statistics subelement 5332 is used for each of each position difference search result centering under statistical simulation search environment
The click result of individual search result；
The assessment subelement 5333 is used for based on the click result counted, and the search strategy to be assessed is commented
Estimate.
On the basis of above-described embodiment, the statistics subelement 5332 is specifically used for, and statistics clicks on the first ranking results
In search result items number more than click on the second ranking results in search result items number total degree G；Statistics point
The number of the search result items in the first ranking results is hit less than the number for clicking on the search result items in the second ranking results
Total degree B；The number that statistics clicks on the search result items in the first ranking results is equal to the search clicked in the second ranking results
The total degree S of the number of result items.
On the basis of above-described embodiment, the statistics subelement 5332 is specifically used for, based on number G, the B counted and
S, calculate the metewand of the search strategy to be assessed.
On the basis of above-described embodiment, the determination subelement 5331 of hitting is specifically used for, based on user's operating characteristics
Data, the user's history number of clicks for counting described one search result items of position difference search result centering respectively are more than, are few
In and equal to another search result items user's history number of clicks situation occur total degree awin, bwin and sum；
The probability for clicking on the search result in first ranking results and click second are calculated based on described awin, bwin and sum
The probability of search result in ranking results；Based on the probability for clicking on the search result in first ranking results and click on the
The probability of search result in two ranking results, using random digit generation method, determine each click in simulation search environment
The search result that Shi Suoshu position difference search result centerings are hit.
On the basis of above-described embodiment, described device also includes：Click on determining module 55；
The click determining module 55 is used to be based on each position difference search knot in the determination subelement 5331 of hitting
Fruit pair and corresponding user's operating characteristics data, determine position difference search during each click in simulation search environment
As a result before the search result that centering is hit, probability is clicked on based on predeterminated position, using random digit generation method, it is determined that described
Position difference search result is searched for whether click behavior occurring, and when click behavior occurs based on each position difference
As a result pair and corresponding user's operating characteristics data, determine that the position difference is searched during each click in simulation search environment
The search result that hitch fruit centering is hit.
The search quality that search quality apparatus for evaluating described in the various embodiments described above is used to perform described in the various embodiments described above is commented
Estimate method, its technical principle is similar with caused technique effect, repeats no more here.
Pay attention to, above are only presently preferred embodiments of the present invention and institute's application technology principle.It will be appreciated by those skilled in the art that
The invention is not restricted to specific embodiment described here, can carry out for a person skilled in the art various obvious changes,
Readjust and substitute without departing from protection scope of the present invention.Therefore, although being carried out by above example to the present invention
It is described in further detail, but the present invention is not limited only to above example, without departing from the inventive concept, also
Other more equivalent embodiments can be included, and the scope of the present invention is determined by scope of the appended claims.
Claims (22)
- A kind of 1. search quality appraisal procedure, it is characterised in that including：Database is assessed based on user's history search record data construction search quality, the search quality is assessed database and included At least one searching keyword, corresponding at least one search result items and the user for the search result items operate special Levy data；For each searching keyword at least one searching keyword, based on baseline search strategy and search to be assessed Strategy, the corresponding search result items assessed respectively in database the search quality be ranked up to obtain the first ranking results with Second ranking results；Based on the user's history operating characteristics data, first ranking results and second ranking results to described to be evaluated The search quality for estimating search strategy is assessed.
- 2. according to the method for claim 1, it is characterised in that based on user's history search record data construction search quality Assessing database includes：Searched for from user's history in record data and extract at least one searching keyword；Searched for from user's history in record data, the user behaviour for extracting search result items corresponding with the searching keyword extracted Make characteristic；By user's operating characteristics of at least one searching keyword, corresponding search result items and the search result items Data assess data as the search quality.
- 3. according to the method for claim 2, it is characterised in that searched for from user's history in record data, extraction and institute After user's operating characteristics data of search result items corresponding to the searching keyword of extraction, methods described also includes：Based on predetermined search result items screening strategy, Screening Treatment is carried out to the search result items, wherein the search knot Fruit item screening strategy is associated with user's operating characteristics data.
- 4. according to any described method in claim 1-3, it is characterised in that the user's history search record data includes User's click logs, and user's operating characteristics data include user's history click data.
- 5. according to the method for claim 1, it is characterised in that for each looking at least one searching keyword Keyword is ask, based on baseline search strategy and search strategy to be assessed, pair in database is assessed to the search quality respectively Search result items are answered to be ranked up to obtain the first ranking results and the second ranking results and include：For each searching keyword at least one searching keyword, the page for extracting corresponding search result items is special Sign；From the page feature extracted, baseline policy characteristics and policy characteristics to be assessed are obtained；Based on the baseline policy characteristics and policy characteristics to be assessed and corresponding baseline Policy model and tactful mould to be assessed Type, the search result items are ranked up to obtain first ranking results and second ranking results.
- 6. according to the method for claim 1, it is characterised in that based on the user's history operating characteristics data, described the One ranking results and second ranking results carry out assessment to the search quality of the search strategy to be assessed to be included：Based on first ranking results and the second ranking results, determine that same sorting position be present corresponds to different search results The sorting position of item situation；Different search result items on the sorting position are formed into position difference search result pair；Based on the position difference search result pair and corresponding user's operating characteristics data, to the search strategy to be assessed Assessed.
- 7. according to the method for claim 6, it is characterised in that based on the difference search result pair and corresponding user Operating characteristics data, carrying out assessment to the search strategy to be assessed includes：Based on each position difference search result pair and corresponding user's operating characteristics data, determine in simulation search environment The search result that the difference search result centering is hit when clicking on every time；The click result of each search result of each position difference search result centering under statistical simulation search environment；Based on the click result counted, the search strategy to be assessed is assessed.
- 8. according to the method for claim 7, it is characterised in that each position difference search knot under statistical simulation search environment The click result of each search result of fruit centering includes：Statistics clicks on the number of the search result items in the first ranking results more than the search result clicked in the second ranking results The total degree G of the number of item；Statistics clicks on the number of the search result items in the first ranking results less than the search result clicked in the second ranking results The total degree B of the number of item；The number that statistics clicks on the search result items in the first ranking results is equal to the search result clicked in the second ranking results The total degree S of the number of item.
- 9. according to the method for claim 8, it is characterised in that based on the click result counted, to be assessed searched to described Rope strategy, which carries out assessment, to be included：Based on number G, B and the S counted, the metewand of the calculating search strategy to be assessed.
- 10. according to the method for claim 7, it is characterised in that based on each position difference search result pair and correspondingly User's operating characteristics data, determine that difference search result centering during each click in simulation search environment is hit Search result includes：Based on user's operating characteristics data, the user of one search result items of the position difference search result centering is counted respectively History click on often in, be less than and equal to another search result items user's history number of clicks situation occur it is total Number awin, bwin and sum；Probability and the click for clicking on the search result in first ranking results are calculated based on described awin, bwin and sum The probability of search result in second ranking results；Based on the probability for clicking on the search result in first ranking results and click on the search result in the second ranking results Probability, using random digit generation method, determine position difference search result during each click in simulation search environment The search result that centering is hit.
- 11. according to the method for claim 7, it is characterised in that based on each position difference search result pair and right The user's operating characteristics data answered, determine position difference search result centering quilt during each click in simulation search environment Before the search result hit, methods described also includes：Probability is clicked on based on predeterminated position, using random digit generation method, determines the position difference search result to whether sending out Raw click behavior, and when click behavior occurs, operated based on each position difference search result pair and corresponding user Characteristic, determine the search knot that the position difference search result centering is hit during each click in simulation search environment Fruit.
- A kind of 12. search quality apparatus for evaluating, it is characterised in that including：Database construction module is assessed, for assessing database, institute based on user's history search record data construction search quality State search quality and assess database comprising at least one searching keyword, corresponding at least one search result items and for institute State user's operating characteristics data of search result items；Order module, for for each searching keyword at least one searching keyword, based on baseline search plan Slightly with search strategy to be assessed, the corresponding search result items in database are assessed to the search quality respectively and are ranked up to obtain First ranking results and the second ranking results；Evaluation module, for based on the user's history operating characteristics data, first ranking results and second sequence As a result the search quality of the search strategy to be assessed is assessed.
- 13. device according to claim 12, it is characterised in that the assessment database construction module is specifically used for, from At least one searching keyword is extracted in user's history search record data；Search in record data, carry from user's history Take user's operating characteristics data of search result items corresponding with the searching keyword extracted；At least one inquiry is closed User's operating characteristics data of keyword, corresponding search result items and the search result items are assessed as the search quality Data.
- 14. device according to claim 13, it is characterised in that described device also includes：Screening module, for being searched in the assessment database construction module from user's history in record data, extract with being taken out After user's operating characteristics data of search result items corresponding to the searching keyword taken, plan is screened based on predetermined search result items Slightly, Screening Treatment is carried out to the search result items, wherein the search result items screening strategy and user's operating characteristics data It is associated.
- 15. according to any described device in claim 12-14, it is characterised in that the user's history searches for record data Include user's history click data including user's click logs, and user's operating characteristics data.
- 16. device according to claim 12, it is characterised in that the order module is specifically used for, for it is described at least Each searching keyword in one searching keyword, extract the page feature of corresponding search result items；From the page extracted In feature, baseline policy characteristics and policy characteristics to be assessed are obtained；Based on the baseline policy characteristics and policy characteristics to be assessed And corresponding baseline Policy model and Policy model to be assessed, the search result items are ranked up to obtain described first Ranking results and second ranking results.
- 17. device according to claim 12, it is characterised in that the evaluation module includes：Position determination unit, for based on first ranking results and the second ranking results, determining same sequence position be present Put the sorting position of corresponding different search result items situations；Pairing unit, for the different search result items on the sorting position to be formed into position difference search result pair；Assessment unit, for based on the position difference search result pair and corresponding user's operating characteristics data, to described Search strategy to be assessed is assessed.
- 18. device according to claim 17, it is characterised in that the assessment unit includes：Determination subelement is hit, for based on each position difference search result pair and corresponding user's operating characteristics data, Determine the search result that the difference search result centering is hit during each click in simulation search environment；Subelement is counted, for each search result of each position difference search result centering under statistical simulation search environment Click on result；Subelement is assessed, for based on the click result counted, assessing the search strategy to be assessed.
- 19. device according to claim 18, it is characterised in that the statistics subelement is specifically used for, and statistics clicks on the Total time of the number of search result items in one ranking results more than the number for clicking on the search result items in the second ranking results Number G；Statistics clicks on the number of the search result items in the first ranking results less than the search result clicked in the second ranking results The total degree B of the number of item；The number that statistics clicks on the search result items in the first ranking results is equal to the sequence knot of click second The total degree S of the number of search result items in fruit.
- 20. device according to claim 19, it is characterised in that the statistics subelement is specifically used for, based on being counted Number G, B and S, calculate the metewand of the search strategy to be assessed.
- 21. device according to claim 18, it is characterised in that the determination subelement of hitting is specifically used for, based on use Family operating characteristics data, the user's history click time of described one search result items of position difference search result centering is counted respectively Number be more than, be less than and equal to another search result items user's history number of clicks situation occur total degree awin, Bwin and sum；The probability for clicking on the search result in first ranking results is calculated based on described awin, bwin and sum With the probability for clicking on the search result in the second ranking results；Based on the general of the search result clicked in first ranking results Rate and the probability for clicking on the search result in the second ranking results, using random digit generation method, are determined in simulation search environment Each click when the search result that is hit of the position difference search result centering.
- 22. device according to claim 18, it is characterised in that described device also includes：Determining module is clicked on, for being based on each position difference search result pair and corresponding in the determination subelement of hitting User's operating characteristics data, determine that the position difference search result centering is hit during each click in simulation search environment Search result before, probability is clicked on based on predeterminated position, using random digit generation method, determines the position difference search knot Whether fruit is to occurring click behavior, and when click behavior occurs, based on each position difference search result pair and correspondingly User's operating characteristics data, determine that the position difference search result centering is hit during each click in simulation search environment In search result.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610645103.2A CN107704467B (en) | 2016-08-09 | 2016-08-09 | Search quality evaluation method and device |
| PCT/CN2016/108422 WO2018028099A1 (en) | 2016-08-09 | 2016-12-02 | Method and device for search quality assessment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610645103.2A CN107704467B (en) | 2016-08-09 | 2016-08-09 | Search quality evaluation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107704467A true CN107704467A (en) | 2018-02-16 |
| CN107704467B CN107704467B (en) | 2021-08-24 |
Family
ID=61161645
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610645103.2A Active CN107704467B (en) | 2016-08-09 | 2016-08-09 | Search quality evaluation method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107704467B (en) |
| WO (1) | WO2018028099A1 (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108897685A (en) * | 2018-06-28 | 2018-11-27 | 百度在线网络技术（北京）有限公司 | Method for evaluating quality, device, server and the medium of search result |
| CN109582751A (en) * | 2018-11-29 | 2019-04-05 | 百度在线网络技术（北京）有限公司 | A kind of measure and server of retrieval effectiveness |
| CN110020209A (en) * | 2019-04-18 | 2019-07-16 | 北京奇艺世纪科技有限公司 | Content and the correlation of search term determine method and system, methods of exhibiting and system |
| CN110727865A (en) * | 2019-10-09 | 2020-01-24 | 北京百度网讯科技有限公司 | Problem positioning method and device of retrieval strategy, electronic equipment and storage medium |
| CN112115340A (en) * | 2020-09-14 | 2020-12-22 | 深圳市欢太科技有限公司 | Search strategy selection method, mobile terminal and readable storage medium |
| CN112115344A (en) * | 2019-06-20 | 2020-12-22 | 百度(中国)有限公司 | Automatic evaluation method, device and system of search results and storage medium |
| CN112784141A (en) * | 2019-10-23 | 2021-05-11 | 腾讯科技（深圳）有限公司 | Search result quality determination method and device, storage medium and computer equipment |
| CN113626715A (en) * | 2021-08-26 | 2021-11-09 | 北京字跳网络技术有限公司 | Query result display method, device, medium and electronic equipment |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109543940B (en) * | 2018-10-12 | 2024-04-09 | 中国平安人寿保险股份有限公司 | Activity evaluation method, activity evaluation device, electronic equipment and storage medium |
| CN111367778B (en) * | 2020-03-13 | 2023-07-07 | 百度在线网络技术（北京）有限公司 | Data analysis method and device for evaluating search strategy |
| CN113781146A (en) * | 2020-11-17 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Recommendation method, device, equipment and storage medium of product information |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1963816A (en) * | 2006-12-01 | 2007-05-16 | 清华大学 | Automatization processing method of rating of merit of search engine |
| US20070185864A1 (en) * | 2006-01-31 | 2007-08-09 | Intellext, Inc. | Methods and apparatus for displaying ranked search results |
| CN103544307A (en) * | 2013-11-04 | 2014-01-29 | 北京中搜网络技术股份有限公司 | Multi-search-engine automatic comparison and evaluation method independent of document library |
| CN103593411A (en) * | 2013-10-23 | 2014-02-19 | 江苏大学 | Method for testing combination properties of evaluation indexes of search engines and testing device |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020143759A1 (en) * | 2001-03-27 | 2002-10-03 | Yu Allen Kai-Lang | Computer searches with results prioritized using histories restricted by query context and user community |
| CN102760138B (en) * | 2011-04-26 | 2015-03-11 | 北京百度网讯科技有限公司 | Classification method and device for user network behaviors and search method and device for user network behaviors |
| US20140059062A1 (en) * | 2012-08-24 | 2014-02-27 | Google Inc. | Incremental updating of query-to-resource mapping |
| CN103870607A (en) * | 2014-04-08 | 2014-06-18 | 北京奇虎科技有限公司 | Sequencing method and device of search results of multiple search engines |
-
2016
- 2016-08-09 CN CN201610645103.2A patent/CN107704467B/en active Active
- 2016-12-02 WO PCT/CN2016/108422 patent/WO2018028099A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070185864A1 (en) * | 2006-01-31 | 2007-08-09 | Intellext, Inc. | Methods and apparatus for displaying ranked search results |
| CN1963816A (en) * | 2006-12-01 | 2007-05-16 | 清华大学 | Automatization processing method of rating of merit of search engine |
| CN103593411A (en) * | 2013-10-23 | 2014-02-19 | 江苏大学 | Method for testing combination properties of evaluation indexes of search engines and testing device |
| CN103544307A (en) * | 2013-11-04 | 2014-01-29 | 北京中搜网络技术股份有限公司 | Multi-search-engine automatic comparison and evaluation method independent of document library |
Non-Patent Citations (1)
| Title |
|---|
| 南通大学教务处: "《南通大学优秀毕业设计（论文）集》", 31 December 2017, 学海图南 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108897685A (en) * | 2018-06-28 | 2018-11-27 | 百度在线网络技术（北京）有限公司 | Method for evaluating quality, device, server and the medium of search result |
| CN108897685B (en) * | 2018-06-28 | 2022-02-25 | 百度在线网络技术（北京）有限公司 | Method, device, server and medium for evaluating quality of search result |
| CN109582751A (en) * | 2018-11-29 | 2019-04-05 | 百度在线网络技术（北京）有限公司 | A kind of measure and server of retrieval effectiveness |
| CN110020209A (en) * | 2019-04-18 | 2019-07-16 | 北京奇艺世纪科技有限公司 | Content and the correlation of search term determine method and system, methods of exhibiting and system |
| CN112115344A (en) * | 2019-06-20 | 2020-12-22 | 百度(中国)有限公司 | Automatic evaluation method, device and system of search results and storage medium |
| CN110727865A (en) * | 2019-10-09 | 2020-01-24 | 北京百度网讯科技有限公司 | Problem positioning method and device of retrieval strategy, electronic equipment and storage medium |
| CN112784141A (en) * | 2019-10-23 | 2021-05-11 | 腾讯科技（深圳）有限公司 | Search result quality determination method and device, storage medium and computer equipment |
| CN112784141B (en) * | 2019-10-23 | 2023-10-31 | 腾讯科技（深圳）有限公司 | Search result quality determination method, apparatus, storage medium and computer device |
| CN112115340A (en) * | 2020-09-14 | 2020-12-22 | 深圳市欢太科技有限公司 | Search strategy selection method, mobile terminal and readable storage medium |
| CN113626715A (en) * | 2021-08-26 | 2021-11-09 | 北京字跳网络技术有限公司 | Query result display method, device, medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018028099A1 (en) | 2018-02-15 |
| CN107704467B (en) | 2021-08-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107704467A (en) | Search quality appraisal procedure and device | |
| CN105701216B (en) | A kind of information-pushing method and device | |
| KR101361182B1 (en) | Systems for and methods of finding relevant documents by analyzing tags | |
| CN102226900B (en) | Phrase-based searching in an information retrieval system | |
| CN102760138B (en) | Classification method and device for user network behaviors and search method and device for user network behaviors | |
| US8494897B1 (en) | Inferring profiles of network users and the resources they access | |
| US8250066B2 (en) | Search results ranking method and system | |
| TWI391834B (en) | Systems for and methods of finding relevant documents by analyzing tags | |
| US20080313115A1 (en) | Behavioral Profiling Using a Behavioral WEB Graph and Use of the Behavioral WEB Graph in Prediction | |
| CN104462553A (en) | Method and device for recommending question and answer page related questions | |
| CN102004757A (en) | System to modify websites for organic search optimization | |
| CN104063476A (en) | Social network-based content recommending method and system | |
| CN102314443B (en) | The modification method of search engine and system | |
| Arapakis et al. | On the feasibility of predicting news popularity at cold start | |
| CN101840544A (en) | Knowledge assessment tool | |
| CN102567494B (en) | Website classification method and device | |
| CN104050197B (en) | A kind of information retrieval system evaluating method and device | |
| CN104217031A (en) | Method and device for classifying users according to search log data of server | |
| CN106294535A (en) | The recognition methods of website and device | |
| CN107193743A (en) | One kind application assessment method, device, computer equipment and storage medium | |
| CN106649738A (en) | Method and device for aggregating personage information message in search engine result page | |
| CN105630937A (en) | Method and device for searching answers to exam questions | |
| CN104408144B (en) | The detection method and device of web search keyword | |
| CN101211368A (en) | Method for classifying search term, device and search engine system | |
| Chen et al. | The best answers? think twice: online detection of commercial campaigns in the CQA forums |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |