CN105573887B - The method for evaluating quality and device of search engine - Google Patents
The method for evaluating quality and device of search engine Download PDFInfo
- Publication number
- CN105573887B CN105573887B CN201510927675.5A CN201510927675A CN105573887B CN 105573887 B CN105573887 B CN 105573887B CN 201510927675 A CN201510927675 A CN 201510927675A CN 105573887 B CN105573887 B CN 105573887B
- Authority
- CN
- China
- Prior art keywords
- user
- depth
- query word
- multimedia resource
- clicked
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
Abstract
The invention discloses a kind of method for evaluating quality of search engine and device, which is used for searching multimedia resource, which includes：User's depth dwell data of single query word is obtained from user journal；According to user's depth dwell data of single query word, user's depth dwell data of full dose query word is obtained；And user's depth dwell data according to full dose query word and original evaluation index, original assessment is carried out to the quality of search engine, wherein, original evaluation index includes that the number for being independently clicked multimedia resource, the mean number for being clicked multimedia resource of each query word, the number of query word less than multimedia resource number threshold value, multimedia resource finish playing than population mean, finish playing than at least one of the number of query word of threshold value less than multimedia resource.The present invention can be not necessarily to carry out manually mark, objective the quality of search engine is assessed in time.
Description
Technical field
The present invention relates to information search and searching field more particularly to the method for evaluating quality and dress of a kind of search engine
It sets.
Background technology
Search engine (Search Engine) refers to being collected mutually according to certain strategy, with specific computer program
Information in networking, after carrying out tissue and processing to information, will treated presentation of information to user, that is, search engine is
The system for providing retrieval service to the user.Search engine includes that full-text index, directory index, META Search Engine, vertical search are drawn
It holds up, aggregation type search engine, portal search engine and free lists of links etc..
The quality evaluation of search engine is constantly subjected to industrial circle and the extensive concern of researcher.Currently, Cranfield is commented
Valence system is widely used in the quality evaluation of search engine, which is by inquiry sample set, correct option collection, comments
Survey the complete evaluation and test scheme that these three parts of index are constituted.The quality that engine is scanned for using Cranfield appraisement systems is commented
When estimating, including following three links：First, representative query word (query) is extracted, the query word extracted is formed one
The inquiry sample set of suitable scale；Then, it for the inquiry sample set, is found corresponding thereto from the corpus of search engine
As a result, i.e. manually marked；Finally, by the query word extracted and with markup information corpus input retrieval system
System, searching system feedback result, then for search engine feedback as a result, using pre-defined evaluation calculation formula, utilizing
The method of numeralization come evaluate search engine feedback result and mark desired result degree of closeness.
Wherein, there are the methods of the result of a variety of evaluation search engine feedbacks, such as accuracy rate-recall rate
(Precision-Recall) method, Rating of the single value (Precision@N) method, average sequence inverse (Mean Reciprocal
Ranking, abbreviation MRR) it method, Average Accuracy mean value (Mean Average Precision, abbreviation MAP) method and loses
Storage gain (Discounted Cumulative Gain, abbreviation DCG) method etc..
However, since traditional information retrieval system data and portfolio are usually little, retrieval input also with respect to specification, because
This can manually choose sample set and artificial mark example result (model answer), still, with internet continuous development with
The increase of internet information amount, the heavy traffic and data magnanimity of search engine on line, in the way of manually mark answer
The evaluation for carrying out network information retrieval system is not only labor intensive but also a time-consuming process, can not possibly have been utilized artificial
The mode of answer is marked to carry out the mark of answer.That is, the shortcomings that Cranfield appraisement systems, is to need manually to select
It takes sample set and needs artificial mark example result.
In order to solve the problems, such as the artificial marks of above-mentioned Cranfield appraisement systems not only labor intensive but also time-consuming, carry
A/B tests (A/B testing) system is gone out.A/B tests system when user searches for, and point of user is automatically determined by system
Group number (Bucket ID) imports different branches so that the user of respective packets sees different product version by extracting flow automatically
The result that this (or different search engines) provides.Behavior of the user under different editions product will be recorded, these behaviors
Data form a series of indexs by data analysis, then by comparing these indexs come obtain between each product version who it is excellent who
Bad conclusion.Wherein, when index calculates, can be subdivided into method based on expert analysis mode and based on click statistics method this
Two methods.
However, with the development of Internet service, the requirement for the promptness of search-engine results quality optimization is also got over
The problem of carrying out higher, traditional A/B test system discovery search engines needs certain expert estimation time, also, due to length
Tail effect (Long Tail Effect), the outstanding representation of query word cannot be to the excellent of whole system involved in A/B test systems
Good mapping is made in different performance.That is, A/B test system the problem of be in face of Internet service scale power not from
The heart.
In addition, the search result of other Rich Medias (Rich Media) search engine of video search engine etc. has it
Itself the characteristics of.User for result video satisfaction whether, cannot be weighed simply by hit, playback volume or sequence.
In many cases, video could there are one compare objective appraisal by viewing for user's needs.This traditional to search with text
The engine evaluation method of Suo Weizhu can not be suitable for the quality evaluation of the video search engine of video this " deep semantic ".And
And the layout of video search result page is no longer common list type but grid type in text search engine on many lines, this
Weaken traditional position.Therefore, assessment is carried out based on position to be unfair.However, either Cranfield
Appraisement system or A/B test system, all do not provide the targetedly solution of the quality evaluation of video search engine.
Invention content
Technical problem
In view of this, the technical problem to be solved by the present invention is to how objective in time to the progress of the quality of search engine
Assessment.
Solution
In order to solve the above-mentioned technical problem, in a first aspect, the present invention provides a kind of quality evaluation sides of search engine
Method, described search engine are used for searching multimedia resource, and the method for evaluating quality includes：
User's depth dwell data of single query word is obtained from user journal, wherein the user of the single query word
Depth dwell data includes：Query word, be clicked multimedia resource set, multimedia resource finish playing than set, Yi Jisuo
It states and is clicked multimedia resource set and finishes playing to the multimedia resource than the mapping function of set；
According to user's depth dwell data of the single query word, the user's depth for obtaining full dose query word stops number
Include according to, wherein user's depth dwell data of the full dose query word：Being clicked under full dose query word, current queries word
Under multimedia resource, current queries word be clicked multimedia resource be clicked under number and current queries word by point
The synthesis for hitting multimedia resource finishes playing ratio；And
According to user's depth dwell data of the full dose query word and original evaluation index, to the matter of described search engine
Amount carries out original assessment,
Wherein, the original evaluation index include the number for being independently clicked multimedia resource, each query word by point
The mean number, the number of query word less than multimedia resource number threshold value, multimedia resource for hitting multimedia resource play
It finishes playing than at least one of the number of query word of threshold value at than population mean, less than multimedia resource.
With reference to first aspect, in the first possible implementation, the method for evaluating quality further includes：
According to user's depth dwell data of the single query word, the user's depth for calculating the single query word stops
Index；And
Index and comprehensive assessment index are stopped according to user's depth, carrying out synthesis to the quality of described search engine comments
Estimate,
Wherein, the comprehensive assessment index includes that user's depth stops exponential average and stops index less than user's depth
At least one of number of query word of threshold value.
The possible realization method of with reference to first aspect the first, in second of possible realization method, the basis
User's depth dwell data of the single query word, the user's depth for calculating the single query word stop index, including：
According to user's depth dwell data of the single query word and use formulaCalculate described single look into
The user's depth for asking word stops index,
Wherein, y is that user's depth of the single query word stops index,
X=VidCount*ClickCount*AveragePerc, VidCount are for being independently clicked multimedia resource
Number, ClickCount are the numbers for being clicked multimedia resource, and AveragePerc is that multimedia resource finishes playing than average
Value.
The possible realization method of with reference to first aspect the first, in the third possible realization method, the basis
User's depth dwell data of the single query word, the user's depth for calculating the single query word stop index, including：
According to user's depth dwell data of the single query word and using formula y=VidCountN*
ClickCountN*AveragePercN, the user's depth for calculating the single query word stop index,
Wherein, y is that user's depth of the single query word stops index,
VidCount is the number for being independently clicked multimedia resource, and ClickCount is time for being clicked multimedia resource
Number, AveragePerc are that multimedia resource finishes playing than average value, and min () is minimized, and max () is maximized.
The possible realization method of with reference to first aspect the first, in the 4th kind of possible realization method, the basis
User's depth dwell data of the single query word, the user's depth for calculating the single query word stop index, including：
According to user's depth dwell data of the single query word and use formula
The user's depth for calculating the single query word stops index,
Wherein, y is that user's depth of the single query word stops index,
VidCount is the number for being independently clicked multimedia resource, and AllVidCount is to utilize the single query word
The summation of all numbers of clicks of the multimedia resource searched, AveragePerc are that multimedia resource finishes playing than flat
Mean value.
In second aspect, the present invention provides a kind of quality assessment device of search engine, described search engine is for searching
Rope multimedia resource, the quality assessment device include：
Acquiring unit, user's depth dwell data for obtaining single query word from user journal, wherein described single
User's depth dwell data of query word includes：Query word, be clicked multimedia resource set, multimedia resource finishes playing ratio
Set and the multimedia resource set that is clicked finish playing to the multimedia resource than the mapping function of set；
Obtaining unit is connect with the acquiring unit, is used for user's depth dwell data according to the single query word,
Obtain user's depth dwell data of full dose query word, wherein user's depth dwell data of the full dose query word includes：Entirely
Amount query word, under current queries word be clicked under multimedia resource, current queries word be clicked multimedia resource by point
The synthesis for being clicked multimedia resource hit under number and current queries word finishes playing ratio；And
Original assessment unit is connect with the obtaining unit, for being stopped according to user's depth of the full dose query word
Data and original evaluation index carry out original assessment to the quality of described search engine,
Wherein, the original evaluation index include the number for being independently clicked multimedia resource, each query word by point
The mean number, the number of query word less than multimedia resource number threshold value, multimedia resource for hitting multimedia resource play
It finishes playing than at least one of the number of query word of threshold value at than population mean, less than multimedia resource.
In conjunction with second aspect, in the first possible implementation, the quality assessment device further includes：
Computing unit is connect with the acquiring unit, is used for user's depth dwell data according to the single query word,
The user's depth for calculating the single query word stops index；And
Comprehensive assessment unit is connect with the computing unit, is commented for stopping index and synthesis according to user's depth
Estimate index, comprehensive assessment carried out to the quality of described search engine,
Wherein, the comprehensive assessment index includes that user's depth stops exponential average and stops index less than user's depth
At least one of number of query word of threshold value.
In conjunction with the first possible realization method of second aspect, in second of possible realization method, the calculating
Unit is specifically used for, and according to user's depth dwell data of the single query word and uses formulaDescribed in calculating
User's depth of single query word stops index,
Wherein, y is that user's depth of the single query word stops index,
X=VidCount*ClickCount*AveragePerc, VidCount are for being independently clicked multimedia resource
Number, ClickCount are the numbers for being clicked multimedia resource, and AveragePerc is that multimedia resource finishes playing than average
Value.
In conjunction with the first possible realization method of second aspect, in the third possible realization method, the calculating
Unit is specifically used for, according to user's depth dwell data of the single query word and using formula y=VidCountN*
ClickCountN*AveragePercN, the user's depth for calculating the single query word stop index,
Wherein, y is that user's depth of the single query word stops index,
VidCount is the number for being independently clicked multimedia resource, and ClickCount is time for being clicked multimedia resource
Number, AveragePerc are that multimedia resource finishes playing than average value, and min () is minimized, and max () is maximized.
In conjunction with the first possible realization method of second aspect, in the 4th kind of possible realization method,
The computing unit is specifically used for, and according to user's depth dwell data of the single query word and uses formulaThe user's depth for calculating the single query word stops index,
Wherein, y is that user's depth of the single query word stops index,
VidCount is the number for being independently clicked multimedia resource, and AllVidCount is to utilize the single query word
The summation of all numbers of clicks of the multimedia resource searched, AveragePerc are that multimedia resource finishes playing than flat
Mean value.
Advantageous effect
The method for evaluating quality and device of the search engine of the embodiment of the present invention, according to the use of the full dose query word obtained
Family depth dwell data and original evaluation index carry out original assessment to the quality of search engine, and thus, it is possible to be not necessarily into pedestrian
Work mark objective is in time assessed the quality of search engine.Also, index and comprehensive assessment are stopped according to user's depth
Index to carry out comprehensive assessment to the quality of search engine, additionally it is possible to stop index directly more arbitrary two by user's depth
The good and bad degree of the search result of search engine under a query word, so as to improve search engine quality evaluation it is accurate
Property.
According to below with reference to the accompanying drawings becoming to detailed description of illustrative embodiments, other feature of the invention and aspect
It is clear.
Description of the drawings
Including in the description and the attached drawing of a part for constitution instruction and specification together illustrate the present invention's
Exemplary embodiment, feature and aspect, and principle for explaining the present invention.
Fig. 1 shows the flow chart of the method for evaluating quality of according to embodiments of the present invention one search engine；
Fig. 2 shows the flow charts of the method for evaluating quality of according to embodiments of the present invention two search engine；
Fig. 3 shows the structure diagram of the quality assessment device of according to embodiments of the present invention three search engine；And
Fig. 4 shows the structure diagram of the quality assessment device of according to embodiments of the present invention four search engine.
Specific implementation mode
Below with reference to attached drawing various exemplary embodiments, feature and the aspect that the present invention will be described in detail.It is identical in attached drawing
Reference numeral indicate functionally the same or similar element.Although the various aspects of embodiment are shown in the accompanying drawings, remove
It non-specifically points out, it is not necessary to attached drawing drawn to scale.
Dedicated word " exemplary " means " being used as example, embodiment or illustrative " herein.Here as " exemplary "
Illustrated any embodiment should not necessarily be construed as preferred or advantageous over other embodiments.
In addition, in order to better illustrate the present invention, numerous details is given in specific implementation mode below.
It will be appreciated by those skilled in the art that without certain details, the present invention can equally be implemented.In some instances, for
Method, means, element and circuit well known to those skilled in the art are not described in detail, in order to highlight the purport of the present invention.
Embodiment 1
Fig. 1 shows the flow chart of the method for evaluating quality of according to embodiments of the present invention one search engine.As shown in Figure 1,
The method for evaluating quality can specifically include：
Step S100, user's depth dwell data of single query word is obtained from user journal.
In the present invention, user depth stop behavior may include：(1) result of page searching of the user in search engine
On stop, i.e. user clicks the behavior of the search result of multiple multimedia resources such as video, audio；And (2) are used
Family is in the stop of search engine played on the page, the i.e. behavior of multimedia resource of user's viewing such as video, audio.
Specifically, four-tuple { query, vids, percs, δ } can be used to stop user's depth of each query word
Behavior is stayed to be portrayed.In other words, single inquiry can be obtained from user journal according to the data model of single query word
User's depth dwell data of word.The process may include carrying out pretreatment and noise removal processing to user journal data, use
The noise of family daily record data may be from illegally inputting, the various aspects of system exception, recording exceptional etc..
Wherein, query is query word, i.e., user inputs in the search each time of search engine, for example, can draw from search
The query word query of user is obtained in the user journal held up.
Vids is to click multimedia resource set, i.e., user clicks more matchmakers by search query word in result of page searching
The set of body resource, for example, can be seen from the multimedia resource of user journal by limiting the source of multimedia resource viewing
It sees to obtain in daily record and clicks multimedia resource set vids.
Percs be multimedia resource finish playing than set, that is, be clicked multimedia resource finish playing than set,
For example, daily record can be watched from the multimedia resource of user journal by carrying out after-treatment to multimedia resource played data
Middle acquisition multimedia resource finishes playing than set percs.It should be noted that since the total time of each multimedia resource is long
Degree may differ by it is larger, therefore, using multimedia resource finish playing than come to user's depth stop behavior portrayed than list
It is purely more objective to be portrayed the stop behavior of user's depth using the reproduction time length of multimedia resource.For example, being directed to
The same query word, if one is clicked multimedia resource and has been played repeatedly, this is clicked broadcasting for multimedia resource
Discharge into than that should be a comprehensive score, for example, can take the query word it is all finish playing than average value, for another example,
Can take the query word it is all finish playing than median etc..
δ is to be clicked multimedia resource set at most media resource plays to complete mapping function than set, for example, can be with
It is finished playing obtaining multimedia resource than pre-defining the mapping function when set.
That is, user's depth dwell data of above-mentioned single query word may include：Query word (query), by point
Hit multimedia resource set (vids), multimedia resource finishes playing than set (percs) and is clicked multimedia resource collection
Multimedia resource is closed to finish playing than the mapping function (δ) of set.
Step S120, according to user's depth dwell data of above-mentioned single query word, the user for obtaining full dose query word is deep
Spend dwell data.
For example, can carry out summarizing polymerization by user's depth dwell data of the single query word to getting, to obtain
Obtain user's depth dwell data of full dose query word.For example, the process may include carrying out after-treatment to data (to obtain
Count field datas) and denoising etc..
Specifically, four-tuple { query, vid, count, perc } can be used to carve the quality of search engine
It draws.In other words, user's depth dwell data (that is, user's depth dwell data of full dose query word) of entire search engine can be with
Including this four fields of query, vid, count and perc.Wherein, query is full dose query word；Vid is current query
Under be clicked multimedia resource；Count be under current query be clicked multimedia resource be clicked number；Perc is
The synthesis for being clicked multimedia resource under current query finishes playing ratio.
That is, user's depth dwell data of above-mentioned full dose query word may include：Full dose query word, current queries
Being clicked under multimedia resource, current queries word under word is clicked being clicked number and currently looking into for multimedia resource
The synthesis for being clicked multimedia resource ask under word finishes playing ratio.For example, user's depth dwell data of entire search engine
May include following three four-tuple { query, vid, count, perc }：{ A1,0001,500,80% }, A2,
0002,100,70% } and { A3,0003,200,90% }, wherein A1, A2 and A3 are full dose query word.
For example, user's depth dwell data of some search engine includes 2329880 datas, including effective query altogether
Being clicked under multimedia resource (vid), current queries word under word (query), current queries word is clicked multimedia resource
The synthesis for being clicked multimedia resource being clicked under number (count) and current queries word finish playing than (perc).
Certain customers' depth dwell data in user's depth dwell data of the full dose query word can be as described in Table 1：
User's depth dwell data example of 1 full dose query word of table
| query | vid | count | perc |
| Red rice note tears machine open | 209907159 | 1 | 0.0442 |
| Red rice note tears machine open | 213535395 | 1 | 0.0587 |
| Red rice note tears machine open | 217417432 | 2 | 0.1470 |
| … | … | … | … |
As shown in table 1, simple Geostatistics analysis is carried out by user's depth dwell data to the full dose query word, it should be able to
Enough know that the number of individual query word is 775734 (that is, the number of full dose query word is 775734), is clicked multimedia resource
Be clicked number be 6330210.
Step S140, according to user's depth dwell data of full dose query word and original evaluation index, to search engine
Quality carries out original assessment.
After the user's depth dwell data for obtaining full dose query word, the full dose query word to being obtained can be passed through
User's depth dwell data carries out simple statistical analysis to obtain above-mentioned original evaluation index, and original assessment is to utilize to be obtained
The statistical property of original value of user's depth dwell data come the quality of the search engine to multimedia resource original comment
Estimate, wherein the original evaluation index that the quality for the search engine to multimedia resource carries out original assessment may include：
It is independently clicked the number (Independent Clicked Video Count, abbreviation ICVC) of multimedia resource,
The number of the independent multimedia resource clicked by all query words.The index reflects backstage multimedia money on the whole
The searched derived degree in source.
The mean number (Average Clicked Video Count, the letter that are clicked multimedia resource of each query word
Claim ACVC), i.e., each query word can averagely click the how many a multimedia resources of export, that is, each query word be clicked it is more
The average value of the number of media resource.The index reflects the searched derived degree of backstage multimedia resource from individual.
Less than number (the Query Count under Count of the query word of multimedia resource number threshold value
Threshold, abbreviation QCUCT), that is, it is clicked the query word of the number of multimedia resource less than multimedia resource number threshold value
Number.Preliminary correlation is not had by the query word scale of " morbid state is presented ", i.e. search result in index reflection search engine
And the case where attraction.Wherein it is possible in conjunction with practical business and consider resource allocation that multimedia resource is flexibly arranged
Number threshold value.For example, for the first time multimedia resource number threshold value can be set to the multimedia resource that is clicked of each query word
The first quartile of number distribution.
Multimedia resource finishes playing than population mean (Average Video Perc, abbreviation AVP), i.e., user is more
The time span for the multimedia resource watched on media resource result of page searching the multimedia resource watched it is total when
Between percentage in length average value.The index reflection good and bad degree of the content quality of search-engine results.
It finishes playing than number (the Query Count under Perc of the query word of threshold value less than multimedia resource
Threshold, abbreviation QCUPT), i.e., multimedia resource is watched the number of few query word, that is, multimedia resource plays
Ratio is completed to finish playing than the number of the query word of threshold value less than multimedia resource.Include in index reflection search engine
The query word scale of " ill content ", i.e. search result do not have the case where depth correlation and attraction.Wherein it is possible to combine real
Border business simultaneously considers resource allocation and flexibly multimedia resource is arranged to finish playing to compare threshold value.For example, for the first time can will be more
Media resource plays are completed to be clicked the finishing playing than the of distribution of multimedia resource than what threshold value was set as each query word
One quartile.
For example, simple statistics analysis is carried out by user's depth dwell data to full dose query word shown in above-mentioned table 1,
It can obtain original evaluation index as shown in table 2 below.
The original evaluation index of 2 certain search engine of table
By above-mentioned table 2：(1) since multimedia resource number threshold value is 2, is less than the multimedia resource number threshold value
The number QCUCT of query word be 450519 and the number of individual query word as described above is 775734, it is therefore, independent
In query word be more than more than half (that is,) query word the number for being clicked multimedia resource all at 2
Below.This illustrates that user often searches for a query word, and the multimedia resource clicked is very few.This reflects the search of search engine
As a result it does badly in terms of preliminary correlation and attraction, search result may also have in terms of hit degree or diversity
Shortcoming.Concrete analysis for this problem is needed to distinguish the classification of query word, be led for example, query word is divided into
Class of navigating query word, info class query word and interactive class query word.Different classes of query word, click behavior are different.
(2) it is 32.98% to be clicked finishing playing than population mean AVP for multimedia resource, and due to multimedia
It is 7.49% that resource, which finishes playing than threshold value, the number QCUPT than the query word of threshold value that finishes playing less than multimedia resource is
194020 and the number of individual query word as described above be 775734, therefore, close to a quarter in individual query word
(that is,) query word multimedia resource search result viewing time of length no more than 7.49%.This
Illustrate that the quality of the search result of search engine can not enable user be satisfied with.The analysis of this problem is needed to reject and whether is deposited
Behavior is watched (it should be noted that if the broadcasting of the multimedia resource under same query word in a large amount of non-genuine search
Time span is too short, then the search should not be true search behavior).
The method for evaluating quality of the search engine of the embodiment of the present invention, according to user's depth dwell data of full dose query word
Come to carry out original assessment to the quality of search engine with original evaluation index, the daily prison to the original evaluation index can be passed through
It surveys and directly to carry out total evaluation to the actual mass of the search engine of multimedia resource rapidly, thus, it is possible to be not necessarily into pedestrian
Work mark objective is in time assessed the quality of search engine.
Embodiment 2
In above-described embodiment one, according to user's depth dwell data of full dose query word and original evaluation index come to searching
It indexes the quality held up and carries out original assessment, however, the original assessment may be clicked multimedia resource and be clicked without utilizing
Multimedia resource finish playing than integrated information, that is to say, that the original assessment may not provide user's depth stop
Degree of integration.In this way so that there are many number for being clicked multimedia resource for example in a query word and each more matchmaker
Body resource finish playing than in the case of all very low, it is for another example seldom in the number for being clicked multimedia resource of a query word
And each multimedia resource finishes playing than in the case of very high, being likely difficult to compare user at which using original assessment
Stop degree higher on the search result of one query word.Also, the interface interacted with search engine in view of user is every
The query word of secondary input, therefore, it is necessary to be assessed (comprehensive assessment) the quality of search engine using the integrated information.Base
In this, the present invention provides the embodiments two that following quality to search engine carry out comprehensive assessment.
Fig. 2 shows the flow charts of the method for evaluating quality of according to embodiments of the present invention two search engine.In Fig. 2 label with
Step identical Fig. 1 function having the same omits the detailed description to these steps for simplicity.
As shown in Fig. 2, the method for evaluating quality of search engine shown in Fig. 2 and the quality of search engine shown in FIG. 1 are commented
The main distinction for estimating method is, other than including step S100, step S120 and the step S140 in above-described embodiment one,
Can also include：
Step S220, according to user's depth dwell data of single query word, the user's depth for calculating single query word is stopped
Stay index.
Specifically, be clicked multimedia resource and be clicked multimedia resource finish playing than integrated information for example may be used
With (same including the number VidCount for being independently clicked multimedia resource, the number ClickCount for being clicked multimedia resource
One multimedia resource may be clicked repeatedly), multimedia resource finishes playing than average value AveragePerc.Therefore, such as
The number for being independently clicked multimedia resource of some query word of fruit is more, it is higher, each to be clicked the number of multimedia resource
Multimedia resource finishes playing than bigger, then it is higher to stop degree for user's depth of the query word.
In one possible implementation, it can indicate that user's depth stops index using sigmoid functions
DeepLinger, that is, user's depth dwell data according to the single query word calculates the use of the single query word
Family depth stops index, including：According to user's depth dwell data of the single query word and use formula
(formula 1), the user's depth for calculating the single query word stop index, wherein y is user's depth of the single query word
Index is stopped, x=VidCount*ClickCount*AveragePerc (formula 2), VidCount are independently to be clicked multimedia
The number of resource, ClickCount are the numbers for being clicked multimedia resource, and AveragePerc is playing for multimedia resource
At than average value.
For example, calculating user using above-mentioned sigmoid functions by user's depth dwell data to single query word
Depth stops index D eepLinger, and the user's depth that can obtain each query word as described in Table 3 stops index.
User's depth of 3 each query word of table stops index
| query | VidCount | ClickCount | AveragePerc | DeepLinger |
| Red rice note tears machine open | 2 | 4 | 0.1164 | 0.4347 |
| Tan lay Buddhist's the heart channel of Hang-Shaoyin | 17 | 17 | 0.0005 | 0.0704 |
| Guo De guiding principles I to pass through | 4 | 6 | 0.6927 | 1.0000 |
In one possible implementation, it can indicate that user's depth is stopped using overall situation Max-Min normalized functions
Stay index D eepLinger, that is, user's depth dwell data according to the single query word calculates the single inquiry
User's depth of word stops index, including：According to user's depth dwell data of the single query word and using formula y=
VidCountN*ClickCountN*AveragePercN (formula 3), the user's depth stop for calculating the single query word refer to
Number, wherein y is that user's depth of the single query word stops index,
VidCount is the number for being independently clicked multimedia resource, and ClickCount is time for being clicked multimedia resource
Number, AveragePerc are that multimedia resource finishes playing than average value, and min () is minimized, and max () is maximized.
In one possible implementation, the linear averaging of the number of clicks based on multimedia resource can be used to broadcasting
It discharges into than summation, to indicate that user's depth stops index D eepLinger, that is, the user according to the single query word
Depth dwell data, the user's depth for calculating the single query word stop index, including：According to the use of the single query word
Family depth dwell data simultaneously uses formula(formula 7) calculates described single look into
The user's depth for asking word stops index, wherein y is that user's depth of the single query word stops index, and VidCount is only
The vertical number for being clicked multimedia resource, AllVidCount is the multimedia resource searched using the single query word
The summation of all numbers of clicks, AveragePerc are that finishing playing for multimedia resource compares average value.
Step S240, index and comprehensive assessment index are stopped according to user's depth, the quality of search engine is integrated
Assessment.
Specifically, it is based on above-mentioned calculated user's depth and stops index, can be come using following comprehensive assessment index pair
The quality of search engine carries out comprehensive assessment：
User's depth stops exponential average (Average Deep Linger Index, abbreviation ADLI), i.e. user's depth
The average value of index, that is, average stop degree of the user on each query word are stopped, which reflects from individual searches
Index holds up the quality for being supplied to the search result of user.
Less than number (the Query Count under Deep Linger for the query word that user's depth stops index threshold
Threshold, abbreviation QCUDLT), i.e., user's depth stops that index stops the query word of index threshold less than user's depth
Number, that is, user's depth stops the number of degree too low query word, and " morbid state knot is returned in index reflection search engine
The case where uncomprehensive correlation of the query word scale of fruit ", i.e. search result and attraction.Wherein it is possible in conjunction with practical business
And considers resource allocation and flexibly to be arranged depth stop index threshold.For example, can depth be stopped index threshold for the first time
The depth for being set as each query word stops the first quartile of exponential distribution.
It stops exponential average that is, above-mentioned comprehensive assessment index may include user's depth and is less than user's depth
Stop at least one of the number of query word of index threshold.
For example, index and comprehensive assessment index as described in Table 4 can be stopped by above-mentioned user's depth, to searching
It indexes the quality held up and carries out comprehensive assessment.
The comprehensive assessment index of 4 certain search engine of table
| Index | Actual value |
| User's depth stops exponential average ADLI | 0.395 |
| Less than the Query amounts QCUDLT that depth stops index threshold | 191218 (threshold values 0.062) |
By above-mentioned table 3 and table 4：(1) user's depth of query word " Tan lay Buddhist's the heart channel of Hang-Shaoyin " stops degree far away from looking into
The user's depth for asking word " red rice note tears machine open " stops user's depth of degree and query word " Guo De guiding principles I to pass through " and stops journey
Degree, this illustrates that the possible very poor or query word the search behavior of the search result of query word " Tan lay Buddhist's the heart channel of Hang-Shaoyin " may be non-real
Real search behavior.(2) it is 0.395 that user's depth, which stops exponential average ADLI, this illustrates that the search result of search engine is whole
Performance may be behaved like with query word " red rice note tears machine open ".(3) for example, being if user's depth stops index threshold
0.062, then the number for being less than the query word that user's depth stops index threshold is about a quarter of the number of full dose query word
(that is,).Index threshold is stopped according to user's depth it is found that the performance of the search result of such query word may
It is similar to the performance of query word " Tan lay Buddhist's the heart channel of Hang-Shaoyin ".
That is, the score for the judge that above-mentioned comprehensive assessment is carried out dependent on user by viewing behavior, the synthesis
Assessment is the result that user carries out the content quality of hit, sequence, diversity and multimedia resource Comprehensive Evaluation.Certainly, also
It is full for the entirety of the search engine of multimedia resource user can promptly to be assessed using above-mentioned two comprehensive assessment index
Meaning degree.
It should be noted that the present embodiment with first carry out it is original assessment carry out again comprehensive assessment (that is, step S100,
Step S220, S240 is executed again after S120, S140) for be illustrated, however, those skilled in the art should be able to be much of that
Solution, the invention is not limited thereto, original assessment and comprehensive assessment is carried out for example, can intersect, for another example, in order to more quickly to searching
It indexes in the case that the quality held up assessed, can only carry out original assessment, for another example, in order to improve the quality of search engine
In the case of the accuracy of assessment, comprehensive assessment can be only carried out.
The method for evaluating quality of the search engine of the embodiment of the present invention, according to user's depth dwell data of full dose query word
Come to carry out original assessment to the quality of search engine with original evaluation index, and index and comprehensive assessment are stopped according to user's depth
Index to carry out comprehensive assessment to the quality of search engine, thus can not only be not necessarily to progress and manually mark, is objective right in time
The quality of search engine is assessed, but also can be stopped index by user's depth and directly be compared any two query word
Under search engine search result good and bad degree, so as to improve search engine quality evaluation accuracy.
Embodiment 3
Fig. 3 is the structure diagram according to the quality assessment device of the search engine of the embodiment of the present invention three.The present embodiment carries
The quality evaluation side for the search engine that the quality assessment device 300 of the search engine of confession provides for realizing embodiment illustrated in fig. 1
Method.As shown in figure 3, the quality assessment device 300 of the search engine may include：
Acquiring unit 320, user's depth dwell data for obtaining single query word from user journal.In the present invention
In, the depth of user stops behavior and may include：(1) stop of the user on the result of page searching of search engine, i.e. user
Click the behavior of the search result of multiple multimedia resources such as video, audio；And (2) user broadcasting in search engine
Put the stop on the page, the i.e. behavior of multimedia resource of user's viewing such as video, audio.
Specifically, four-tuple { query, vids, percs, δ } can be used to stop user's depth of each query word
Behavior is stayed to be portrayed.In other words, single inquiry can be obtained from user journal according to the data model of single query word
User's depth dwell data of word.The process may include carrying out pretreatment and noise removal processing to user journal data, use
The noise of family daily record data may be from illegally inputting, the various aspects of system exception, recording exceptional etc..
Wherein, query is query word, i.e., user inputs in the search each time of search engine, for example, can draw from search
The query word query of user is obtained in the user journal held up.
Vids is to click multimedia resource set, i.e., user clicks more matchmakers by search query word in result of page searching
The set of body resource, for example, can be seen from the multimedia resource of user journal by limiting the source of multimedia resource viewing
It sees to obtain in daily record and clicks multimedia resource set vids.
Percs be multimedia resource finish playing than set, that is, be clicked multimedia resource finish playing than set,
For example, daily record can be watched from the multimedia resource of user journal by carrying out after-treatment to multimedia resource played data
Middle acquisition multimedia resource finishes playing than set percs.It should be noted that since the total time of each multimedia resource is long
Degree may differ by it is larger, therefore, using multimedia resource finish playing than come to user's depth stop behavior portrayed than list
It is purely more objective to be portrayed the stop behavior of user's depth using the reproduction time length of multimedia resource.For example, being directed to
The same query word, if one is clicked multimedia resource and has been played repeatedly, this is clicked broadcasting for multimedia resource
Discharge into than that should be a comprehensive score, for example, can take the query word it is all finish playing than average value, for another example,
Can take the query word it is all finish playing than median etc..
δ is to be clicked multimedia resource set at most media resource plays to complete mapping function than set, for example, can be with
It is finished playing obtaining multimedia resource than pre-defining the mapping function when set.
That is, user's depth dwell data of above-mentioned single query word may include：Query word (query), by point
Hit multimedia resource set (vids), multimedia resource finishes playing than set (percs) and is clicked multimedia resource collection
Multimedia resource is closed to finish playing than the mapping function (δ) of set.
Obtaining unit 340 is connect with acquiring unit 320, for user's depth dwell data according to single query word, is obtained
Obtain user's depth dwell data of full dose query word.
For example, can carry out summarizing polymerization by user's depth dwell data of the single query word to getting, to obtain
Obtain user's depth dwell data of full dose query word.For example, the process may include carrying out after-treatment to data (to obtain
Count field datas) and denoising etc..
Specifically, four-tuple { query, vid, count, perc } can be used to carve the quality of search engine
It draws.In other words, user's depth dwell data (that is, user's depth dwell data of full dose query word) of entire search engine can be with
Including this four fields of query, vid, count and perc.Wherein, query is full dose query word；Vid is current query
Under be clicked multimedia resource；Count be under current query be clicked multimedia resource be clicked number；Perc is
The synthesis for being clicked multimedia resource under current query finishes playing ratio.
That is, user's depth dwell data of above-mentioned full dose query word may include：Full dose query word, current queries
Being clicked under multimedia resource, current queries word under word is clicked being clicked number and currently looking into for multimedia resource
The synthesis for being clicked multimedia resource ask under word finishes playing ratio.
Specific example may refer to the associated description of step S120 in above-described embodiment one.
Original assessment unit 360 is connect with obtaining unit 340, for stopping number according to user's depth of full dose query word
According to original evaluation index, original assessment is carried out to the quality of search engine.
After the user's depth dwell data for obtaining full dose query word, the full dose query word to being obtained can be passed through
User's depth dwell data carries out simple statistical analysis to obtain above-mentioned original evaluation index, and original assessment is to utilize to be obtained
The statistical property of original value of user's depth dwell data come the quality of the search engine to multimedia resource original comment
Estimate, wherein the original evaluation index that the quality for the search engine to multimedia resource carries out original assessment may include：
It is independently clicked the number (Independent Clicked Video Count, abbreviation ICVC) of multimedia resource,
The number of the independent multimedia resource clicked by all query words.The index reflects backstage multimedia money on the whole
The searched derived degree in source.
The mean number (Average Clicked Video Count, the letter that are clicked multimedia resource of each query word
Claim ACVC), i.e., each query word can averagely click the how many a multimedia resources of export, that is, each query word be clicked it is more
The average value of the number of media resource.The index reflects the searched derived degree of backstage multimedia resource from individual.
Less than number (the Query Count under Count of the query word of multimedia resource number threshold value
Threshold, abbreviation QCUCT), that is, it is clicked the query word of the number of multimedia resource less than multimedia resource number threshold value
Number.Preliminary correlation is not had by the query word scale of " morbid state is presented ", i.e. search result in index reflection search engine
And the case where attraction.Wherein it is possible in conjunction with practical business and consider resource allocation that multimedia resource is flexibly arranged
Number threshold value.For example, for the first time multimedia resource number threshold value can be set to the multimedia resource that is clicked of each query word
The first quartile of number distribution.
Multimedia resource finishes playing than population mean (Average Video Perc, abbreviation AVP), i.e., user is more
The time span for the multimedia resource watched on media resource result of page searching the multimedia resource watched it is total when
Between percentage in length average value.The index reflection good and bad degree of the content quality of search-engine results.
It finishes playing than number (the Query Count under Perc of the query word of threshold value less than multimedia resource
Threshold, abbreviation QCUPT), i.e., multimedia resource is watched the number of few query word, that is, multimedia resource plays
Ratio is completed to finish playing than the number of the query word of threshold value less than multimedia resource.Include in index reflection search engine
The query word scale of " ill content ", i.e. search result do not have the case where depth correlation and attraction.Wherein it is possible to combine real
Border business simultaneously considers resource allocation and flexibly multimedia resource is arranged to finish playing to compare threshold value.For example, for the first time can will be more
Media resource plays are completed to be clicked the finishing playing than the of distribution of multimedia resource than what threshold value was set as each query word
One quartile.
Specific example may refer to the associated description of step S140 in above-described embodiment one.
The quality assessment device of the search engine of the embodiment of the present invention, what original assessment unit was obtained according to obtaining unit
User's depth dwell data of full dose query word and original evaluation index to carry out original assessment to the quality of search engine, can
By to the original evaluation index it is daily monitoring come rapidly directly to the actual mass of the search engine of multimedia resource into
Row total evaluation, thus, it is possible to without carry out manually mark, objective the quality of search engine is assessed in time.
Embodiment 4
Fig. 4 is the structure diagram according to the quality assessment device of the search engine of the embodiment of the present invention four.The present embodiment carries
The quality evaluation side for the search engine that the quality assessment device 400 of the search engine of confession provides for realizing embodiment illustrated in fig. 2
Method.Wherein, component identical with Fig. 3 labels in Fig. 4, including：Acquiring unit 320, obtaining unit 340 and original assessment unit
360, have and omits the detailed description to these components for simplicity with aforementioned essentially identical function.
In addition, by comparing Fig. 3 and Fig. 4 it is found that the main distinction of embodiment illustrated in fig. 4 embodiment as shown in figure 3 is,
On the basis of embodiment shown in Fig. 3, the quality assessment device 400 of the search engine can also include：
Computing unit 420 is connect with acquiring unit 320, for user's depth dwell data according to single query word, meter
The user's depth for calculating single query word stops index.
Specifically, be clicked multimedia resource and be clicked multimedia resource finish playing than integrated information for example may be used
With (same including the number VidCount for being independently clicked multimedia resource, the number ClickCount for being clicked multimedia resource
One multimedia resource may be clicked repeatedly), multimedia resource finishes playing than average value AveragePerc.Therefore, such as
The number for being independently clicked multimedia resource of some query word of fruit is more, it is higher, each to be clicked the number of multimedia resource
Multimedia resource finishes playing than bigger, then it is higher to stop degree for user's depth of the query word.
In one possible implementation, it can indicate that user's depth stops index using sigmoid functions
DeepLinger, that is, user's depth dwell data according to the single query word calculates the use of the single query word
Family depth stops index, including：According to user's depth dwell data of the single query word and use formula
(formula 1), the user's depth for calculating the single query word stop index, wherein y is user's depth of the single query word
Index is stopped, x=VidCount*ClickCount*AveragePerc (formula 2), VidCount are independently to be clicked multimedia
The number of resource, ClickCount are the numbers for being clicked multimedia resource, and AveragePerc is playing for multimedia resource
At than average value.
Specific example may refer to the associated description of step S220 in above-described embodiment two.
In one possible implementation, it can indicate that user's depth is stopped using overall situation Max-Min normalized functions
Stay index D eepLinger, that is, user's depth dwell data according to the single query word calculates the single inquiry
User's depth of word stops index, including：According to user's depth dwell data of the single query word and using formula y=
VidCountN*ClickCountN*AveragePercN (formula 3), the user's depth stop for calculating the single query word refer to
Number, wherein y is that user's depth of the single query word stops index,
VidCount is the number for being independently clicked multimedia resource, and ClickCount is time for being clicked multimedia resource
Number, AveragePerc are that multimedia resource finishes playing than average value, and min () is minimized, and max () is maximized.
In one possible implementation, the linear averaging of the number of clicks based on multimedia resource can be used to broadcasting
It discharges into than summation, to indicate that user's depth stops index D eepLinger, that is, the user according to the single query word
Depth dwell data, the user's depth for calculating the single query word stop index, including：According to the use of the single query word
Family depth dwell data simultaneously uses formula(formula 7) calculates described single look into
The user's depth for asking word stops index, wherein y is that user's depth of the single query word stops index, and VidCount is only
The vertical number for being clicked multimedia resource, AllVidCount is the multimedia resource searched using the single query word
The summation of all numbers of clicks, AveragePerc are that finishing playing for multimedia resource compares average value.
Comprehensive assessment unit 440 is connect with computing unit 420, for stopping index and comprehensive assessment according to user's depth
Index carries out comprehensive assessment to the quality of search engine.
Specifically, it is based on above-mentioned calculated user's depth and stops index, can be come using following comprehensive assessment index pair
The quality of search engine carries out comprehensive assessment：
User's depth stops exponential average (Average Deep Linger Index, abbreviation ADLI), i.e. user's depth
The average value of index, that is, average stop degree of the user on each query word are stopped, which reflects from individual searches
Index holds up the quality for being supplied to the search result of user.
Less than number (the Query Count under Deep Linger for the query word that user's depth stops index threshold
Threshold, abbreviation QCUDLT), i.e., user's depth stops that index stops the query word of index threshold less than user's depth
Number, that is, user's depth stops the number of degree too low query word, and " morbid state knot is returned in index reflection search engine
The case where uncomprehensive correlation of the query word scale of fruit ", i.e. search result and attraction.Wherein it is possible in conjunction with practical business
And considers resource allocation and flexibly to be arranged depth stop index threshold.For example, can depth be stopped index threshold for the first time
The depth for being set as each query word stops the first quartile of exponential distribution.
It stops exponential average that is, above-mentioned comprehensive assessment index may include user's depth and is less than user's depth
Stop at least one of the number of query word of index threshold.
Specific example may refer to the associated description of step S240 in above-described embodiment two.
It should be noted that the present embodiment with elder generation by original assessment unit carry out it is original assessment again by comprehensive assessment unit into
It is illustrated for row comprehensive assessment, however, those skilled in the art should be able to understand, the invention is not limited thereto, for example, can
Original assessment and comprehensive assessment are carried out to be intersected by original assessment unit and comprehensive assessment unit, for another example, in order to more quickly
In the case of assessing the quality of search engine, can original assessment only be carried out by original assessment unit, for another example, in order to
In the case of the accuracy for improving the quality evaluation of search engine, can comprehensive assessment only be carried out by comprehensive assessment unit.
The quality assessment device of the search engine of the embodiment of the present invention, what original assessment unit was obtained according to obtaining unit
User's depth dwell data of full dose query word and original evaluation index to carry out original assessment to the quality of search engine, and
Comprehensive assessment unit stops index and comprehensive assessment index come to search engine according to the calculated user's depth of computing unit
Quality carries out comprehensive assessment, thus can not only be not necessarily to progress and manually mark, is objective in time to the progress of the quality of search engine
Assessment, but also the search that index directly compares the search engine under any two query word can be stopped by user's depth
As a result good and bad degree, so as to improve search engine quality evaluation accuracy.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, protection scope of the present invention should be based on the protection scope of the described claims.
Claims (10)
1. a kind of method for evaluating quality of search engine, described search engine is used for searching multimedia resource, which is characterized in that institute
Stating method for evaluating quality includes：
User's depth dwell data of single query word is obtained from user journal, wherein user's depth of the single query word
Dwell data includes：Query word, be clicked multimedia resource set, multimedia resource finish playing than set and the quilt
Multimedia resource set is clicked to finish playing than the mapping function of set to the multimedia resource；
According to user's depth dwell data of the single query word, user's depth dwell data of full dose query word is obtained,
In, user's depth dwell data of the full dose query word includes：It is clicked multimedia under full dose query word, current queries word
Being clicked under number and current queries word for multimedia resource that be clicked under resource, current queries word is clicked more matchmakers
The synthesis of body resource finishes playing ratio；And
According to user's depth dwell data of the full dose query word and original evaluation index, to the quality of described search engine into
The original assessment of row,
Wherein, the original evaluation index include the number for being independently clicked multimedia resource, each query word be clicked it is more
The mean number of media resource, the number of query word less than multimedia resource number threshold value, multimedia resource finish playing ratio
Population mean finishes playing less than multimedia resource than at least one of the number of query word of threshold value.
2. method for evaluating quality according to claim 1, which is characterized in that further include：
According to user's depth dwell data of the single query word, the user's depth stop for calculating the single query word refers to
Number；And
Index and comprehensive assessment index are stopped according to user's depth, comprehensive assessment is carried out to the quality of described search engine,
Wherein, the comprehensive assessment index includes that user's depth stops exponential average and stops index threshold less than user's depth
At least one of the number of query word.
3. method for evaluating quality according to claim 2, which is characterized in that the user according to the single query word
Depth dwell data, the user's depth for calculating the single query word stop index, including：
According to user's depth dwell data of the single query word and use formulaCalculate the single query word
User's depth stop index,
Wherein, y is that user's depth of the single query word stops index,
X=VidCount*ClickCount*AveragePerc, VidCount are the numbers for being independently clicked multimedia resource,
ClickCount is the number for being clicked multimedia resource, and AveragePerc is that finishing playing for multimedia resource compares average value.
4. method for evaluating quality according to claim 2, which is characterized in that the user according to the single query word
Depth dwell data, the user's depth for calculating the single query word stop index, including：
According to user's depth dwell data of the single query word and using formula y=VidCountN*ClickCountN*
AveragePercN, the user's depth for calculating the single query word stop index,
Wherein, y is that user's depth of the single query word stops index,
VidCount is the number for being independently clicked multimedia resource, and ClickCount is the number for being clicked multimedia resource,
AveragePerc is that multimedia resource finishes playing than average value, and min () is minimized, and max () is maximized.
5. method for evaluating quality according to claim 2, which is characterized in that the user according to the single query word
Depth dwell data, the user's depth for calculating the single query word stop index, including：
According to user's depth dwell data of the single query word and use formula
The user's depth for calculating the single query word stops index,
Wherein, y is that user's depth of the single query word stops index,
VidCount is the number for being independently clicked multimedia resource, and AllVidCount is searched for using the single query word
The summation of all numbers of clicks of the multimedia resource arrived, AveragePerc are that finishing playing for multimedia resource compares average value.
6. a kind of quality assessment device of search engine, described search engine is used for searching multimedia resource, which is characterized in that institute
Stating quality assessment device includes：
Acquiring unit, user's depth dwell data for obtaining single query word from user journal, wherein the single inquiry
User's depth dwell data of word includes：Query word, be clicked multimedia resource set, multimedia resource finish playing than collection
It closes and the multimedia resource set that is clicked finishes playing to the multimedia resource than the mapping function of set；
Obtaining unit is connect with the acquiring unit, for user's depth dwell data according to the single query word, is obtained
User's depth dwell data of full dose query word, wherein user's depth dwell data of the full dose query word includes：Full dose is looked into
Ask word, being clicked under multimedia resource, current queries word under current queries word is clicked being clicked time for multimedia resource
The synthesis for being clicked multimedia resource under number and current queries word finishes playing ratio；And
Original assessment unit is connect with the obtaining unit, for user's depth dwell data according to the full dose query word
With original evaluation index, original assessment is carried out to the quality of described search engine,
Wherein, the original evaluation index include the number for being independently clicked multimedia resource, each query word be clicked it is more
The mean number of media resource, the number of query word less than multimedia resource number threshold value, multimedia resource finish playing ratio
Population mean finishes playing less than multimedia resource than at least one of the number of query word of threshold value.
7. quality assessment device according to claim 6, which is characterized in that further include：
Computing unit is connect with the acquiring unit, for user's depth dwell data according to the single query word, is calculated
User's depth of the single query word stops index；And
Comprehensive assessment unit is connect with the computing unit, for being referred to according to user's depth stop index and comprehensive assessment
Mark carries out comprehensive assessment to the quality of described search engine,
Wherein, the comprehensive assessment index includes that user's depth stops exponential average and stops index threshold less than user's depth
At least one of the number of query word.
8. quality assessment device according to claim 7, which is characterized in that the computing unit is specifically used for, according to institute
It states user's depth dwell data of single query word and uses formulaCalculate user's depth of the single query word
Index is stopped,
Wherein, y is that user's depth of the single query word stops index,
X=VidCount*ClickCount*AveragePerc, VidCount are the numbers for being independently clicked multimedia resource,
ClickCount is the number for being clicked multimedia resource, and AveragePerc is that finishing playing for multimedia resource compares average value.
9. quality assessment device according to claim 7, which is characterized in that the computing unit is specifically used for, according to institute
State user's depth dwell data of single query word and using formula y=VidCountN*ClickCountN*
AveragePercN, the user's depth for calculating the single query word stop index,
Wherein, y is that user's depth of the single query word stops index,
VidCount is the number for being independently clicked multimedia resource, and ClickCount is the number for being clicked multimedia resource,
AveragePerc is that multimedia resource finishes playing than average value, and min () is minimized, and max () is maximized.
10. quality assessment device according to claim 7, which is characterized in that the computing unit is specifically used for, according to institute
It states user's depth dwell data of single query word and uses formulaCalculate institute
The user's depth for stating single query word stops index,
Wherein, y is that user's depth of the single query word stops index,
VidCount is the number for being independently clicked multimedia resource, and AllVidCount is searched for using the single query word
The summation of all numbers of clicks of the multimedia resource arrived, AveragePerc are that finishing playing for multimedia resource compares average value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510927675.5A CN105573887B (en) | 2015-12-14 | 2015-12-14 | The method for evaluating quality and device of search engine |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510927675.5A CN105573887B (en) | 2015-12-14 | 2015-12-14 | The method for evaluating quality and device of search engine |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105573887A CN105573887A (en) | 2016-05-11 |
| CN105573887B true CN105573887B (en) | 2018-07-13 |
Family
ID=55884053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510927675.5A Expired - Fee Related CN105573887B (en) | 2015-12-14 | 2015-12-14 | The method for evaluating quality and device of search engine |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105573887B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105956204B (en) * | 2016-07-01 | 2019-08-02 | 北京奇虎科技有限公司 | The method and device of session Session satisfaction assessment |
| CN106777248A (en) * | 2016-12-27 | 2017-05-31 | 努比亚技术有限公司 | A kind of search engine test evaluation method and apparatus |
| CN108733707B (en) * | 2017-04-20 | 2022-10-04 | 腾讯科技（深圳）有限公司 | Method and device for determining stability of search function |
| CN107273404A (en) * | 2017-04-26 | 2017-10-20 | 努比亚技术有限公司 | Appraisal procedure, device and the computer-readable recording medium of search engine |
| CN107688595B (en) * | 2017-05-10 | 2019-03-15 | 平安科技（深圳）有限公司 | Information retrieval Accuracy Evaluation, device and computer readable storage medium |
| CN108632670B (en) * | 2018-03-15 | 2021-03-26 | 北京奇艺世纪科技有限公司 | Video satisfaction determining method and device |
| US11682029B2 (en) | 2018-03-23 | 2023-06-20 | Lexisnexis, A Division Of Reed Elsevier Inc. | Systems and methods for scoring user reactions to a software program |
| US11301909B2 (en) * | 2018-05-22 | 2022-04-12 | International Business Machines Corporation | Assigning bias ratings to services |
| CN108897685B (en) * | 2018-06-28 | 2022-02-25 | 百度在线网络技术（北京）有限公司 | Method, device, server and medium for evaluating quality of search result |
| CN111753184B (en) * | 2019-03-29 | 2024-01-02 | 北京达佳互联信息技术有限公司 | Information recommendation method and device |
| CN110674638B (en) * | 2019-09-23 | 2023-12-01 | 百度在线网络技术（北京）有限公司 | Corpus labeling system and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1963816A (en) * | 2006-12-01 | 2007-05-16 | 清华大学 | Automatization processing method of rating of merit of search engine |
| CN102902806A (en) * | 2012-10-17 | 2013-01-30 | 深圳市宜搜科技发展有限公司 | Method and system for performing inquiry expansion by using search engine |
| CN103164537A (en) * | 2013-04-09 | 2013-06-19 | 浙江鸿程计算机系统有限公司 | Method of search engine log data mining facing user information requirements |
| CN103365839A (en) * | 2012-03-26 | 2013-10-23 | 腾讯科技（深圳）有限公司 | Recommendation search method and device for search engines |
| CN103593411A (en) * | 2013-10-23 | 2014-02-19 | 江苏大学 | Method for testing combination properties of evaluation indexes of search engines and testing device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070078820A1 (en) * | 2000-05-08 | 2007-04-05 | Eva Lana | Mindmatch: method and system for mass customization of test preparation |
-
2015
- 2015-12-14 CN CN201510927675.5A patent/CN105573887B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1963816A (en) * | 2006-12-01 | 2007-05-16 | 清华大学 | Automatization processing method of rating of merit of search engine |
| CN103365839A (en) * | 2012-03-26 | 2013-10-23 | 腾讯科技（深圳）有限公司 | Recommendation search method and device for search engines |
| CN102902806A (en) * | 2012-10-17 | 2013-01-30 | 深圳市宜搜科技发展有限公司 | Method and system for performing inquiry expansion by using search engine |
| CN103164537A (en) * | 2013-04-09 | 2013-06-19 | 浙江鸿程计算机系统有限公司 | Method of search engine log data mining facing user information requirements |
| CN103593411A (en) * | 2013-10-23 | 2014-02-19 | 江苏大学 | Method for testing combination properties of evaluation indexes of search engines and testing device |
Non-Patent Citations (1)
| Title |
|---|
| 基于聚类分析的搜索引擎自动性能评价研究;吴世勇;《中国优秀硕士学位论文全文数据库》;20120215;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105573887A (en) | 2016-05-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105573887B (en) | The method for evaluating quality and device of search engine | |
| Iofciu et al. | Identifying users across social tagging systems | |
| Lv et al. | Learning to model relatedness for news recommendation | |
| Shepitsen et al. | Personalized recommendation in social tagging systems using hierarchical clustering | |
| Dourisboure et al. | Extraction and classification of dense communities in the web | |
| CN103365839B (en) | The recommendation searching method and device of a kind of search engine | |
| CN103678618B (en) | Web service recommendation method based on socializing network platform | |
| Saez-Trumper et al. | Finding trendsetters in information networks | |
| CN106202211B (en) | Integrated microblog rumor identification method based on microblog types | |
| CN105723402A (en) | Systems and methods for determining influencers in a social data network | |
| CN104424231B (en) | The processing method and processing device of multidimensional data | |
| CN108304490A (en) | Text based similarity determines method, apparatus and computer equipment | |
| CN106708953A (en) | Discrete particle swarm optimization based local community detection collaborative filtering recommendation method | |
| Zalaghi et al. | Measuring customer loyalty using an extended RFM and clustering technique | |
| Zhou et al. | Relevance feature mapping for content-based multimedia information retrieval | |
| CN107341194A (en) | A kind of enterprise's duplication of name people differentiating method and device | |
| CN104077392B (en) | Reminding method and device are suggested in a kind of search | |
| Qiu et al. | Clustering Analysis for Silent Telecom Customers Based on K-means++ | |
| CN107341199A (en) | A kind of recommendation method based on documentation ＆ info general model | |
| CN111612519B (en) | Method, device and storage medium for identifying potential customers of financial products | |
| Jie et al. | A unified search federation system based on online user feedback | |
| CN103593355A (en) | User original content recommending method and device | |
| Lee et al. | Reinforcing recommendation using implicit negative feedback | |
| Dean-Hall et al. | Evaluating Contextual Suggestion. | |
| Kawamae | Latent interest-topic model: finding the causal relationships behind dyadic data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180713 Termination date: 20181214 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |