CN105573887A - Quality evaluation method and device of search engine - Google Patents

Abstract

The invention discloses a quality evaluation method and device of a search engine. The search engine is used for searching multimedia resources. The quality evaluation method comprises the following steps: obtaining the user depth residence data of a single query from a user log; according to the user depth residence data of the single query, obtaining the user depth residence data of a full-quantity query; and according to the user depth residence data of the full-quantity query and an original evaluation index, carrying out original evaluation on the quality of the search engine, wherein the original evaluation index comprises at least one of a number of multimedia resources which are independently clicked, an average number of the clicked multimedia resources of each query, a number of the queries which are lower than the threshold value of the number of the multimedia resources, a total average value of multimedia resource playing percent complete and a number of the queries which are lowered than the threshold value of the multimedia resource playing percent complete. The quality of the search engine can be objectively and timely evaluated without manual annotation.

Description

The method for evaluating quality of search engine and device

Technical field

The present invention relates to information search and searching field, particularly relate to a kind of method for evaluating quality and device of search engine.

Background technology

Search engine (SearchEngine) refers to according to certain strategy, the information used on specific computer program collection internet, after information being organized and processes, by the information displaying after process to user, that is, search engine is for user provides the system of retrieval service.Search engine comprises full-text index, directory index, META Search Engine, vertical search engine, aggregation type search engine, door search engine and free lists of links etc.

The quality evaluation of search engine is subject to the extensive concern of industrial community and researchist always.At present, Cranfield appraisement system is widely used in the quality evaluation of search engine, and this appraisement system is the complete evaluation and test scheme be made up of inquiry sample collection, correct option collection, these three parts of evaluation metrics.When using Cranfield appraisement system to carry out the quality evaluation of search engine, comprise following three links: first, extract representational query word (query), the query word extracted is formed the inquiry sample collection of a suitable scale; Then, for this inquiry sample collection, from the corpus of search engine, find result corresponding thereto, namely manually mark; Finally, searching system is inputted by the query word extracted with the corpus of markup information, searching system feedback result, again for the result of search engine feedback, use predefined evaluation computing formula, utilize the method quantized to the degree of closeness of the desired result of the result and mark of evaluating search engine feedback.

Wherein, there is the method for the result of multiple evaluation search engine feedback, such as accuracy rate-recall rate (Precision-Recall) method, Rating of the single value (PrecisionN) method, on average to sort inverse (MeanReciprocalRanking, be called for short MRR) method, Average Accuracy average (MeanAveragePrecision, be called for short MAP) method and lose storage gain (DiscountedCumulativeGain, be called for short DCG) method etc.

But, due to traditional information retrieval system data and portfolio usually little, retrieval input is relative specification also, therefore, it is possible to manually choose sample collection and manually mark example result (model answer), but, along with the development of internet and the increase of internet information amount, the heavy traffic of search engine on line and data magnanimity, the evaluation utilizing the mode of artificial mark answer to carry out network information retrieval system is not only a labor intensive but also time-consuming process, and the mode of artificial mark answer can not have been utilized to carry out the mark of answer.That is, the shortcoming of Cranfield appraisement system is to need manually to choose sample collection and needs manually to mark example result.

In order to solve artificial the mark not only labor intensive but also time-consuming problem of above-mentioned Cranfield appraisement system, proposing A/B and testing (A/Btesting) system.A/B test macro is when user search, automatically the packet number (BucketID) of user is determined by system, import different branch by Automatic Extraction flow, make the user of respective packets see the result that different product version (or different search engine) provides.The behavior of user under different editions product will go on record, and these behavioral datas form a series of index by data analysis, then obtain by comparing these indexs the conclusion that between each product version, which is better and which is worse.Wherein, when index calculate, can be subdivided into based on the method for expert analysis mode with based on these the two kinds of methods of method clicking statistics.

But, along with the development of Internet service, requirement for the promptness of search-engine results quality optimization is also more and more higher, traditional A/B test macro finds that the problem of search engine needs certain expert estimation time, and, due to long tail effect (LongTailEffect), the outstanding representation relating to query word in A/B test macro can not make good mapping to the outstanding representation of whole system.That is, what the problem of A/B test macro was in the face of Internet service scale is unable to do what one wishes.

In addition, the Search Results of other Rich Media (RichMedia) search engine of such as video search engine etc. has the feature of himself.User for result video satisfaction whether, can not weigh simply by hit, playback volume or sequence.In a lot of situation, user needs just can there be one by viewing video and compares objective appraisal.This makes traditional engine evaluation method based on text search cannot be applicable to the quality assessment of the video search engine of video this " deep semantic ".And on many lines, the layout of video search result page is no longer list type conventional in text search engine but grid type, and this weakens traditional position.Therefore, position-based carries out assessing being unfair.But, no matter be Cranfield appraisement system or A/B test macro, all do not provide the solution targetedly of the quality evaluation of video search engine.

Summary of the invention

technical matters

In view of this, how objective the technical problem to be solved in the present invention is, assess in time the quality of search engine.

solution

In order to solve the problems of the technologies described above, in first aspect, the invention provides a kind of method for evaluating quality of search engine, described search engine is used for searching multimedia resource, and described method for evaluating quality comprises:

User's degree of depth dwell data of single query word is obtained from user journal, wherein, user's degree of depth dwell data of described single query word comprises: query word, clicked multimedia resource set, multimedia resource finish playing and finish playing than the mapping function of set than set and described clicked multimedia resource set to described multimedia resource;

According to user's degree of depth dwell data of described single query word, obtain user's degree of depth dwell data of full dose query word, wherein, user's degree of depth dwell data of described full dose query word comprises: the ratio that comprehensively finishes playing of the clicked multimedia resource under the clicked number of times of the clicked multimedia resource under full dose query word, current queries word, the clicked multimedia resource under current queries word and current queries word; And

According to user's degree of depth dwell data and the original evaluation index of described full dose query word, original assessment is carried out to the quality of described search engine,

Wherein, described original evaluation index comprise the independent number of clicked multimedia resource, the clicked multimedia resource of each query word mean number, finish playing than population mean lower than the number of the query word of multimedia resource number threshold value, multimedia resource, finish playing than at least one in the number of the query word of threshold value lower than multimedia resource.

In conjunction with first aspect, in the implementation that the first is possible, described method for evaluating quality also comprises:

According to user's degree of depth dwell data of described single query word, the user's degree of depth calculating described single query word stops index; And

Stop exponential sum comprehensive assessment index according to described user's degree of depth, comprehensive assessment carried out to the quality of described search engine,

Wherein, described comprehensive assessment index comprise user's degree of depth stop exponential average and lower than user's degree of depth stop index threshold query word number at least one.

In conjunction with the first possible implementation of first aspect, in the implementation that the second is possible, described user's degree of depth dwell data according to described single query word, the user's degree of depth calculating described single query word stops index, comprising:

Adopt formula according to user's degree of depth dwell data of described single query word the user's degree of depth calculating described single query word stops index,

Wherein, y is that user's degree of depth of described single query word stops index,

X=VidCount*ClickCount*AveragePerc, VidCount are the numbers of independent clicked multimedia resource, and ClickCount is the number of times of clicked multimedia resource, and AveragePerc is that finishing playing of multimedia resource compares mean value.

In conjunction with the first possible implementation of first aspect, in the implementation that the third is possible, described user's degree of depth dwell data according to described single query word, the user's degree of depth calculating described single query word stops index, comprising:

Adopt formula y=VidCountN*ClickCountN*AveragePercN according to user's degree of depth dwell data of described single query word, the user's degree of depth calculating described single query word stops index,

Wherein, y is that user's degree of depth of described single query word stops index,

$VidCountN=\frac{VidCount-\min \left(VidCount\right)}{\max \left(VidCount\right)-\min \left(VidCount\right)},$

$ClickCountN=\frac{ClickCount-\min \left(ClickCount\right)}{\max \left(ClickCount\right)-\min \left(ClickCount\right)},$

$AveragePercN=\frac{AveragePerc-\min \left(AveragePerc\right)}{\max \left(AveragePerc\right)-\min \left(AveragePerc\right)},$

VidCount is the number of independent clicked multimedia resource, ClickCount is the number of times of clicked multimedia resource, AveragePerc be multimedia resource finish playing than mean value, min () gets minimum value, and max () gets maximal value.

In conjunction with the first possible implementation of first aspect, in the 4th kind of possible implementation, described user's degree of depth dwell data according to described single query word, the user's degree of depth calculating described single query word stops index, comprising:

Adopt formula according to user's degree of depth dwell data of described single query word the user's degree of depth calculating described single query word stops index,

Wherein, y is that user's degree of depth of described single query word stops index,

VidCount is the number of independent clicked multimedia resource, and AllVidCount is the summation of all numbers of clicks of the multimedia resource utilizing described single query word to search, and AveragePerc is that finishing playing of multimedia resource compares mean value.

In second aspect, the invention provides a kind of quality assessment device of search engine, described search engine is used for searching multimedia resource, and described quality assessment device comprises:

Acquiring unit, for obtaining user's degree of depth dwell data of single query word from user journal, wherein, user's degree of depth dwell data of described single query word comprises: query word, clicked multimedia resource set, multimedia resource finish playing and finish playing than the mapping function of set than set and described clicked multimedia resource set to described multimedia resource;

Obtain unit, be connected with described acquiring unit, for the user's degree of depth dwell data according to described single query word, obtain user's degree of depth dwell data of full dose query word, wherein, user's degree of depth dwell data of described full dose query word comprises: the ratio that comprehensively finishes playing of the clicked multimedia resource under the clicked number of times of the clicked multimedia resource under full dose query word, current queries word, the clicked multimedia resource under current queries word and current queries word; And

Original assessment unit, is connected with described acquisition unit, for according to user's degree of depth dwell data of described full dose query word and original evaluation index, carries out original assessment to the quality of described search engine,

Wherein, described original evaluation index comprise the independent number of clicked multimedia resource, the clicked multimedia resource of each query word mean number, finish playing than population mean lower than the number of the query word of multimedia resource number threshold value, multimedia resource, finish playing than at least one in the number of the query word of threshold value lower than multimedia resource.

In conjunction with second aspect, in the implementation that the first is possible, described quality assessment device also comprises:

Computing unit, is connected with described acquiring unit, and for the user's degree of depth dwell data according to described single query word, the user's degree of depth calculating described single query word stops index; And

Comprehensive assessment unit, is connected with described computing unit, for stopping exponential sum comprehensive assessment index according to described user's degree of depth, carries out comprehensive assessment to the quality of described search engine,

Wherein, described comprehensive assessment index comprise user's degree of depth stop exponential average and lower than user's degree of depth stop index threshold query word number at least one.

In conjunction with the first possible implementation of second aspect, in the implementation that the second is possible, described computing unit specifically for, adopt formula according to user's degree of depth dwell data of described single query word the user's degree of depth calculating described single query word stops index,

Wherein, y is that user's degree of depth of described single query word stops index,

X=VidCount*ClickCount*AveragePerc, VidCount are the numbers of independent clicked multimedia resource, and ClickCount is the number of times of clicked multimedia resource, and AveragePerc is that finishing playing of multimedia resource compares mean value.

In conjunction with the first possible implementation of second aspect, in the implementation that the third is possible, described computing unit specifically for, adopt formula y=VidCountN*ClickCountN*AveragePercN according to user's degree of depth dwell data of described single query word, the user's degree of depth calculating described single query word stops index

Wherein, y is that user's degree of depth of described single query word stops index,

$VidCountN=\frac{VidCount-\min \left(VidCount\right)}{\max \left(VidCount\right)-\min \left(VidCount\right)},$

$ClickCountN=\frac{ClickCount-\min \left(ClickCount\right)}{\max \left(ClickCount\right)-\min \left(ClickCount\right)},$

$AveragePercN=\frac{AveragePerc-\min \left(AveragePerc\right)}{\max \left(AveragePerc\right)-\min \left(AveragePerc\right)},$

VidCount is the number of independent clicked multimedia resource, ClickCount is the number of times of clicked multimedia resource, AveragePerc be multimedia resource finish playing than mean value, min () gets minimum value, and max () gets maximal value.

In conjunction with the first possible implementation of second aspect, in the 4th kind of possible implementation, described computing unit specifically for, adopt formula according to user's degree of depth dwell data of described single query word the user's degree of depth calculating described single query word stops index,

Wherein, y is that user's degree of depth of described single query word stops index,

VidCount is the number of independent clicked multimedia resource, and AllVidCount is the summation of all numbers of clicks of the multimedia resource utilizing described single query word to search, and AveragePerc is that finishing playing of multimedia resource compares mean value.

beneficial effect

The method for evaluating quality of the search engine of the embodiment of the present invention and device, according to user's degree of depth dwell data and the original evaluation index of obtained full dose query word, original assessment is carried out to the quality of search engine, thus can without the need to manually marking, objectively in time the quality of search engine to be assessed.And, stop exponential sum comprehensive assessment index according to user's degree of depth and comprehensive assessment is carried out to the quality of search engine, by the good and bad degree of the Search Results of the search engine under user's degree of depth stop index directly more any two query words, thus can also can improve the accuracy of the quality evaluation of search engine.

According to below with reference to the accompanying drawings to detailed description of illustrative embodiments, further feature of the present invention and aspect will become clear.

Accompanying drawing explanation

Comprise in the description and form the accompanying drawing of a part for instructions and instructions together illustrates exemplary embodiment of the present invention, characteristic sum aspect, and for explaining principle of the present invention.

Fig. 1 illustrates the process flow diagram of the method for evaluating quality of the search engine according to the embodiment of the present invention one;

Fig. 2 illustrates the process flow diagram of the method for evaluating quality of the search engine according to the embodiment of the present invention two;

Fig. 3 illustrates the structured flowchart of the quality assessment device of the search engine according to the embodiment of the present invention three; And

Fig. 4 illustrates the structured flowchart of the quality assessment device of the search engine according to the embodiment of the present invention four.

Embodiment

Various exemplary embodiment of the present invention, characteristic sum aspect is described in detail below with reference to accompanying drawing.The same or analogous element of Reference numeral presentation function identical in accompanying drawing.Although the various aspects of embodiment shown in the drawings, unless otherwise indicated, accompanying drawing need not be drawn in proportion.

Word " exemplary " special here means " as example, embodiment or illustrative ".Here need not be interpreted as being better than or being better than other embodiment as any embodiment illustrated by " exemplary ".

In addition, in order to better the present invention is described, in embodiment hereafter, give numerous details.It will be appreciated by those skilled in the art that do not have some detail, the present invention can implement equally.In some instances, the method known for those skilled in the art, means, element and circuit are not described in detail, so that highlight purport of the present invention.

embodiment 1

Fig. 1 illustrates the process flow diagram of the method for evaluating quality of the search engine according to the embodiment of the present invention one.As shown in Figure 1, this method for evaluating quality specifically can comprise:

Step S100, obtain user's degree of depth dwell data of single query word from user journal.

In the present invention, the degree of depth of user stops behavior and can comprise: the stop of (1) user on the result of page searching of search engine, and namely user clicks the behavior of the Search Results of the multimedia resource of multiple such as video, audio frequency etc.; And the stop of (2) user on the broadcasting page of search engine, namely user watches the behavior of the multimedia resource of such as video, audio frequency etc.

Particularly, four-tuple { query, vids, percs, δ } can be used to portray the behavior that stops of user's degree of depth of each query word.In other words, user's degree of depth dwell data of single query word can be obtained from user journal according to the data model of single query word.This process can comprise carries out pre-service and noise removal process to user journal data, and the noise of user journal data may from the many-side of such as illegally input, system exception, recording exceptional etc.

Wherein, query is query word, and namely user inputs in the search each time of search engine, such as, can obtain the query word query of user from the user journal of search engine.

Vids is for clicking multimedia resource set, namely user clicks the set of multimedia resource at result of page searching by search query word, such as, click multimedia resource set vids can be obtained by the source limiting multimedia resource viewing from the multimedia resource viewing daily record of user journal.

Percs is that multimedia resource finishes playing than set, namely clicked multimedia resource finish playing than set, such as, multimedia resource can be obtained finish playing than set percs from the multimedia resource of user journal viewing daily record by carrying out secondary treating to multimedia resource played data.It should be noted that, because the T.T. length of each multimedia resource may differ larger, therefore, multimedia resource is used to finish playing more objective than merely using the reproduction time length of multimedia resource to portray the behavior that stops of user's degree of depth than portraying user's degree of depth stop behavior.Such as, for same query word, if clicked multimedia resource is played repeatedly, then this clicked multimedia resource finish playing than being an integrate score, such as, can get this query word all finish playing than mean value, and for example, can get this query word all finish playing than median etc.

δ be clicked multimedia resource set at the most media resource plays complete than set mapping function, such as, can obtain multimedia resource finish playing than set time pre-define this mapping function.

That is, user's degree of depth dwell data of above-mentioned single query word can comprise: query word (query), clicked multimedia resource set (vids), multimedia resource finish playing and to finish playing than the mapping function (δ) gathered to multimedia resource than set (percs) and clicked multimedia resource set.

Step S120, user's degree of depth dwell data according to above-mentioned single query word, obtain user's degree of depth dwell data of full dose query word.

Such as, by gathering polymerization to user's degree of depth dwell data of the single query word got, user's degree of depth dwell data of full dose query word can be obtained.Such as, this process can comprise and carries out secondary treating (obtain count field data) and denoising etc. to data.

Particularly, { query, vid, count, the perc} quality to search engine is portrayed can to use four-tuple.In other words, user's degree of depth dwell data (that is, user's degree of depth dwell data of full dose query word) of whole search engine can comprise these four fields of query, vid, count and perc.Wherein, query is full dose query word; Vid is the clicked multimedia resource under current query; Count is the clicked number of times of the clicked multimedia resource under current query; Perc is the ratio that comprehensively finishes playing of the clicked multimedia resource under current query.

That is, user's degree of depth dwell data of above-mentioned full dose query word can comprise: the ratio that comprehensively finishes playing of the clicked multimedia resource under the clicked number of times of the clicked multimedia resource under full dose query word, current queries word, the clicked multimedia resource under current queries word and current queries word.Such as, user's degree of depth dwell data of whole search engine can comprise following three four-tuple { query, vid, count, perc}:{A1,0001,500,80%}, { A2,0002,100,70%} and { A3,0003,200,90%}, wherein, A1, A2 and A3 are full dose query word.

Such as, user's degree of depth dwell data of certain search engine comprises altogether 2329880 data, comprises comprehensively finishing playing than (perc) of the clicked multimedia resource under the clicked multimedia resource (vid) under effective query word (query), current queries word, the clicked number of times (count) of the clicked multimedia resource under current queries word and current queries word.Certain customers' degree of depth dwell data in user's degree of depth dwell data of this full dose query word can as described in Table 1:

User's degree of depth dwell data example of table 1 full dose query word

query	vid	count	perc 6 -->
				Red rice note tears machine open	209907159	1	0.0442
Red rice note tears machine open	213535395	1	0.0587
				Red rice note tears machine open	217417432	2	0.1470
…	…	…	…

As shown in table 1, by carrying out statistical study simply to user's degree of depth dwell data of this full dose query word, the number should be able to knowing independent query word is 775734 (that is, the number of full dose query word is 775734), and the clicked number of times of clicked multimedia resource is 6330210.

Step S140, according to user's degree of depth dwell data of full dose query word and original evaluation index, original assessment is carried out to the quality of search engine.

After the user's degree of depth dwell data obtaining full dose query word, above-mentioned original evaluation index can be obtained by carrying out simple statistical study to user's degree of depth dwell data of obtained full dose query word, original assessment utilizes the statistical property of the original value of the user's degree of depth dwell data obtained to carry out original assessment to the quality of the search engine of multimedia resource, wherein, the original evaluation index carrying out original assessment for the quality of the search engine to multimedia resource can comprise:

The number (IndependentClickedVideoCount is called for short ICVC) of independent clicked multimedia resource, the number of the independent multimedia resource namely clicked by all query words.This index reflects the degree of the searched derivation of backstage multimedia resource on the whole.

Mean number (the AverageClickedVideoCount of the clicked multimedia resource of each query word, be called for short ACVC), namely each query word on average can be clicked derives how many multimedia resources, that is the mean value of the number of the clicked multimedia resource of each query word.This index reflects the degree of the searched derivation of backstage multimedia resource from individuality.

Lower than the number (QueryCountunderCountThreshold, be called for short QCUCT) of the query word of multimedia resource number threshold value, namely the number of clicked multimedia resource is lower than the number of the query word of multimedia resource number threshold value.This index reflects by the query word scale of " morbid state presents " in search engine, and namely Search Results does not have the situation of preliminary correlativity and attractive force.Wherein, Resourse Distribute can be considered in conjunction with practical business multimedia resource number threshold value is set flexibly.Such as, multimedia resource number threshold value can be set to the first quartile of the number distribution of the clicked multimedia resource of each query word for the first time.

Multimedia resource finishes playing than population mean (AverageVideoPerc, be called for short AVP), namely user's time span of multimedia resource of watching on multimedia resource result of page searching at this by the mean value of the number percent of the T.T. of the multimedia resource watched in length.This index reflects the good and bad degree of the content quality of search-engine results.

Finish playing than the number (QueryCountunderPercThreshold of the query word of threshold value lower than multimedia resource, be called for short QCUPT), namely multimedia resource is watched the number of very few query word, that is multimedia resource finishes playing and to finish playing than the number of the query word of threshold value than lower than multimedia resource.This index reflects in search engine the query word scale comprising " ill content ", and namely Search Results does not have the situation of degree of depth correlativity and attractive force.Wherein, Resourse Distribute can be considered in conjunction with practical business multimedia resource is set flexibly finishes playing and compare threshold value.Such as, multimedia resource can be finished playing for the first time than threshold value be set to the clicked multimedia resource of each query word finishing playing than distribution first quartile.

Such as, by carrying out simple statistics analysis to user's degree of depth dwell data of the full dose query word shown in above-mentioned table 1, original evaluation index as shown in table 2 below can be obtained.

The original evaluation index of certain search engine of table 2

Known by above-mentioned table 2: (1) is 2, is 450519 lower than the number QCUCT of the query word of this multimedia resource number threshold value and the number of independent query word as above is 775734 due to multimedia resource number threshold value, therefore, exceed in independent query word over half (namely ) the number of clicked multimedia resource of query word all below 2.This illustrates that user often searches for a query word, and its multimedia resource clicked is very few.This reflects that the Search Results of search engine is done badly in preliminary correlativity and attractive force, and Search Results also may be weak in hit degree or diversity.For the concrete analysis of this problem, need to distinguish the classification of query word, such as, query word is divided into navigation type query word, info class query word and interactive class query word.Different classes of query word, its click behavior is different.

(2) clicked multimedia resource to finish playing than population mean AVP be 32.98%, and due to multimedia resource finish playing than threshold value be 7.49%, finishing playing lower than multimedia resource is 194020 than the number QCUPT of the query word of threshold value and the number of independent query word as above is 775734, therefore, in independent query word close to 1/4th (namely ) the viewing time length of Search Results of multimedia resource of query word be no more than 7.49%.This illustrates that the quality of the Search Results of search engine can not make user satisfied.Whether the analysis needs rejecting for this problem exists a large amount of fict search viewing behaviors (it should be noted that, if the reproduction time length of the multimedia resource under same query word is too short, then this search should not be real search behavior).

The method for evaluating quality of the search engine of the embodiment of the present invention, according to user's degree of depth dwell data of full dose query word and original evaluation index, original assessment is carried out to the quality of search engine, can by directly carrying out total evaluation to the actual mass of the search engine of multimedia resource rapidly to monitoring every day of this original evaluation index, thus can without the need to manually marking, objectively in time the quality of search engine to be assessed.

embodiment 2

In above-described embodiment one, according to user's degree of depth dwell data of full dose query word and original evaluation index, original assessment is carried out to the quality of search engine, but, this original assessment may not utilize clicked multimedia resource and clicked multimedia resource finish playing than integrated information, that is, this original assessment may not provide the degree of integration that user's degree of depth stops.Like this, make such as when a lot of and each multimedia resource of the number of the clicked multimedia resource of a query word finish playing than very low, and for example when the clicked multimedia resource of a query word number seldom and finishing playing than very high of each multimedia resource, utilize original assessment may be difficult to compare the stop degree of user on the Search Results of which query word higher.Further, consider that user and the mutual interface of search engine are the query words at every turn inputted, therefore, be necessary to utilize the quality of this integrated information to search engine to assess (comprehensive assessment).Based on this, the invention provides the embodiment two that the following quality to search engine carries out comprehensive assessment.

Fig. 2 illustrates the process flow diagram of the method for evaluating quality of the search engine according to the embodiment of the present invention two.The step that in Fig. 2, label is identical with Fig. 1 has identical function, for simplicity's sake, omits the detailed description to these steps.

As shown in Figure 2, the key distinction of the method for evaluating quality of the search engine shown in the method for evaluating quality of the search engine shown in Fig. 2 and Fig. 1 is, except comprising step S100, step S120 and the step S140 in above-described embodiment one, can also comprise:

Step S220, user's degree of depth dwell data according to single query word, the user's degree of depth calculating single query word stops index.

Particularly, clicked multimedia resource and clicked multimedia resource finish playing than integrated information such as can comprise the number VidCount of independent clicked multimedia resource, the number of times ClickCount (same multimedia resource may be repeatedly clicked) of clicked multimedia resource, the finishing playing than mean value AveragePerc of multimedia resource.Therefore, if higher, each multimedia resource of number of times of more, the clicked multimedia resources of the number of the clicked multimedia resource of the independence of certain query word finish playing than larger, then this query word user's degree of depth stop degree higher.

In a kind of possible implementation, sigmoid function can be used to represent that user's degree of depth stops index D eepLinger, namely, described user's degree of depth dwell data according to described single query word, the user's degree of depth calculating described single query word stops index, comprising: adopt formula according to user's degree of depth dwell data of described single query word (formula 1), the user's degree of depth calculating described single query word stops index, wherein, y is that user's degree of depth of described single query word stops index, x=VidCount*ClickCount*AveragePerc (formula 2), VidCount is the number of independent clicked multimedia resource, and ClickCount is the number of times of clicked multimedia resource, and AveragePerc is that finishing playing of multimedia resource compares mean value.

Such as, calculate user's degree of depth by using above-mentioned sigmoid function to user's degree of depth dwell data of single query word and stop index D eepLinger, user's degree of depth that can obtain each query word as described in Table 3 stops index.

User's degree of depth of each query word of table 3 stops index

query	VidCount	ClickCount	AveragePerc	DeepLinger
					Red rice note tears machine open	2	4	0.1164	0.4347
Tan lay Buddhist's the heart channel of Hang-Shaoyin	17	17	0.0005	0.0704
					Guo De guiding principle I to pass through	4	6	0.6927	1.0000

In a kind of possible implementation, overall Max-Min normalized function can be used to represent that user's degree of depth stops index D eepLinger, namely, described user's degree of depth dwell data according to described single query word, the user's degree of depth calculating described single query word stops index, comprise: adopt formula y=VidCountN*ClickCountN*AveragePercN (formula 3) according to user's degree of depth dwell data of described single query word, the user's degree of depth calculating described single query word stops index, wherein, y is that user's degree of depth of described single query word stops index,

$VidCountN=\frac{VidCount-\min \left(VidCount\right)}{\max \left(VidCount\right)-\min \left(VidCount\right)}$ (formula 4),

$ClickCountN=\frac{ClickCount-\min \left(ClickCount\right)}{\max \left(ClickCount\right)-\min \left(ClickCount\right)}$ (formula 5),

$AveragePercN=\frac{AveragePerc-\min \left(AveragePerc\right)}{\max \left(AveragePerc\right)-\min \left(AveragePerc\right)}$ (formula 6),

VidCount is the number of independent clicked multimedia resource, ClickCount is the number of times of clicked multimedia resource, AveragePerc be multimedia resource finish playing than mean value, min () gets minimum value, and max () gets maximal value.

In a kind of possible implementation, linear averaging based on the number of clicks of multimedia resource can be used finishing playing than summation, represent that user's degree of depth stops index D eepLinger, namely, described user's degree of depth dwell data according to described single query word, the user's degree of depth calculating described single query word stops index, comprising: adopt formula according to user's degree of depth dwell data of described single query word (formula 7), the user's degree of depth calculating described single query word stops index, wherein, y is that user's degree of depth of described single query word stops index, VidCount is the number of independent clicked multimedia resource, AllVidCount is the summation of all numbers of clicks of the multimedia resource utilizing described single query word to search, and AveragePerc is that finishing playing of multimedia resource compares mean value.

Step S240, stop exponential sum comprehensive assessment index according to user's degree of depth, comprehensive assessment is carried out to the quality of search engine.

Particularly, stop index based on the above-mentioned user's degree of depth calculated, following comprehensive assessment index can be used to carry out comprehensive assessment to the quality of search engine:

User's degree of depth stops exponential average (AverageDeepLingerIndex, be called for short ADLI), namely user's degree of depth stops the mean value of index, that is the averaged residence degree of user on each query word, this index reflects from individuality the quality that search engine is supplied to the Search Results of user.

Number (the QueryCountunderDeepLingerThreshold of the query word of index threshold is stopped lower than user's degree of depth, be called for short QCUDLT), namely user's degree of depth stops index stops the query word of index threshold number lower than user's degree of depth, that is user's degree of depth stops the number of the too low query word of degree, this index reflects in search engine the query word scale returning " pathology results ", and namely Search Results does not have the situation of comprehensive correlativity and attractive force.Wherein, can in conjunction with practical business and consider Resourse Distribute arrange flexibly the degree of depth stop index threshold.Such as, the degree of depth can be stopped the first quartile that index threshold is set to the degree of depth stop exponential distribution of each query word for the first time.

That is, above-mentioned comprehensive assessment index can comprise user's degree of depth stop exponential average and lower than user's degree of depth stop index threshold query word number at least one.

Such as, index and comprehensive assessment index as described in Table 4 can be stopped by above-mentioned user's degree of depth, comprehensive assessment is carried out to the quality of search engine.

The comprehensive assessment index of certain search engine of table 4

Index	Actual value
		User's degree of depth stops exponential average ADLI	0.395
The Query stopping index threshold lower than the degree of depth measures QCUDLT	191218 (threshold value is 0.062)

By above-mentioned table 3 and table 4 known: user's degree of depth that user's degree of depth of (1) query word " Tan lay Buddhist's the heart channel of Hang-Shaoyin " degree that stops stops degree and query word " Guo De guiding principle I will pass through " far away from user's degree of depth of query word " red rice note tears machine open " stops degree, and this illustrates that the Search Results of query word " Tan lay Buddhist's the heart channel of Hang-Shaoyin " may search behavior that is very poor or this query word may be non-actual search behavior.(2) user's degree of depth stops exponential average ADLI is 0.395, and this illustrates that the Search Results general performance of search engine may be similar with the performance of query word " red rice note tears machine open ".(3) such as, if it is 0.062 that user's degree of depth stops index threshold, then (that is, the number stopping the query word of index threshold lower than user's degree of depth is about 1/4th of the number of full dose query word ).Stop index threshold according to user's degree of depth, the performance of the Search Results of such query word may be similar to the performance of query word " Tan lay Buddhist's the heart channel of Hang-Shaoyin ".

That is, above-mentioned comprehensive assessment depends on the score of the judge that user is undertaken by viewing behavior, the result of this comprehensive assessment to be user to the content quality of hit, sequence, diversity and multimedia resource carry out Comprehensive Evaluation.Certainly, above-mentioned two comprehensive assessment index can also be utilized promptly to assess the total satisfactory grade of user for the search engine of multimedia resource.

It should be noted that, (namely the present embodiment carries out comprehensive assessment again first to carry out original assessment, in step S100, S120, step S220 is performed again after S140, S240) for example is illustrated, but, those skilled in the art should be able to understand, the present invention is not limited thereto, such as, can intersect and carry out original assessment and comprehensive assessment, and for example, when in order to assess the quality of search engine more quickly, only can carry out original assessment, for another example, when the accuracy of the quality evaluation in order to improve search engine, only can carry out comprehensive assessment.

The method for evaluating quality of the search engine of the embodiment of the present invention, according to user's degree of depth dwell data of full dose query word and original evaluation index, original assessment is carried out to the quality of search engine, and according to user's degree of depth stop exponential sum comprehensive assessment index, comprehensive assessment is carried out to the quality of search engine, thus can not only without the need to manually marking, objectively in time the quality of search engine to be assessed, but also can by the good and bad degree of the Search Results of the search engine under user's degree of depth stop index directly more any two query words, thus the accuracy of the quality evaluation of search engine can be improved.

embodiment 3

Fig. 3 is the structured flowchart of the quality assessment device of search engine according to the embodiment of the present invention three.The quality assessment device 300 of the search engine that the present embodiment provides is for realizing the method for evaluating quality of the search engine provided embodiment illustrated in fig. 1.As shown in Figure 3, the quality assessment device 300 of this search engine can comprise:

Acquiring unit 320, for obtaining user's degree of depth dwell data of single query word from user journal.In the present invention, the degree of depth of user stops behavior and can comprise: the stop of (1) user on the result of page searching of search engine, and namely user clicks the behavior of the Search Results of the multimedia resource of multiple such as video, audio frequency etc.; And the stop of (2) user on the broadcasting page of search engine, namely user watches the behavior of the multimedia resource of such as video, audio frequency etc.

Particularly, four-tuple { query, vids, percs, δ } can be used to portray the behavior that stops of user's degree of depth of each query word.In other words, user's degree of depth dwell data of single query word can be obtained from user journal according to the data model of single query word.This process can comprise carries out pre-service and noise removal process to user journal data, and the noise of user journal data may from the many-side of such as illegally input, system exception, recording exceptional etc.

Wherein, query is query word, and namely user inputs in the search each time of search engine, such as, can obtain the query word query of user from the user journal of search engine.

Vids is for clicking multimedia resource set, namely user clicks the set of multimedia resource at result of page searching by search query word, such as, click multimedia resource set vids can be obtained by the source limiting multimedia resource viewing from the multimedia resource viewing daily record of user journal.

Percs is that multimedia resource finishes playing than set, namely clicked multimedia resource finish playing than set, such as, multimedia resource can be obtained finish playing than set percs from the multimedia resource of user journal viewing daily record by carrying out secondary treating to multimedia resource played data.It should be noted that, because the T.T. length of each multimedia resource may differ larger, therefore, multimedia resource is used to finish playing more objective than merely using the reproduction time length of multimedia resource to portray the behavior that stops of user's degree of depth than portraying user's degree of depth stop behavior.Such as, for same query word, if clicked multimedia resource is played repeatedly, then this clicked multimedia resource finish playing than being an integrate score, such as, can get this query word all finish playing than mean value, and for example, can get this query word all finish playing than median etc.

δ be clicked multimedia resource set at the most media resource plays complete than set mapping function, such as, can obtain multimedia resource finish playing than set time pre-define this mapping function.

That is, user's degree of depth dwell data of above-mentioned single query word can comprise: query word (query), clicked multimedia resource set (vids), multimedia resource finish playing and to finish playing than the mapping function (δ) gathered to multimedia resource than set (percs) and clicked multimedia resource set.

Obtain unit 340, be connected with acquiring unit 320, for the user's degree of depth dwell data according to single query word, obtain user's degree of depth dwell data of full dose query word.

Such as, by gathering polymerization to user's degree of depth dwell data of the single query word got, user's degree of depth dwell data of full dose query word can be obtained.Such as, this process can comprise and carries out secondary treating (obtain count field data) and denoising etc. to data.

Particularly, { query, vid, count, the perc} quality to search engine is portrayed can to use four-tuple.In other words, user's degree of depth dwell data (that is, user's degree of depth dwell data of full dose query word) of whole search engine can comprise these four fields of query, vid, count and perc.Wherein, query is full dose query word; Vid is the clicked multimedia resource under current query; Count is the clicked number of times of the clicked multimedia resource under current query; Perc is the ratio that comprehensively finishes playing of the clicked multimedia resource under current query.

That is, user's degree of depth dwell data of above-mentioned full dose query word can comprise: the ratio that comprehensively finishes playing of the clicked multimedia resource under the clicked number of times of the clicked multimedia resource under full dose query word, current queries word, the clicked multimedia resource under current queries word and current queries word.

Concrete example can see the associated description of step S120 in above-described embodiment one.

Original assessment unit 360, is connected with acquisition unit 340, for according to user's degree of depth dwell data of full dose query word and original evaluation index, carries out original assessment to the quality of search engine.

After the user's degree of depth dwell data obtaining full dose query word, above-mentioned original evaluation index can be obtained by carrying out simple statistical study to user's degree of depth dwell data of obtained full dose query word, original assessment utilizes the statistical property of the original value of the user's degree of depth dwell data obtained to carry out original assessment to the quality of the search engine of multimedia resource, wherein, the original evaluation index carrying out original assessment for the quality of the search engine to multimedia resource can comprise:

The number (IndependentClickedVideoCount is called for short ICVC) of independent clicked multimedia resource, the number of the independent multimedia resource namely clicked by all query words.This index reflects the degree of the searched derivation of backstage multimedia resource on the whole.

Mean number (the AverageClickedVideoCount of the clicked multimedia resource of each query word, be called for short ACVC), namely each query word on average can be clicked derives how many multimedia resources, that is the mean value of the number of the clicked multimedia resource of each query word.This index reflects the degree of the searched derivation of backstage multimedia resource from individuality.

Lower than the number (QueryCountunderCountThreshold, be called for short QCUCT) of the query word of multimedia resource number threshold value, namely the number of clicked multimedia resource is lower than the number of the query word of multimedia resource number threshold value.This index reflects by the query word scale of " morbid state presents " in search engine, and namely Search Results does not have the situation of preliminary correlativity and attractive force.Wherein, Resourse Distribute can be considered in conjunction with practical business multimedia resource number threshold value is set flexibly.Such as, multimedia resource number threshold value can be set to the first quartile of the number distribution of the clicked multimedia resource of each query word for the first time.

Multimedia resource finishes playing than population mean (AverageVideoPerc, be called for short AVP), namely user's time span of multimedia resource of watching on multimedia resource result of page searching at this by the mean value of the number percent of the T.T. of the multimedia resource watched in length.This index reflects the good and bad degree of the content quality of search-engine results.

Finish playing than the number (QueryCountunderPercThreshold of the query word of threshold value lower than multimedia resource, be called for short QCUPT), namely multimedia resource is watched the number of very few query word, that is multimedia resource finishes playing and to finish playing than the number of the query word of threshold value than lower than multimedia resource.This index reflects in search engine the query word scale comprising " ill content ", and namely Search Results does not have the situation of degree of depth correlativity and attractive force.Wherein, Resourse Distribute can be considered in conjunction with practical business multimedia resource is set flexibly finishes playing and compare threshold value.Such as, multimedia resource can be finished playing for the first time than threshold value be set to the clicked multimedia resource of each query word finishing playing than distribution first quartile.

Concrete example can see the associated description of step S140 in above-described embodiment one.

The quality assessment device of the search engine of the embodiment of the present invention, original assessment unit carries out original assessment according to the user's degree of depth dwell data and original evaluation index that obtain the full dose query word that unit obtains to the quality of search engine, can by directly carrying out total evaluation to the actual mass of the search engine of multimedia resource rapidly to monitoring every day of this original evaluation index, thus can without the need to manually marking, objectively in time the quality of search engine to be assessed.

embodiment 4

Fig. 4 is the structured flowchart of the quality assessment device of search engine according to the embodiment of the present invention four.The quality assessment device 400 of the search engine that the present embodiment provides is for realizing the method for evaluating quality of the search engine provided embodiment illustrated in fig. 2.Wherein, assembly identical with Fig. 3 label in Fig. 4, comprising: acquiring unit 320, acquisition unit 340 and original assessment unit 360, have and aforementioned substantially identical function, for simplicity's sake, omit the detailed description to these assemblies.

In addition, by comparison diagram 3 and Fig. 4 known, the key distinction embodiment illustrated in fig. 4 with embodiment illustrated in fig. 3 is, on the basis of the embodiment shown in Fig. 3, the quality assessment device 400 of this search engine can also comprise:

Computing unit 420, is connected with acquiring unit 320, and for the user's degree of depth dwell data according to single query word, the user's degree of depth calculating single query word stops index.

Particularly, clicked multimedia resource and clicked multimedia resource finish playing than integrated information such as can comprise the number VidCount of independent clicked multimedia resource, the number of times ClickCount (same multimedia resource may be repeatedly clicked) of clicked multimedia resource, the finishing playing than mean value AveragePerc of multimedia resource.Therefore, if higher, each multimedia resource of number of times of more, the clicked multimedia resources of the number of the clicked multimedia resource of the independence of certain query word finish playing than larger, then this query word user's degree of depth stop degree higher.

In a kind of possible implementation, sigmoid function can be used to represent that user's degree of depth stops index D eepLinger, namely, described user's degree of depth dwell data according to described single query word, the user's degree of depth calculating described single query word stops index, comprising: adopt formula according to user's degree of depth dwell data of described single query word (formula 1), the user's degree of depth calculating described single query word stops index, wherein, y is that user's degree of depth of described single query word stops index, x=VidCount*ClickCount*AveragePerc (formula 2), VidCount is the number of independent clicked multimedia resource, and ClickCount is the number of times of clicked multimedia resource, and AveragePerc is that finishing playing of multimedia resource compares mean value.

Concrete example can see the associated description of step S220 in above-described embodiment two.

In a kind of possible implementation, overall Max-Min normalized function can be used to represent that user's degree of depth stops index D eepLinger, namely, described user's degree of depth dwell data according to described single query word, the user's degree of depth calculating described single query word stops index, comprise: adopt formula y=VidCountN*ClickCountN*AveragePercN (formula 3) according to user's degree of depth dwell data of described single query word, the user's degree of depth calculating described single query word stops index, wherein, y is that user's degree of depth of described single query word stops index,

$VidCountN=\frac{VidCount-\min \left(VidCount\right)}{\max \left(VidCount\right)-\min \left(VidCount\right)}$ (formula 4),

$ClickCountN=\frac{ClickCount-\min \left(ClickCount\right)}{\max \left(ClickCount\right)-\min \left(ClickCount\right)}$ (formula 5),

$AveragePercN=\frac{AveragePerc-\min \left(AveragePerc\right)}{\max \left(AveragePerc\right)-\min \left(AveragePerc\right)}$ (formula 6),

VidCount is the number of independent clicked multimedia resource, ClickCount is the number of times of clicked multimedia resource, AveragePerc be multimedia resource finish playing than mean value, min () gets minimum value, and max () gets maximal value.

In a kind of possible implementation, linear averaging based on the number of clicks of multimedia resource can be used finishing playing than summation, represent that user's degree of depth stops index D eepLinger, namely, described user's degree of depth dwell data according to described single query word, the user's degree of depth calculating described single query word stops index, comprising: adopt formula according to user's degree of depth dwell data of described single query word (formula 7), the user's degree of depth calculating described single query word stops index, wherein, y is that user's degree of depth of described single query word stops index, VidCount is the number of independent clicked multimedia resource, AllVidCount is the summation of all numbers of clicks of the multimedia resource utilizing described single query word to search, and AveragePerc is that finishing playing of multimedia resource compares mean value.

Comprehensive assessment unit 440, is connected with computing unit 420, for stopping exponential sum comprehensive assessment index according to user's degree of depth, carries out comprehensive assessment to the quality of search engine.

Particularly, stop index based on the above-mentioned user's degree of depth calculated, following comprehensive assessment index can be used to carry out comprehensive assessment to the quality of search engine:

User's degree of depth stops exponential average (AverageDeepLingerIndex, be called for short ADLI), namely user's degree of depth stops the mean value of index, that is the averaged residence degree of user on each query word, this index reflects from individuality the quality that search engine is supplied to the Search Results of user.

Number (the QueryCountunderDeepLingerThreshold of the query word of index threshold is stopped lower than user's degree of depth, be called for short QCUDLT), namely user's degree of depth stops index stops the query word of index threshold number lower than user's degree of depth, that is user's degree of depth stops the number of the too low query word of degree, this index reflects in search engine the query word scale returning " pathology results ", and namely Search Results does not have the situation of comprehensive correlativity and attractive force.Wherein, can in conjunction with practical business and consider Resourse Distribute arrange flexibly the degree of depth stop index threshold.Such as, the degree of depth can be stopped the first quartile that index threshold is set to the degree of depth stop exponential distribution of each query word for the first time.

That is, above-mentioned comprehensive assessment index can comprise user's degree of depth stop exponential average and lower than user's degree of depth stop index threshold query word number at least one.

Concrete example can see the associated description of step S240 in above-described embodiment two.

It should be noted that, the present embodiment carries out comprehensive assessment by comprehensive assessment unit be again illustrated first to carry out original assessment by original assessment unit, but, those skilled in the art should be able to understand, the present invention is not limited thereto, such as, can be intersected by original assessment unit and comprehensive assessment unit and carry out original assessment and comprehensive assessment, and for example, when in order to assess the quality of search engine more quickly, only can carry out original assessment by original assessment unit, for another example, when the accuracy of the quality evaluation in order to improve search engine, only can carry out comprehensive assessment by comprehensive assessment unit.

The quality assessment device of the search engine of the embodiment of the present invention, original assessment unit carries out original assessment according to the user's degree of depth dwell data and original evaluation index that obtain the full dose query word that unit obtains to the quality of search engine, and comprehensive assessment unit stops exponential sum comprehensive assessment index according to user's degree of depth that computing unit calculates carries out comprehensive assessment to the quality of search engine, thus can not only without the need to manually marking, objectively in time the quality of search engine to be assessed, but also can by the good and bad degree of the Search Results of the search engine under user's degree of depth stop index directly more any two query words, thus the accuracy of the quality evaluation of search engine can be improved.

The above; be only the specific embodiment of the present invention, but protection scope of the present invention is not limited thereto, is anyly familiar with those skilled in the art in the technical scope that the present invention discloses; change can be expected easily or replace, all should be encompassed within protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection domain of described claim.

Claims (10)

1. a method for evaluating quality for search engine, described search engine is used for searching multimedia resource, and it is characterized in that, described method for evaluating quality comprises:

User's degree of depth dwell data of single query word is obtained from user journal, wherein, user's degree of depth dwell data of described single query word comprises: query word, clicked multimedia resource set, multimedia resource finish playing and finish playing than the mapping function of set than set and described clicked multimedia resource set to described multimedia resource;

According to user's degree of depth dwell data of described single query word, obtain user's degree of depth dwell data of full dose query word, wherein, user's degree of depth dwell data of described full dose query word comprises: the ratio that comprehensively finishes playing of the clicked multimedia resource under the clicked number of times of the clicked multimedia resource under full dose query word, current queries word, the clicked multimedia resource under current queries word and current queries word; And

According to user's degree of depth dwell data and the original evaluation index of described full dose query word, original assessment is carried out to the quality of described search engine,

Wherein, described original evaluation index comprise the independent number of clicked multimedia resource, the clicked multimedia resource of each query word mean number, finish playing than population mean lower than the number of the query word of multimedia resource number threshold value, multimedia resource, finish playing than at least one in the number of the query word of threshold value lower than multimedia resource.

2. method for evaluating quality according to claim 1, is characterized in that, also comprises:

According to user's degree of depth dwell data of described single query word, the user's degree of depth calculating described single query word stops index; And

Stop exponential sum comprehensive assessment index according to described user's degree of depth, comprehensive assessment carried out to the quality of described search engine,

Wherein, described comprehensive assessment index comprise user's degree of depth stop exponential average and lower than user's degree of depth stop index threshold query word number at least one.

3. method for evaluating quality according to claim 2, is characterized in that, described user's degree of depth dwell data according to described single query word, and the user's degree of depth calculating described single query word stops index, comprising:

Adopt formula according to user's degree of depth dwell data of described single query word the user's degree of depth calculating described single query word stops index,

Wherein, y is that user's degree of depth of described single query word stops index,

X=VidCount*ClickCount*AveragePerc, VidCount are the numbers of independent clicked multimedia resource, and ClickCount is the number of times of clicked multimedia resource, and AveragePerc is that finishing playing of multimedia resource compares mean value.

4. method for evaluating quality according to claim 2, is characterized in that, described user's degree of depth dwell data according to described single query word, and the user's degree of depth calculating described single query word stops index, comprising:

Adopt formula y=VidCountN*ClickCountN*AveragePercN according to user's degree of depth dwell data of described single query word, the user's degree of depth calculating described single query word stops index,

Wherein, y is that user's degree of depth of described single query word stops index,

$VidCountN=\frac{VidCount-\min \left(VidCount\right)}{\max \left(VidCount\right)-\min \left(VidCount\right)},$

$ClickCountN=\frac{ClickCount-\min \left(ClickCount\right)}{\max \left(ClickCount\right)-\min \left(ClickCount\right)},$

$AveragePercN=\frac{AveragePerc-\min \left(AveragePerc\right)}{\max \left(AveragePerc\right)-\min \left(AveragePerc\right)},$

VidCount is the number of independent clicked multimedia resource, ClickCount is the number of times of clicked multimedia resource, AveragePerc be multimedia resource finish playing than mean value, min () gets minimum value, and max () gets maximal value.

5. method for evaluating quality according to claim 2, is characterized in that, described user's degree of depth dwell data according to described single query word, and the user's degree of depth calculating described single query word stops index, comprising:

Adopt formula according to user's degree of depth dwell data of described single query word the user's degree of depth calculating described single query word stops index,

Wherein, y is that user's degree of depth of described single query word stops index,

VidCount is the number of independent clicked multimedia resource, and AllVidCount is the summation of all numbers of clicks of the multimedia resource utilizing described single query word to search, and AveragePerc is that finishing playing of multimedia resource compares mean value.

6. a quality assessment device for search engine, described search engine is used for searching multimedia resource, and it is characterized in that, described quality assessment device comprises:

Acquiring unit, for obtaining user's degree of depth dwell data of single query word from user journal, wherein, user's degree of depth dwell data of described single query word comprises: query word, clicked multimedia resource set, multimedia resource finish playing and finish playing than the mapping function of set than set and described clicked multimedia resource set to described multimedia resource;

Obtain unit, be connected with described acquiring unit, for the user's degree of depth dwell data according to described single query word, obtain user's degree of depth dwell data of full dose query word, wherein, user's degree of depth dwell data of described full dose query word comprises: the ratio that comprehensively finishes playing of the clicked multimedia resource under the clicked number of times of the clicked multimedia resource under full dose query word, current queries word, the clicked multimedia resource under current queries word and current queries word; And

Original assessment unit, is connected with described acquisition unit, for according to user's degree of depth dwell data of described full dose query word and original evaluation index, carries out original assessment to the quality of described search engine,

Wherein, described original evaluation index comprise the independent number of clicked multimedia resource, the clicked multimedia resource of each query word mean number, finish playing than population mean lower than the number of the query word of multimedia resource number threshold value, multimedia resource, finish playing than at least one in the number of the query word of threshold value lower than multimedia resource.

7. quality assessment device according to claim 6, is characterized in that, also comprises:

Computing unit, is connected with described acquiring unit, and for the user's degree of depth dwell data according to described single query word, the user's degree of depth calculating described single query word stops index; And

Comprehensive assessment unit, is connected with described computing unit, for stopping exponential sum comprehensive assessment index according to described user's degree of depth, carries out comprehensive assessment to the quality of described search engine,

Wherein, described comprehensive assessment index comprise user's degree of depth stop exponential average and lower than user's degree of depth stop index threshold query word number at least one.

8. quality assessment device according to claim 7, is characterized in that, described computing unit specifically for, adopt formula according to user's degree of depth dwell data of described single query word the user's degree of depth calculating described single query word stops index,

Wherein, y is that user's degree of depth of described single query word stops index,

X=VidCount*ClickCount*AveragePerc, VidCount are the numbers of independent clicked multimedia resource, and ClickCount is the number of times of clicked multimedia resource, and AveragePerc is that finishing playing of multimedia resource compares mean value.

9. quality assessment device according to claim 7, it is characterized in that, described computing unit specifically for, adopt formula y=VidCountN*ClickCountN*AveragePercN according to user's degree of depth dwell data of described single query word, the user's degree of depth calculating described single query word stops index

Wherein, y is that user's degree of depth of described single query word stops index,

$VidCountN=\frac{VidCount-\min \left(VidCount\right)}{\max \left(VidCount\right)-\min \left(VidCount\right)},$

$ClickCountN=\frac{ClickCount-\min \left(ClickCount\right)}{\max \left(ClickCount\right)-\min \left(ClickCount\right)},$

$AveragePercN=\frac{AveragePerc-\min \left(AveragePerc\right)}{\max \left(AveragePerc\right)-\min \left(AveragePerc\right)},$

VidCount is the number of independent clicked multimedia resource, ClickCount is the number of times of clicked multimedia resource, AveragePerc be multimedia resource finish playing than mean value, min () gets minimum value, and max () gets maximal value.

10. quality assessment device according to claim 7, is characterized in that, described computing unit specifically for, adopt formula according to user's degree of depth dwell data of described single query word the user's degree of depth calculating described single query word stops index,

Wherein, y is that user's degree of depth of described single query word stops index,

VidCount is the number of independent clicked multimedia resource, and AllVidCount is the summation of all numbers of clicks of the multimedia resource utilizing described single query word to search, and AveragePerc is that finishing playing of multimedia resource compares mean value.