KR101448166B1 - Method and server apparatus for evaluating search results - Google Patents

Abstract

A method and server for evaluating search results are provided. A method for evaluating a search result including a plurality of contents arranged in a grid form, which is a search result of a search term evaluation server for a specific search term, comprises the steps of: (a) A step of calculating an average residence time of the mouse pointer for each of the contents for each user on the basis of the residence information, (b) calculating the average residence time of the mouse pointer (C) calculating a quality evaluation score for the search result based on the average evaluation score of the entire user and the residence probability value for the fitness of each content and the specific search word; and .

Description

METHOD AND SERVER APPARATUS FOR EVALUATING SEARCH RESULTS

The present invention relates to a method and server for evaluating search results, and more particularly to a method and server for evaluating search results exposed in a grid form.

The evaluation of the search results exposed by entering a specific search term in the search window of the web page is determined as a good search result as the result more suitable to the user's intention is displayed at the upper part, The evaluation is generally performed using the Normalized Discount Cumulative Gain (nDCG) and the Expected Reciprocal Rank (ERR) method.

However, the nDCG and ERR methods are based on the assumption that the search results of the top-down exposure type are moved from the top to the bottom based on the eye tracking of the user, and the search results are stored in a plurality of grid- In the case of contents, there is a problem that the corresponding quality score can not be utilized.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems of the prior art, and it is an object of the present invention to provide an efficient quality evaluation score for a search result exposed in a grid form.

In order to achieve the above object, a search result evaluation server according to an embodiment of the present invention evaluates the search result including a plurality of contents arranged in a grid form as a search result for a specific search word (A) calculating an average residence time of a mouse pointer for each content for each user on the basis of the stay information of the mouse pointer of the user terminal for each content during a specific time, (b) Calculating a residence probability value of a mouse pointer for each content of the entire user on the basis of a residence time of the user; and (c) And calculating a quality score of the search result based on the search result.

In one aspect of the present invention, the stay information includes a position value of a mouse pointer for each of the contents, a stay time, and a number of click events of the mouse, wherein the mouse pointer stays for a predetermined time or longer It is determined that the mouse pointer has stayed in each of the contents only when the mouse click event has occurred.

In one aspect of the present invention, the step (b) calculates the residence probability value based on a sum of average residence times for each of the contents and an average residence time for each of the contents.

According to an aspect of the present invention, the step (c) includes the steps of: applying the average evaluation score to a DCG (Discount Cumulative Gain) method, and then calculating a first quality score and an Ideal Discount Cumulative Gain (IDCG) The quality evaluation score is calculated by a normalized discount cumulative gain (nDCG) method based on the second quality score obtained by multiplying the average score by the residence probability value.

According to an aspect of the present invention, there is provided a server for evaluating a search result including a plurality of contents arranged in a grid, which is a search result for a specific search term, according to an embodiment of the present invention, A residence time calculating unit for calculating an average residence time of the mouse pointer for each user on the basis of the residence information of the mouse pointer of the user terminal for each content of the content, A retention probability calculation unit for calculating a retention probability value of a mouse pointer for each content of the entire user; and a retention probability calculation unit for calculating a retention probability of each content based on the average evaluation score of the whole user and the retention probability value, And a quality score calculation unit for calculating a quality score for the search result.

In one aspect of the present invention, the stay information includes a position value of a mouse pointer for each of the contents, a stay time, and a number of click events of the mouse, and the average stay time calculating unit calculates the average stay time, It is determined that the mouse pointer has stayed in each of the contents only when the mouse click event has occurred.

In one aspect of the present invention, the residence probability calculating unit calculates the residence probability value based on a sum of average residence times for each of the contents and an average residence time for each of the contents.

According to an aspect of the present invention, the quality score calculating unit may calculate a first quality score and an Ideal Discount Cumulative Gain (IDCG) obtained by multiplying the average score by a DCG (Discount Cumulative Gain) The quality evaluation score is calculated by a normalized discount cumulative gain (nDCG) method based on a second quality score obtained by multiplying the average score by the stochastic value.

According to an embodiment of the present invention, a realistic quality score for a search result exposed in a grid form can be presented.

In addition, it can be used for various purposes such as building an learning rank (LTR) learning set.

It should be understood that the effects of the present invention are not limited to the above effects and include all effects that can be deduced from the detailed description of the present invention or the configuration of the invention described in the claims.

1 is a diagram showing a configuration of a search result evaluation system 100 according to an embodiment of the present invention.
2 is a block diagram showing a configuration of a search result evaluation server 120 according to an embodiment of the present invention.
FIG. 3 is a table showing average evaluation scores of users for search results and probability of staying of mouse pointer for each image according to an exemplary embodiment of the present invention. Referring to FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "indirectly connected" . Also, when an element is referred to as "comprising ", it means that it can include other elements, not excluding other elements unless specifically stated otherwise.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing a configuration of a search result evaluation system 100 according to an embodiment of the present invention.

The search result evaluation system 100 according to an embodiment of the present invention may include a plurality of user terminals 110 and a search result evaluation server 120. [

Each of the user terminals 110 may access the search result evaluation server 120 and receive a website from the search result evaluation server 120 and display the website on a screen.

1 illustrates a case where a search result evaluation server 120 provides a web site, a web server (not shown) for providing a web site and a search result evaluation server 120 for evaluating a search result, May be present separately.

Then, the user terminal 110 transmits a specific search term entered in the search window of the corresponding web site to the search result evaluation server 120, and transmits the search result from the search result evaluation server 120 to the web Page (hereinafter referred to as " search result ") and display it on the screen.

Here, the 'search result' may include a plurality of contents arranged in a grid form, and the contents may include at least one of text, image, flash, and moving picture.

And, the 'grid shape' may include a specific matrix form of n X m, such as 2 rows and 3 columns.

Also, the user terminal 110 may obtain the stay information of the mouse pointer (hereinafter, referred to as " stay information ") for a specific time with respect to the search result displayed on the screen and transmit the acquired information to the search result evaluation server 120.

Here, the 'stay information' may include a location value of the mouse pointer for each content, a stay time, and whether or not the mouse is clicked (counted). In addition, one or more of the identifier of the user terminal 110 and the identifier of the user As shown in FIG.

For reference, the user terminal 110 may receive and install a separate application from the search result evaluation server 120 in order to acquire the stay information. Through the operation of the application, the user terminal 110 may receive the movement of the mouse pointer And the event by the mouse can be detected to obtain the stay information.

In addition, a specific script for acquiring the stay information may be inserted into the web page including the search result received from the search result evaluation server 120. In this case, So that the stay information can be obtained by the operation.

In addition, the search result evaluation server 120 may detect the movement of the pointer of the mouse connected to the user terminal 110. In this case, the search result evaluation server 120 can acquire the stay information.

Hereinafter, the case where the user terminal 110 obtains the stay information will be described.

The user terminal 110 calculates the average residence time of the mouse pointer for each content based on the residence information obtained by the method and transmits the residence information including the calculated average residence time to the search result evaluation server 120 .

At this time, the user terminal 110 may determine that the mouse pointer has stayed in each content only when the mouse pointer has stayed on each content for a specific time or more and a mouse click event has occurred.

In the present invention, the fact that the mouse pointer stays in the content may mean that the user is gazing the corresponding content

Therefore, the 'residence time of the mouse pointer on the content' can be understood as the time when the user takes the content. The 'probability of staying of the mouse pointer on the content' It can be understood as a probability value for striking the corresponding content among the contents.

As described above, the user terminal 110 may calculate the average residence time for each content and transmit the result to the search result evaluation server 120. Alternatively, the user terminal 110 may calculate the average residence time for each content, And the event information (number of events) to the search result evaluation server 120 so that the search result evaluation server 120 calculates the average residence time for each content.

Meanwhile, when the user terminal 110 is connected, the search result evaluation server 120 provides a website to the user terminal 110 and transmits a search result for a specific search term input from the user terminal 110 And transmit it to the user terminal 110.

Here, the 'search result for a specific search word' may include a plurality of contents arranged in a grid form, and the 'grid form' may include a specific matrix form of n X m, such as 2 rows and 3 columns.

The search result evaluation server 120 receives the stay information from the user terminal 110 and stores the stay information in a DB (not shown) for each user, or stores the stay information of each user in a database (DB) Not shown).

Also, the search result evaluation server 120 calculates an average residence time (hereinafter, referred to as 'average residence time') of mouse pointers for each user on the basis of the residence information of each user stored in the DB (not shown) Can be calculated.

In addition, the search result evaluation server 120 may calculate a residence probability value (hereinafter referred to as residence probability value) of the mouse pointer for each content of the entire user based on the average residence time calculated for each user .

Here, the residence probability value for the specific content can be calculated based on the sum of the average residence time for each content and the average residence time for the specific content.

That is, the probability that a user actually sees a content exposed as a search result for a specific search term can be calculated based on a time when the mouse pointer stays in each content.

Also, the search result evaluation server 120 can calculate the quality evaluation score for the search result based on the average evaluation score and the residence probability value of all users for each content.

The quality score calculation method for the search result will be described later with reference to FIG.

2 is a block diagram showing a configuration of a search result evaluation server 120 according to an embodiment of the present invention.

The search result evaluation server 120 according to an embodiment of the present invention includes a stay time calculation unit 121, a stay information storage unit 122, a stay probability calculation unit 123, and a quality score calculation unit 124 can do.

The residence time calculator 121 may calculate an average residence time for each user based on the residence information received from the user terminal 110. [

Here, the 'stay information' may include a position value of the mouse pointer for each content, a stay time, and whether or not the mouse is clicked (counted). The average stay time calculating unit 121 calculates It is possible to determine that the mouse pointer has stayed on the content only when the mouse pointer has stayed for more than a predetermined time and a mouse click event has occurred.

Meanwhile, the stay information storage unit 122 may store the stay information received from the user terminal 110 in a DB (not shown) for each specific search term. At this time, have.

In addition, the residence information storage unit 122 may further store the average residence time for each content calculated by the residence time calculating unit 121 in a DB (not shown) for each specific search word and for each user.

On the other hand, the residence probability calculation unit 123 can calculate the residence probability value for each content of the entire user based on the average residence time of each user stored in the DB (not shown).

Here, the 'residence probability value' can be calculated based on the sum of the average residence time for each content and the average residence time for each content.

On the other hand, the quality score calculation unit 124 can calculate the quality score of the search result based on the average score and the stay probability value of all users for each content.

At this time, the quality score calculation unit 124 calculates the average score of the IDCG (Ideal Discount Cumulative Gain) method and the first quality score obtained by multiplying the stay probability value after applying the average score to the DCG (Discount Cumulative Gain) The quality evaluation score of the search result can be calculated by the nDCG (Normalized Discount Cumulative Gain) method based on the second quality score obtained by multiplying the staying probability value after the application.

A detailed description thereof will be described later with reference to Fig.

FIG. 3 is a table showing average evaluation scores of users for search results and probability of staying of mouse pointer for each image according to an exemplary embodiment of the present invention. Referring to FIG.

5, five images (A, B, C, D, and E) are searched as a search result for a specific search word, and the average evaluation score of all users for each image is maximum 4 points and minimum 0 points (Where 4 is very fit, 3 is fit, 2 is moderate, 1 is inadequate, 0 means very inadequate).

The average score of the total users for the five images A, B, C, D and E is 2, 3, 4, 0, 1 and the probability of retention of the mouse pointer is 0.6, 0.4, 0.2, And they are shown in the table of FIG. 3 when they are matched.

According to the value of the table shown in FIG. 3, the quality score calculation unit 124 applies a mean cumulative score of all users to each image to a DCG (Discount Cumulative Gain) method, and then calculates a first quality evaluation The score can be calculated.

That is, the first quality score =? (The value obtained by applying the average evaluation score to the DCG method and the residence probability value of the mouse pointer). The equation of the DCG method is as follows.

p = particular rank position, reli = graded relevance of the result at position i

For reference, the log value may be omitted in the above equation. The reason for applying the log value in the DCG method is that it does not take into account the disadvantage of the Cumulative Gain (CG) method, that is, the position of the search result (determined from down to down) It is difficult to impose differentiation on the basis of the facts.

However, in the case where the search result is a plurality of contents arranged in a grid form, the search results are not arranged in descending order from top to bottom, and the patterns in which users look at the contents presented in the search result are not constant. We applied the users' probability of residence to search results.

That is, the present invention applies a probability that users will gaze at each content, rather than applying an exposure rank to a plurality of contents arranged in a grid form, so that the log value may be omitted in the above equation.

Hereinafter, the case where the log value is omitted in calculating the first quality evaluation score and the second quality evaluation score to be described later will be described.

In the DCG method, the first quality score, in which the log value is omitted,

A first quality score ^{= (2 2 -1) X 0.6} + (2 3 -1) X 0.4 + (2 4 -1) X 0.2 + (2 0 -1) X 0.15 + (2 1 -1) X 0.1 = 7.7.

In addition, the quality score calculation unit 124 applies an average score of all users for each image to an IDCG (Ideal Discount Cumulative Gain) method in which each image exposed as a search result is ideally placed, The second quality score can be calculated.

The ideal search results for the five images A, B, C, D, and E are placed in order from highest to lowest in average scores of all users, and from highest to lowest so that the probability of retention of the mouse pointer also matches will be.

That is, when the average scores of all users for the five images C, B, A, E and D are 4, 3, 2, 1 and 0, C, and B, A, E, and D may be disposed at positions corresponding to retention probability values of 0.4, 0.2, 0.15, and 0.1, respectively.

In this case, the second quality score (log value omitted) calculated through the IDCG method is a value

Second quality score = (2 ⁴ -1) X 0.6 + (2 ³ -1) X 0.4 + (2 ² -1) X 0.2 + (2 ¹ -1) X 0.15 + (2 ⁰ -1) X 0.1 = 12.55.

Thereafter, the quality score calculating unit 124 may calculate the quality score of the search result by the nDCG (Normalized Discount Cumulative Gain) method based on the first quality score and the second score have.

Since the equation of the nDCG method is as follows,

The quality score for the search results is 7.7 / 12.55 = 0.61354.

As described above, the search result evaluation method using the conventional nDCG method is based on the assumption that search results are arranged from top to bottom on the basis of eye tracking of users, but a search result including a plurality of contents in a grid form If shown, the eye tracking pattern of the users is not constant and can only be random.

Therefore, in consideration of this situation, the present invention multiplies the calculated probabilities based on the evaluation scores of the search results and the stay information of the mouse pointer to calculate the quality score, And can provide an efficient quality score for the product.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be.

It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included within the scope of the present invention.

100: Search result evaluation system
110: User terminal
120: Search result evaluation server
121: Retention time calculating section
122: stay information storage unit
123: Retention probability calculating section
124: Quality score calculation unit

Claims (8)

A method for evaluating a search result including a plurality of contents arranged in a grid form, which is a search result for a specific search term,
(a) calculating an average residence time of the mouse pointer for each of the contents for each user on the basis of the stay information of the mouse pointer of the user terminal for each content for a specific time;
(b) calculating a residence probability value of a mouse pointer for each of the contents of the entire user based on an average residence time of the mouse pointer for each of the contents calculated for each user; And
(c) calculating a quality score of the search result based on the average evaluation score of the entire user for the fitness of the content and the specific search word and the probability of staying of the mouse pointer for each content;
Wherein the search result evaluation method comprises:
The method according to claim 1,
Wherein the stay information includes a position value of a mouse pointer for each content, a stay time, and a number of click events of the mouse,
The step (a)
Wherein the determination unit determines that the mouse pointer has stayed in each of the contents only when the mouse pointer stays for a predetermined time or longer and the mouse click event occurs.
The method according to claim 1,
The step (b)
And calculating a probability of staying of the mouse pointer for each of the contents based on the sum of the average residence times of the mouse pointers for the respective contents and the average residence time of the mouse pointer for each of the contents.
The method according to claim 1,
The step (c)
Applying the average evaluation score to the DCG (Discount Cumulative Gain) method, applying the average evaluation score to a first quality score and an ideal discount cumulative gain (IDCG) method in which the stagnation probability value is multiplied and then multiplying the stagnation probability value by a multiplication And calculating the quality score based on a second quality score using a normalized discount cumulative gain (nDCG) method.
A server apparatus for evaluating a search result including a plurality of contents arranged in a grid form, which is a search result for a specific search term,
A residence time calculating unit for calculating an average residence time of the mouse pointer for each of the contents on the basis of the residence information of the mouse pointer of the user terminal for each of the contents for a specific time;
A residence probability calculation unit for calculating a residence probability value of a mouse pointer for each of the contents of all users based on an average residence time of the mouse pointer for each of the contents calculated for each user; And
A quality score calculating unit for calculating a score of a quality score for the search result based on the average score of the total user for the fitness of each content and the specific search word and the probability of staying of the mouse pointer for each content, ;
And a search result evaluation server device.
6. The method of claim 5,
Wherein the stay information includes a position value of a mouse pointer for each content, a stay time, and a number of click events of the mouse,
The average residence time calculating unit
Wherein the determination unit determines that the mouse pointer has stayed in each of the contents only when the mouse pointer stays for a predetermined time or more and a mouse click event occurs.
6. The method of claim 5,
The residence probability calculation unit
And calculates a residence probability value of the mouse pointer for each of the contents based on the sum of the average residence times of the mouse pointers for the respective contents and the average residence time of the mouse pointer for each of the contents.
6. The method of claim 5,
The quality score calculation unit
Applying the average evaluation score to the DCG (Discount Cumulative Gain) method, applying the average evaluation score to a first quality score and an ideal discount cumulative gain (IDCG) method in which the stagnation probability value is multiplied and then multiplying the stagnation probability value by a multiplication And calculates the quality score based on a second quality score using a normalized discount cumulative gain (nDCG) method.

Images