JP2011059947A - Device, method and program for generating ranking function - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve generation of a ranking function using two or more out of a plurality of case data different in property. <P>SOLUTION: The respective domain case DBs 010, 020 store case data different in property. A case DB generation functional part 030 integrates the case data of both DBs 010, 020 to retain the same in a case DB 040. An order relationship DB generation functional part 050 imparts the order relationship to cases of the case DB 040 and stores the cases in an order relationship DB 060. A weak ranking function generation functional part 070 generates a ranking function for ranking the order relationship stored in the order relationship DB 060. A case select functional part 110 sequentially updates the stored data of the order relationship DB 060, and stores each ranking function generated based on the updated order relationship in a ranking function storage DB. A weak ranking function integrate functional part 100 integrates the stored data of the ranking function storage DB to generate a final ranking function. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for generating a ranking function for calculating a score for ranking search results in a web search system.

  In recent years, with the spread of the Internet, the importance of a web search system that accurately retrieves information required by a user from an enormous web document group (electronic document group) on the Internet has increased. In this web search system, a score for ranking search results is calculated. The method of Non-Patent Document 1.2 is used to generate the ranking function for calculating the score.

  Describing the outline based on FIG. 6, the ranking function generating device 1 receives the stored data of the target domain case DB 010 as an input, the order relation DB generation function unit 050, the order relation DB 060, the weak ranking function generation function part 070, and the case selection. The ranking function 120 is output through the function unit 110, the weak ranking function storage DB 090, and the weak ranking function integration function unit 100. At this time, the target domain case DB 010 stores case data for data construction of the order relation DB 060 necessary for generating the ranking function based on the machine learning algorithm used in Non-Patent Document 1.2 or the like. Yes.

Yoav Freund, Raj Iyer, Robert E. Shapire, Yoram Singer. “An Effective Boosting Algorithm for Combining Preferences”. Journal of Machine Learning Research, Vol. 4, pp. 933≡969, 2003. Thorsten Joachims, “Optimizing Search Engines Using Clickthrough Data”, Proceedings of the ACM Conference on Knowledge Discovery and Data in CM2

  According to the prior art, as an input to the ranking function generating device 1, based on manually evaluated data or a click-through log as in Non-Patent Document 2, that is, a log in which a search system stores a user's click history for a search query. Case data generated in the past have been used.

  However, the supervised machine learning algorithm as described in Non-Patent Document 1.2 has a possibility that the accuracy of the ranking function may be lowered when each case data having different properties is used at the same time. there were.

  The present invention has been made to solve the above-described problems, and a problem to be solved is to generate a ranking function using a plurality of case data having different properties as described above. .

  Therefore, the present invention realizes highly accurate ranking function generation by performing learning while selecting a case close to the target domain case data from different case data for a plurality of case data having different properties. .

  One aspect of the present invention is an apparatus for generating a ranking function for ranking search results obtained by searching an electronic document group based on a search keyword, and integrating case data having different properties stored in a plurality of domain case databases Case data integrating means, order relation generating means for assigning order relations between cases to the case data integrated by the case integrating means and storing them in an order relation database, and order relations saved in the order relation database Ranking function generating means for generating a ranking function for ranking, and the stored data of the order relation database are sequentially updated, and each ranking function generated by the ranking function generation means based on the updated order relation is stored in the function storage database. The example selection means to be stored in the database and the function storage database Integrate data comprises a ranking function integration means for generating a final ranking function, a.

  Another aspect of the present invention is a method for generating a ranking function for ranking a search result obtained by searching an electronic document group based on a search keyword, wherein case data integration means is stored in a plurality of domain case databases. A first step of integrating case data having different properties, and a second step in which the order relation generating means assigns an order relation between the cases for the case data integrated in the first step and stores the order relation in the order relation database. The ranking function generating means generates a ranking function for ranking the order relations stored in the order relation database; and the case selection means sequentially updates the order relations stored in the order relation database. Each ranking function generated by repeating the third step is stored in the function storage database. A fourth step of, ranking function integration means integrates the stored data of said function storage database, and a fifth step of generating the final ranking function, a.

  In addition, this invention can be comprised in the program which functions a computer as said apparatus. You may provide this program with the aspect recorded on the recording medium.

  According to the present invention, it is possible to generate a ranking function using a plurality of case data having different properties, for example, training data obtained from manual evaluation data or click-through logs.

The block diagram of the ranking function production | generation apparatus which concerns on embodiment of this invention. The flowchart figure which shows the same whole process. The flowchart figure which shows the process of the example DB production | generation function part. The flowchart figure which shows the process of the same order relation Db production | generation function part. The flowchart figure which shows the process of the case selection function part. The block diagram of the ranking function production | generation apparatus of a prior art example.

  Hereinafter, a ranking function generation device according to an embodiment of the present invention will be described. This ranking function generating device preferably uses evaluation data such as manual evaluation data conventionally used for ranking function generation as target domain case data and stores it in the target domain case DB. On the other hand, data such as a click-through log that can be used for ranking function generation is stored in another domain case DB as another domain case data, although the property is different from the target domain case data.

  Here, different domains indicate that the nature of the data is different. At this time, since the correct answer to which the score that is finally desired to be achieved is used as the target domain data, it is appropriate to use human evaluation data as the target domain data.

≪Example of device configuration≫
A configuration example of the ranking function generation device will be described with reference to FIG. Here, the ranking function generation device 2 constitutes a Web search system (so-called search engine) that searches an electronic document group in response to a search instruction from a user terminal (not shown).

  Specifically, the ranking function generation device 2 includes hardware resources of a normal computer, such as a CPU, a memory (RAM), a hard disk drive device, and a communication interface.

  As a result of the cooperation between the hardware resource and the software resource, the ranking function generation device 2 is configured such that the target domain case DB 010, the separate domain case DB 020, the case DB generation function unit 030, the case DB 040, and the order relation DB generation function unit. 050, an order relation DB 060, a weak ranking function generation function unit 070, a case selection function unit 110, a weak ranking function storage DB 090, and a weak ranking function integration function unit 100. Here, it differs from the prior art in that it has another domain case DB 020 and a case DB generation function unit 030.

  Of these, each DB010.020.0.00.00.00.090 is assumed to be built on the hard disk drive device. Each domain case DB 010.020 does not need to be built in the ranking function generation device 2 and may be constructed as an external database capable of transmitting and receiving data. Table 1.2 shows an example of the data structure stored in each domain case DB 010.020.

  Table 1 shows a data configuration example of the target domain case DB 010, in which case data used for generating a ranking function is stored. Here, in the query “qid 1”, a document with a score of 5 indicates information positioned higher than a document with a score of 4. Features 1 to K indicate the value of each feature when searched by the query “qid 1”. For example, this characteristic includes a value depending on a query such as word frequency and a static value such as importance of a page. The importance of the page is calculated using link information of the web page.

  Table 2 shows a data configuration example of the separate domain case DB 020. Here, the separate domain case DB 20 is a database that holds case data having properties different from those of the target domain case DB 010. Basically, the scores for queries and documents have the same format as the data held by the target domain case DB 10. Has been granted. As another domain example, for example, a click-through log (user click history) and the like can be cited.

  Further, the outline of each functional unit 030.050.070.100.110 will be described. The case DB creation functional unit 030 is configured such that the target domain, another domain is based on the data input from each domain case DB 010.020. Is stored in the case DB 40. Further, the order relation DB generation function unit 050 sets a weight initial value for each order relation in order to generate a ranking function with reference to the case DB 040. This weighted order relation data is stored in the order relation DB 060.

  The weak ranking function generation function unit 070 refers to the order relation DB 060 and based on the weighted order relation data, the ranking function (hereinafter referred to as a weak ranking function in order to distinguish it from the finally generated ranking function). Is generated.

  The case selection function unit 110 receives the weak ranking function generated by the weak ranking function generation function unit 070, the storage data of the order relation DB 60, and the storage data of the case DB 40 as inputs, and stores them in the order relation DB 060. Update the weight of the stored order relation. Generation of the weak ranking function is tried many times based on the updated weighted order relation data.

  At this time, in order to update the weight of the order relation, unlike the conventional technique, a useful order relation is selected from the different domain data by distinguishing between the target domain stored in the case DB 040 and the different domain. That is, the target domain data is updated by increasing the weight of the order relation that is erroneously ranked by the generated weak ranking function.

  Thereby, in the next trial, a weak ranking function that works effectively for the case where ranking was not successful in the previous trial can be generated. Also, by updating the different domain data by reducing the weight of the ranking that is erroneously ranked, it is possible to eliminate the influence of the order relationship that causes noise in the target domain learning in the next trial. Information on the weak ranking function generated as a result of each trial as described above is stored in the weak ranking function DB 090 through the case selection function unit 110.

  The weak ranking function integration function unit 100 integrates weak ranking functions generated as a result of each trial with reference to the weak ranking function DB 090, and generates and outputs one final ranking function 120. The output ranking function 120 is used for score calculation for ranking search results in a Web search system (so-called search engine).

  FIG. 2 shows the overall processing steps (S001 to S009) of the ranking function generator 2. Here, when the process is started, the storage data of the case DB 040 is created through the case DB generation function unit 030 using the storage data of the target domain case DB 010 and the separate domain case DB 020 as input (S001). Further, the storage data of the order relation DB 060 is created through the order relation DB creation function unit 050 using the storage data of the case DB 040 as an input (S002).

  Next, the weak ranking function generation function unit 070 tries to generate a weak ranking function until a preset total number N of weak ranking function generation trials is reached (S003 to S005). Here, the weak ranking function is generated with the number of trials I = 0 as an initial value, and the processes of S006 to S008 are repeated as long as “the number of trials I <the total number of trials N” (S004.S005). The total number of trials N may be set in a program or the like as a threshold value.

  That is, the weak ranking function is generated by the weak ranking function generation function unit 070 using the stored data of the order relation DB 060 and the case DB 040 as input (S006). Also, the generated weak ranking function and the stored data of the order relation DB 060 and the case DB 040 are input, and the case selection function unit 110 updates the stored data of the order relation DB 060 and the weak ranking function storage DB 090 (S007). .

  Then, “I + 1” is substituted for the number of trials “I” (S 008), and if “trial number I = total number of trials N” is established, the process proceeds to S 009 (S 005). In S009, the stored data of the weak ranking function storage DB 090 is input, the ranking function 120 is generated by the weak ranking function integration function unit 100, and the process ends. Hereinafter, specific processing contents of the respective functional units 030.050.070.110.100 will be described.

≪Case DB generation function unit 030≫
The case DB generation function unit 030 combines input data from each domain case DB 010.020 and stores the combined data in the case DB 040. At this time, serial number case ID and domain information (whether it is a target domain or another domain) are added. As a result, the case DB 040 that holds information such as the target domain (target) and another domain (different) based on the input data is constructed. The contents of this processing will be described based on the flowchart of FIG.

  S101 to S103: First, when the process is started, “1” is substituted for the number “i” of the case ID stored in the case DB 40 (S101), and the target domain case DB 010 is referred to in the input data. An unprocessed record is acquired (S102). The record obtained by adding the case ID number “i” and the domain (target) column to the head of the acquired record is added to the case DB 40 (S103).

  S104. S105: “i + 1” is substituted into the case ID number “i” (S104), and it is confirmed whether or not an unprocessed record exists in the target domain DB 010 (S105). As a result of the confirmation, if there is an unprocessed record, the process returns to S102 and repeats the process of S103.

  S106. S107: Next, an unprocessed record is acquired from the separate domain case DB 020 (S106). Here, a record in which the column of the case ID “i” and the domain “different” is added to the top of the acquired record is added to the case DB 040 (S107).

  S108. S109: “i + 1” is substituted for the case ID number “i” (S108), and it is confirmed whether or not an unprocessed record exists in the separate domain case DB 020 (S109). As a result of the confirmation, if there is an unprocessed record in the separate domain case DB 020, the process returns to S106 and repeats the process of S107, while if there is no unprocessed record, the process is terminated.

  Table 3 shows an example of records stored in the case DB 040 as a result of the processing of S101 to S109. Here, in the case DB 040, information on domain (target / different), query (qid), score, and features 1 to K is stored for each case ID.

<< Order relation DB generation function unit 050 >>
The order relation DB creation function unit 050 receives the stored data of the case DB 040 as an input and generates the order relation DB 060. Here, for records of the same query (qid) and the same domain, cases having different scores are stored in the order relation DB 060 as one order relation.

  At this time, the order relation DB 060 holds information (weight) on the order relation between cases. The weight is used when the weak ranking function is generated using the weak ranking function generation function unit 070. Hereinafter, the processing of the creation function unit 050 will be described with reference to the flowchart of FIG. Here, the total amount of stored data (total number of records) in the case DB 040 is represented as “M”.

  S201. S202: When processing is started, first, “0” is substituted for “M” (S201), and a unique set of queries (qid) is acquired for each domain from the case DB 040 (S202). Let Q be the query set acquired here.

  S203: It is confirmed whether or not an unprocessed query (qid) exists in the set Q acquired in S202. If there is an unprocessed query (qid) as a result of the confirmation, the process proceeds to S204, and if not, the process proceeds to S211.

  S204. S205: The next unprocessed query q and domain d are extracted from the query set Q (S204). Next, records including the query q and the domain d are acquired from the case DB 040, sorted in the order of score, and set as “R” (S205).

  S206: It is checked whether or not there is an unprocessed record in the sort result R of S205. If there is an unprocessed record, the process proceeds to S207. If there is no unprocessed record, the process returns to S203 and the next query (qid) Perform the process.

  S207 to 209: The next unprocessed record including the query q and the domain d is acquired from the case DB 040 (S207). Let the acquired record be “r”. Here, among the records after “r” included in the sort result R, records having the same domain with a score lower than “r” are acquired (S208). Let the acquired record be “R ′”.

  For each “r ′” included in “R ′”, the “target / differential” domain, the “r” case ID as the upper case ID, and the “r ′” case ID as the lower ID are stored in the order relation DB 60. Output (S209).

  S210. S211: “M + 1” is substituted for “M” (S210), and the process returns to S206 to check whether there is an unprocessed record. At this time, if there is no unprocessed record in the sort result R, the presence or absence of an unprocessed query is confirmed in S203.

  If there is no unprocessed query in the set Q, the weights of all records are set to 1 / M (S211), and the process is terminated. The set weight is an initial value. Table 4 shows an example of records stored in the order relation DB 060 as a result of the processing of S201 to S211. For each domain (target / differential), the upper case ID, the lower case ID, and the weight (initial value) Information is stored.

≪Weak ranking function generation function unit 070≫
For the weak ranking function generation function unit 070, for example, the “WeakLearn” algorithm of Non-Patent Document 1 can be used. Here, the name is a weak ranking function generation function unit, but a normal ranking function generation device can be used.

  The “WeakLearn” algorithm of Non-Patent Document 1 is a ranking function that focuses on one feature and gives “1” if the value is equal to or greater than a threshold value, and “0” if the value is equal to or less than the threshold value. Therefore, the weak ranking function can be reproduced by holding three pieces of information, that is, superiority, feature ID, and threshold.

  Specifically, the weak ranking function generation function unit 070 generates a weak ranking function in consideration of the weight of the order relation stored in the order relation DB 060. At this time, the order relationship having a large weight is appropriately ranked by regarding the order relationship having a large weight as the error cost (dominance) having a large weight and the order relationship having a small weight being regarded as having a small error cost (dominance). To generate a weak ranking function.

<< Case Selection Function Unit 110 >>
Hereinafter, the processing content of the case selection function unit 110 will be described with reference to the flowchart of FIG. Here, the weight of the order relation stored in the order relation DB 060, that is, the weight at the time of weak ranking function generation is sequentially updated.

  S301: First, a superiority α that is a weighted correct answer rate with respect to a source of weak ranking function generated through the weak ranking function generation function unit 070, that is, the order relationship of the order relationship DB 060 is calculated. As a method of calculating α, for example, the method of Non-Patent Document 1 is used.

Here, among the records whose domain of the order relation DB 060 is “target”, the sum of the weights of the order relations erroneously ranked by the generated weak ranking function is normalized by the total weight sum as “{EMBED Equation.3, } ”, The degree of superiority α can be obtained using equation (1).

  S302. S302: “0” is substituted into the normalized total amount Z for normalization (S302), and then an unprocessed record is acquired from the order relation DB 060 (S303). The record acquired here is assumed to be “p”.

  S304. S305: Ranking the upper case and lower case of the record p acquired in S303 by the weak ranking function (S304). This ranking uses features 1 to K input from the case DB 040. At this time, if the record p is correctly ranked, the process proceeds to S307. If the record p is not correctly ranked, the process proceeds to S306 (S305).

  S306: The weight of the record p of the order relation in the order relation DB 060 is updated. The method of updating the weight is different between the target domain case (target) and another domain case (different). Here, if the weight in the previous weak ranking function generation trial is “Wt”, the weight can be updated by, for example, the update formula of Formula (2).

  At this time, the weight is lowered for another domain. For example, “α” in equation (2) is defined as equation (3). In this equation (3), “n” represents the number of records in the “different” domain of the order relation DB 060, and N represents the total number of trials set for weak ranking function generation.

  S307 to S309: The weight of “Z + p” is substituted into the normalized total amount Z (S307), and it is confirmed whether or not an unprocessed record exists in the order relation DB 60 (S308). As a result of the confirmation, if there is an unprocessed record, the process returns to S303 and the subsequent processing is resumed. On the other hand, if there is no unprocessed record, the process proceeds to S309. In S309, the weights of all the records in the order relation DB 060 are updated to values obtained by dividing by the normalized total amount Z. The weak ranking function DB 090 is constructed by performing the processing of S301 to S309 every time a weak ranking function is generated.

  Table 5 shows an example of the data structure stored in the weak ranking function DB 090. Here, a data configuration example is shown in which a weak ranking function is generated using the “WeakLearn” algorithm of Non-Patent Document 1. In this data configuration example, the superiority sequentially updated in S306, the threshold value when the weak ranking function is generated, and the feature ID used for ranking in S304 are stored.

≪Weak ranking function integration function unit 100≫
The weak ranking function integration function unit 100 integrates the weak ranking functions by a method disclosed in Non-Patent Document 1, and generates a final ranking function 120. Specifically, each weak ranking function (trial 1 to trial N in Table 5) stored in the weak ranking function DB 090 is weighted with the respective superiority and added to generate the ranking function 120.

  At this time, instead of all weak ranking functions of trials 1 to N, it is also possible to specify the number to be added, for example, by adding N / 2 to N weak ranking functions after half. Therefore, according to the ranking function generating device 2, a ranking function is obtained by using a plurality of case data having different properties that cannot be realized by the prior art, that is, a plurality of training data obtained from manual evaluation data, click-through logs, and the like. Can be generated.

  The generated ranking function 120 is used to calculate a score for ranking the search results obtained by searching the electronic document group based on the search keyword in the Web search system (so-called search engine). Are sorted according to the score and returned to the user terminal.

≪Programs≫
The present invention can also be configured as a ranking function generation program that causes a computer to function as a part or all of the components 010.012.030 to 070,090.100.110 of the ranking function generation device 2. According to this program, S001 to S009. S101 to S109. S201-S211. All or some of steps S301 to S309 can be executed by a computer.

  The program can be provided through a network such as a website or e-mail. The program is recorded on a recording medium such as a CD-ROM, DVD-ROM, CD-R, CD-RW, DVD-R, DVD-RW, MO, HDD, Blu-ray Disk (registered trademark). It is also possible to save and distribute. This recording medium is read using a recording medium driving device, and the program code itself realizes the processing of the above embodiment, so that the recording medium also constitutes the present invention.

2 ... Ranking function generator 010 ... Target domain case DB (domain case database)
012… Another domain case DB (domain case database)
030 ... Case DB generation function unit (case data integration means)
040 ... Case DB (Case Database)
050 ... Order relation DB generation function section (order relation generation means)
060 ... Order relation DB (order relation database)
070 ... Weak ranking function generation function section (ranking function generation means)
090 ... weak ranking function storage DB (function storage database)
100: Weak ranking function integration function (ranking function integration means)
110 ... Case selection function part (case selection means)
120 ... Ranking function

Claims (7)

An apparatus for generating a ranking function for ranking search results obtained by searching an electronic document group based on a search keyword,
Case data integration means for integrating case data with different properties stored in multiple domain case databases;
Order relation generating means for assigning an order relation between cases for the case data integrated by the case data integration means and storing it in an order relation database;
Ranking function generating means for generating a ranking function for ranking the order relation stored in the order relation database;
Case selection means for sequentially updating the storage data of the order relation database, and storing each ranking function generated by the ranking function generation means based on the updated order relation in a function storage database;
Ranking function integration means for integrating the stored data of the function storage database and generating a final ranking function;
A ranking function generating device comprising: Case data stored in each domain case database has a score of each electronic document and a feature value at the time of query search for each query,
The case data integration means adds a domain type for each case to the case data stored in each domain case database and saves it in the case database.
The order relation generation means stores the weight information of the order relation between cases having a difference in scores for the case data of the same query and the same domain stored in the case database in the order relation database for each domain,
The ranking function generating means generates a ranking function according to weight information stored in the order relation database,
The ranking function generation device according to claim 1, wherein the case selection unit distinguishes an update method of weight information stored in the order relation database according to a domain. The case selection means updates the weight information for a specific domain out of the data stored in the order relation database,
3. The ranking function generating apparatus according to claim 2, wherein the weighting information is reduced and updated for other domains. A method for generating a ranking function for ranking a search result obtained by searching an electronic document group based on a search keyword,
A first step in which case data integration means integrates case data having different properties stored in a plurality of domain case databases;
A second step in which the order relation generating means assigns order relations to the case data integrated in the first step and stores them in the order relation database;
A ranking function generating means for generating a ranking function for ranking the order relation stored in the order relation database;
A fourth step in which the case selection means sequentially updates the order relation stored in the order relation database, and stores each ranking function generated by repeating the third step in the function storage database;
A ranking function integration means for integrating the stored data of the function storage database and generating a final ranking function;
A ranking function generation method characterized by comprising: The first step adds a domain for each case to the case data of each domain case database containing the score of each electronic document and the feature value at the time of query search for each query, and saves it in the case database. ,
In the second step, for the records of the same query and the same domain stored in the domain case database, the weight information of the order relationship between cases having a score difference is stored in the order relationship database for each domain.
The third step generates a ranking function according to the weight information stored in the order relation database,
The ranking function generation device according to claim 4, wherein the fourth step distinguishes update methods of weight information stored in the order relation database according to domains. In the fourth step, the weight information is increased and updated for a specific domain in the stored data of the order relation database,
The ranking function generation method according to claim 5, wherein the weight information is updated while being reduced with respect to another domain.   The ranking function generation program for functioning a computer as each means of the ranking function generation apparatus of any one of Claims 1-3.

Images