JP5700566B2 - Scoring model generation device, learning data generation device, search system, scoring model generation method, learning data generation method, search method and program thereof - Google Patents

Description

  The present invention relates to a learning data generation apparatus and method for generating learning data used when learning a scoring model in document search or voice document search based on statistical model learning, and learning based on the generated learning data. The present invention relates to a search apparatus and a method for searching a document by using a scoring model.

  Document search is a problem of selecting documents closely related to an input query (hereinafter also referred to as “input query”) from a finite number of documents given in advance. Typically, documents are arranged in order of relevance to the input query. At this time, it is necessary to calculate a score representing the relationship between the input query and each document, and the documents are arranged in the order of score. Therefore, the score calculation method is the technical core. Note that a query is a word string (in other words, a word string to be searched) designated by a user who performs a search, and may be a sentence, a sentence, a phrase, a word, a symbol, and a combination thereof. A document is usually a web page or text file containing one or more sentences or sentences, and is a target of document search.

  The voice document search is a document search whose search target is a voice document. An audio document is an audio file that records audio. This is realized by applying speech recognition to a voice document, converting it into text, and applying a document search technique. In some cases, the query is given by voice, and voice recognition is generally applied in the same manner. However, the calculation of the score representing the relation depth between the query and each voice document is performed in consideration of misrecognition of voice recognition and the presence of unknown words (words not registered in the voice recognition system). Hereinafter, a document and an audio document are also simply referred to as a document.

  Conventionally, in document search, heuristic methods (a method of solving problems through a process of trial and error, experiments, and examinations, heuristic methods) are used to calculate a score that represents the depth of association between the input query and each document. It was. However, as seen in many problems in the natural language processing field, a further improvement in accuracy can be expected by calculating a score using a model generated based on statistical model learning. A model used for calculating a score representing the depth of association between the input query and each document is called a scoring model. Non-Patent Document 1 is known as a conventional technique for generating a scoring model based on statistical model learning. The scoring model will be described below.

Let f _{q, d} be a feature vector extracted from a set of a query q and a document d. The feature vector is extracted based on a rule defined in advance. For example as elements (feature), a common word _{w i} to query q and document d, inverse document frequency (Inverse Document Frequency) number c _(w i, d) of the logarithm of the sum and words _{w i} in the document d idf The sum of logarithmic values of (w _i ) can be used. When the parameter vector of the scoring model is Φ, the score given to the set of the query q and the document d by this scoring model is expressed as S _Φ (f _{q, d} ). This score S _Φ (f _{q, d} ) represents the relation depth between the query q and the document d.

One of the widely used scoring models is a linear model. In the linear model, for example, the score is calculated by the following equation.
S _Φ (f _{q, d} ) = Φ ・ f _{q, d} (1)
Note that · is an inner product operator.

  The parameter vector Φ is obtained in advance by a statistical model learning method. Learning data is prepared, and the value of the parameter vector Φ can be obtained using an existing learning method. The learning data is generally a set of a set of a query and a reference label (a label indicating a document closely related to the query, and the number of documents closely related may be plural). The learning data is prepared by manually determining a document that is considered to be closely related to each query.

  In Non-Patent Document 2, a document search method using a language model is shown. It can be said that the model is statistically learned because it learns the parameters of the language model using statistical data, but because it does not use reference labels, it directly learns whether the document is related to a query. Absent. Therefore, in this specification, it is positioned as a heuristic method.

Ramesh Nallapati, “Discriminative Models for Information Retrieval”, Proceedings of ACM SIGIR, 2004, pp.64-71 Jay M. Ponte and W. Bruce Croft, “A Language Modeling Approach to Information Retrieval”, Proceedings of ACM SIGIR, 1998, pp. 275-281

  In general, in order to obtain an appropriate parameter estimation result, the larger the number of parameters (the higher the dimension of the parameter vector Φ), the more learning data (a set of query and reference label) is required. However, as described above, it is necessary to prepare learning data manually, and it is difficult to prepare a large amount. Therefore, the prior art parameter vector Φ is a low-dimensional vector.

For example, in the linear model of Equation (1), if the parameter vector Φ is 8 dimensions, the feature vector f _{q, d} is also 8 dimensions. This means that at most eight types of features should be used from the query q and the document d, and there is a high possibility that important features are dropped when performing a document search. In general, in the language processing field, high-dimensional models such as tens of thousands to tens of millions of dimensions are used, whereas in document search, scoring models based on extremely low-dimensional parameter vectors are used. This is because it is difficult to prepare a large amount of learning data manually.

  The advantages of generating a scoring model based on statistical model learning are that (1) the relationship between a query and a reference label is explicitly learned, and (2) a simple model such as a linear model is used. Various features can be easily introduced. If a scoring model based on a high-dimensional parameter vector becomes available, various features can be used, and the relationship between a query and a reference label can be learned more precisely. Therefore, if a scoring model based on a high-dimensional parameter vector can be used, even if it is less accurate than the existing method, at least features that are different from the existing method can be captured. In the calculation of, accuracy can be expected by calculating the score by superimposing the score obtained by the existing method and the score obtained by the scoring model based on the high-dimensional parameter vector.

An object of this invention is to provide the scoring model production | generation apparatus which learns a scoring model based on statistical model learning, and the learning data production | generation apparatus which produces | generates the learning data used in that case.

In order to solve the above problem, according to the first aspect of the present invention, the scoring model generation device learns a scoring model in document search based on statistical model learning. The scoring model generation device sets M as the number of documents, N as the number of queries generated from one document, m = 1, 2,..., M, n = 1, 2,. The learning data s _mn is received, the feature vector f _{q, d} is extracted from the query q included in the learning data s _mn and each document d _, and the inner product of the parameter vector Φ and the feature vector f _{q, d} is the document The parameter vector Φ is learned so that a positive value is taken when d is a related document of the query q, and a negative value is taken when the document d is not a related document of the query q.
In order to solve the above problem, according to another aspect of the present invention, a learning data generation device is provided with a plurality of documents, and learning used when learning a scoring model in document search based on statistical model learning Generate data. The learning data generation device includes a word string generation unit and a learning data generation unit. For each given document, the word string generation means generates one or more word strings including words included in the document. The learning data generation means uses the generated word string and the label indicating the document used when generating the word string as a query and a reference, respectively, and sets the query and reference as learning data.

  According to the present invention, there is an effect that learning data used when learning a scoring model based on statistical model learning can be automatically generated without depending on human hands. High-dimensional parameter vectors can be appropriately estimated using a large amount of learning data, and information that is difficult to handle with heuristic methods can be used. Therefore, a search device using the parameter vectors is more accurate. Search is possible.

1 is a configuration diagram of a search system 1 according to a first embodiment. The figure which shows the processing flow of the search system 1 which concerns on 1st embodiment. The functional block diagram of the learning data generation apparatus 11 which concerns on 1st embodiment. The figure which shows the processing flow of the learning data generation apparatus 11 which concerns on 1st embodiment. The figure which shows the result of the simulation 1 of the search device 13 of 1st embodiment. The figure which shows the result of the simulation 2 of the search device 13 of 1st embodiment.

  Hereinafter, embodiments of the present invention will be described. In the drawings used for the following description, constituent parts having the same function and steps for performing the same process are denoted by the same reference numerals, and redundant description is omitted.

<First embodiment>
FIG. 1 shows a configuration example of the search system 1, and FIG. 2 shows a processing flow thereof. The search system 1 includes a learning data generation device 11, a scoring model generation device 12, and a search device 13.

The learning data generation device 11 receives M documents d ₁ , d ₂ ,..., D _M as input, generates M × N learning data s _mn (s 11), and outputs it to the scoring model generation device 12. However, m = 1, 2,..., M, n = 1, 2,..., N, and N is the number of word strings (queries) generated from one document. Details will be described later.

The scoring model generation device 12 learns and generates a parameter vector Φ used in the scoring model using the M × N pieces of learning data s _mn (s12), and outputs it to the search device 13.

The search device 13 sets the parameter vector Φ as a scoring model. When receiving the search query q _u from the terminal 2 operated by the user, the search device 13 searches for a document corresponding to the search query q _u (s13). The document search is performed as follows, for example. Using the scoring model, the score of each document for the search query q _u is calculated, and document information D _{u in} which a part of the document, title, URL, etc. are arranged in descending order of score is generated and transmitted to the terminal 2. . The user can browse a document closely related to the search query q _u by accessing a URL or the like included in the document information D _u .

<Learning data generation device 11>
FIG. 3 is a functional block diagram of the learning data generation apparatus 11, and FIG. 4 shows its processing flow. The learning data generation device 11 includes a storage unit 111, an individual language model generation unit 112, a comprehensive language model generation unit 113, a word string generation unit 114, and a learning data generation unit 115.

(Storage unit 111)
The storage unit 111 stores the given M documents d ₁ , d ₂ ,..., D _M. In addition, data during processing and various parameters are stored. Each means reads / writes predetermined data and parameters to / from the storage unit 111 in each process. However, each means does not necessarily have to read / write each data with respect to the memory | storage part 111, You may control so that data may be passed directly between each part. The storage unit 111 corresponds to an auxiliary storage device, a RAM (Random Access Memory), a register, another buffer memory, a cache memory, or the like, or a storage area using these in combination.

(Comprehensive language model generation means 113)
The comprehensive language model generation unit 113 extracts _M documents d ₁ , d ₂ ,..., D _M from the storage unit 111, generates a probabilistic language model B for all the documents (s 1101), and stores it in the storage unit 111. To do. Examples of the probabilistic language model include an n-gram language model, a back-off n-gram language model, a hidden Markov model, and a maximum entropy model.

(Individual language model generation means 112)
Individual language model generating unit 112 retrieves the document _{d m} from the storage unit 111, generates a probabilistic language model _{L m} for the document _{d m} (S1103), and stores in the storage unit 111. The individual language model generation means 112 generates probabilistic language models L ₁ , L ₂ ,..., L _M for _M documents d ₁ , d ₂ ,..., D _M , respectively (s1102, s1109, s1110).

(Word string generation means 114)
Word sequence generation unit 114 takes out the two probabilistic language model B and _{L m} from the storage unit 111, it generates a word sequence _{q mn} based on two stochastic language model B and _{L m} (S1105), the storage unit 111 To store. In the present embodiment, a sentence q _mn composed of a word string is generated.

For example, a probability P (W) of a word W given by a linear combination of two probabilistic language models B and L _m is obtained by the following formula, and a sentence q _mn is randomly generated according to the probability distribution.
P (W) = λP _Lm (W) + (1-λ) P _B (W) (2)
Where P _Lm (W) is the probability of the word W given by the probabilistic language model L _m , P _B (W) is the probability of the word W given by the probabilistic language model B, and λ is 0 <λ It is a weighting coefficient consisting of real values of ≦ 1. In the learning of the language model, it is assumed that the end-of-sentence symbol exists in each document. When a word string is randomly generated, the output of the sentence end symbol is regarded as one word string (query). Incidentally generates N statement _{q mn} for one document _{d m (s1104, s1107, s1108} ).

(Learning data generation means 115)
Learning data generating means 115 takes out a label m pointing to document d _m from the storage unit 111, and a word string q _mn to the word train generating means 114 is generated from the document d _m, a label m as a reference word string q _mn is used as a query, and the set is set as learning data s _mn = (m, q _mn ) (s 1106) and stored in the storage unit 111. This processing is performed for all sentences q _mn (s1104, s1107, s1108).

The learning data generation device 11 generates M × N learning data s _mn in this way, and transmits the learning data s _mn to the scoring model generation device 12.

<Effect>
With such a configuration, learning data used when learning a scoring model based on statistical model learning can be automatically generated without relying on humans.

  Hereinafter, an outline of a scoring model generation method using the learning data generated by the learning data generation device 11 and a document search method using the generated scoring model will be described.

<Search device 13>
The search device 13 performs a document search using a scoring model learned using M × N pieces of learning data s _mn obtained by the learning data generation device 11. In the present embodiment, in consideration of the fact that the automatically generated learning data and the true learning data are greatly different, the scoring model learned using the learning data s _mn is used as a conventional search method in order to ensure the search accuracy. Introducing as a re-scoring against. A method for learning the scoring model will be described later.

When the relevance between the document d and the query q given by the baseline search system is expressed as D (f _q , f _d ), in this embodiment, the relevance is obtained using the parameter vector Φ and the feature vector f _{q, d} of the linear model. D (f _q , f _d ) is corrected (rescored) by the following equation.
D (f _q , f _d ) + αΦ ・ f _{q, d} (3)
Here, α is a constant for adjusting both scales.

For example, a technique based on the distance between feature vectors (see Reference 1) can be used as the degree of association D (f _q , f _d ) of the baseline search system.
(Reference 1) Yu Uno, Hitoshi Ito, Akinori Ito, Shozo Makino, “Index improvement using WWW for speech document retrieval”, Proceedings of the 4th Speech Document Processing Workshop, 2010

A feature vector extracted from the query q is denoted by f _q , and a feature vector extracted from the document d is denoted by f _d . This feature vector extraction is executed based on a predefined rule. For example, a vector composed of tf-idf (term frequency and inverse document frequency) such as a word can be used as the feature vector. A score representing the relation depth between the query q and the document d is calculated from the distance D (f _q , f _d ) between both vectors. For example, a cosine distance can be used for the distance D (f _q , f _d ). In other words, tf-idf (x) is a unigram tf-idf vector of word string x, cosine (y, z) is a cosine distance between y and z, and a degree of association D (f _q , f _d ) is calculated by the following equation: To do.
D (f _q , f _d ) = cosine (tf-idf (d), tf-idf (q))

The parameter vector Φ and the feature vector f _{q, d} have the same number of dimensions as the total number of word types when a vector related to unigram frequency is used. At this time, the feature vector f _{q, d} uses the appearance frequency of each word in the document d as an element value. However, the value of an element corresponding to a word that does not appear in the query q is set to 0.

  Since it is difficult to prepare a large amount of learning data in the prior art, the high-dimensional parameter vector Φ cannot be estimated appropriately. Therefore, the parameter vector Φ has a low dimension, and the feature vector has to have a low dimension. In this embodiment, since a large amount of learning data can be prepared in advance, a high-dimensional parameter vector Φ can be estimated appropriately, and a high-dimensional feature vector having the same number of dimensions as the total number of word types can be obtained. Can be used. This means that many features can be used from the query q and the document d, which means that important features can be used without missing a document search.

  The search device 13 calculates a score for each document by the equation (3), and determines the rank as a document high-order candidate in descending order of the value.

<Scoring model generator 12>
The scoring model generation device 12 receives M × N pieces of learning data s _mn , learns and generates a parameter vector Φ used in the scoring model, and transmits it to the search device 13. For example, a method based on an existing maximum entropy model can be used for learning the parameter vector Φ (Non-Patent Document 1). This indicates that when r (d, q) indicates whether or not the document d is a related document of the query q _, the sign of Φ · f _{q, d} matches the sign of r (d, q). It means that learning is done. Note that information r (d, q) indicating whether the document d is a document related to the query q is generated from M × N pieces of learning data s _mn = (m, q _mn ).

Specifically, first, a value that minimizes the following expression is obtained for the two parameter vectors Φ ₊₁ and Φ ₋₁ .

Then, Φ is obtained by Φ ₊₁ −Φ ₋₁ . Note that || x || ² ₂ is an L2-norm, c is a constant, and c = 1 in this embodiment. The L-BFGS algorithm can be used to solve the minimization problem.

<Simulation 1>
An evaluation experiment was conducted on the search device 13 of the first embodiment using the spoken document retrieval test collection (see Reference 2) of the Japanese spoken language corpus CSJ.
(Reference 2) Tomoyosi Akiba, Kiyoaki Aikawa, Yoshiaki Itoh, Tatsuya Kawahara, Hiroaki Nanjo, Hiromitsu Nishizaki, Norihito Yasuda, Yoichi Yamashita, Katunobu Itou, "Construction of a Test Collection for Spoken Document Retrieval from Lecture Audio Data", IPSJ Journal , 2009.Vol.50, No.2, pp.501-513,

  This test collection contains 2702 voice documents and their speech recognition results, 39 queries and their reference labels. The 39 queries were divided into 9 and 30 to make a development set and an evaluation set, respectively. The development set was used only to determine α in equation (3).

In the learning data generating device 11, using a back-off tri-gram language model as the language model, and a language model L _m generated from the language model B and the documents created from all the document using those linear combinations. In Equation (2), λ is any one of 0.9, 0.8, 0.7, 0.6, and 0.5, and the word string generation unit 114 generates 50 queries for each λ of each document. That is, N = 50 × 5 (total number of λ values) = 250 queries are generated from one document. Therefore, the total number of queries generated for learning is 250 × 2702 (total number of documents) = 675500. According to this method, the possibility that a query including only words that do not appear in the document d is generated is not zero. However, such queries are only a fraction of the total and are expected to have little impact on parameter estimation. Therefore, all the queries were used for learning without confirming the existence of the queries. In addition, it is good also as a structure which excludes the query comprised only from the word which does not appear in the document d from learning data.

  The scoring model generation device 12 estimated the parameter vector Φ using 675500 pieces of learning data. In addition, the method based on the existing maximum entropy model was used for learning (refer nonpatent literature 1).

The search device 13 uses the parameter vector Φ to calculate the score of the document for the query of the development set according to the expression (3), and determines the rank as the top candidate of the document in descending order of the value. The total number of types of words is about 27,000, and the dimensions of Φ and f _{q, d} coincide with this.

  The evaluation scale is MAP (mean average precision), R-precition, nDCG (normalized discounted cumulative gain) at the fifth place. In either case, the larger the value, the better the performance. The values in the evaluation set are as shown in FIG. It can be seen that the evaluation of the search device 13 is higher than the evaluation of the baseline search system in any evaluation scale.

<Simulation 2>
In the simulation 1, the word frequency is used for the feature, but a higher-dimensional model can be obtained by using the n-gram frequency. It is also possible to use features related to subwords such as parts of speech, characters and phonemes, so that a robust search result can be expected even when an unknown word appears. Furthermore, by using the reliability of voice recognition as a feature, a robust search against misrecognition of voice recognition can be expected. Since a large amount of learning data can be generated by the learning data generation device 11, the parameter vector can be appropriately learned and generated even if the feature is added to increase the number of dimensions of the feature vector.

  In simulation 2, phonemes are added as features. Also, in simulation 2, 39 queries included 4 that contain unknown words for speech recognition (1 in the development set and 3 in the evaluation set), which were also used for separate evaluation. In the simulation 2, the scoring model generation device 12 learns and generates the parameter vector Φ by the equation (4).

  In simulation 2, nDCG related to MAP and the top 10 is used as an evaluation scale. The results are shown in FIG. In the figure, “eval” indicates the accuracy for an evaluation set of 30 queries, and “oov” indicates the accuracy for 4 queries including unknown words. First, with respect to eval, performance is greatly improved by performing re-scoring (+ words) using a model generated using the scoring model generation device of the first embodiment, compared with the search accuracy of the baseline alone (Baseline). doing. This result is the same as simulation 1. Apart from the presence of unknown words and recognition errors, it is considered that the search ability is essentially improved. The phoneme feature resulted in reduced accuracy with respect to eval. However, focusing on oov, the search accuracy is greatly improved with respect to the baseline. From this, it can be seen that in the first embodiment, the robustness against unknown words is improved simply by adding phoneme features.

<First modification>
The learning data generation device 11 of the first embodiment may be configured not to include the comprehensive language model generation unit 113. In that case, s1101 of FIG. 4 is not performed. Seeking a word train generating means 114 in the document _d probabilistic language model for _{m L} probability of a word W given by _{m P} Lm (W), randomly generates a sentence _{q mn} according to the probability distribution. However, if you generate a random sentence _{q mn} according to the probability distribution of the probability _P Lm _{(W), q mn} is made only in the general vocabulary that appears in the document _{d m.} Since it is rare that all the words appearing in the query appear in the reference document, it is considered that the accuracy is higher when the probabilistic language model for all the documents is used as in the first embodiment.

<Second modification>
The learning data generation device 11 of the first modification may be configured not to further include the individual language model generation unit 112. In that case, s1103 of FIG. 4 is not performed. Word from the word train generating section 114 document d _m at random, phrase, extracting a sentence, a query statement (sentence) that can be connected to it. However, extracting only words at random is equivalent to the case where the individual language model generation unit 112 generates a word unigram language model.

  It is also possible to randomly determine the word, phrase, sentence unit and number to be extracted. In reality, it is rare that all the words appearing in the query appear in the reference document. As a countermeasure, some words and phrases can be replaced or inserted with words and phrases extracted from other documents. The number and position of replacement or insertion can also be determined randomly. However, “random” means following a distribution according to various probability models including a uniform distribution. For example, if it is the number of words constituting the word string, it may be determined according to the Poisson distribution.

  If the relationship between the first embodiment, the first modified example, and the second modified example is seen, it is possible to create a word string (query) by clearly combining any of them. For example, a part of the words extracted by the first embodiment or the first modification may be replaced or inserted by the method of the second modification.

<Other variations>
Comprehensive language model generating means 113, not all may not generate a probabilistic language model for a document, another (background) is a stochastic language model L _m to the document d _m language model, in other words, language model L Any language model including a vocabulary not included in _m may be used.

The search device 13 does not necessarily need to introduce the scoring model of the present embodiment as rescoring for the conventional search method. That is, the search device 13 obtains a score by the equation (1) using only the parameter vector Φ and the feature vector f _{d, p} without using D (f _q , f _d ) as the relevance of the baseline search system. Also good.

Although the first embodiment queries are generated from one document d _m are the N number may change the number of queries generated for each document. For example it may be changed N according to the length of the document d _m.

  The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

<Program and recording medium>
The learning data generation device, scoring model generation device, and search device described above can also be operated by a computer. In this case, each process of a program for causing a computer to function as a target device (a device having the functional configuration shown in the drawings in various embodiments) or a processing procedure thereof (shown in each embodiment) is processed by the computer. A program to be executed by the computer may be downloaded from a recording medium such as a CD-ROM, a magnetic disk, or a semiconductor storage device or via a communication line into the computer, and the program may be executed.

Claims (15)

  A scoring model generation device for learning a scoring model in document search based on statistical model learning,
  M × N pieces of learning data s, where M is the number of documents and N is the number of queries generated from one document. _mn Receive
  The learning data s _mn Feature vector f from query q and document d included in _{q, d} Extract
  Parameter vector Φ and feature vector f _{q, d} The parameter vector Φ is learned so that the inner product of and takes a positive value when the document d is a related document of the query q and a negative value when the document d is not a related document of the query q. ,
  Scoring model generator.   The scoring model generation device according to claim 1,
  || x || ² ₂ Is a L2-norm, c is a constant, information indicating whether the document d is a related document of the query q is r (d, q), and a parameter vector Φ that minimizes the following expression ₊₁ And Φ _-1 Seeking

The parameter vector Φ is obtained by the following equation:
Φ = Φ ₊₁ −Φ _-1
  Scoring model generator. A learning data generation device that is provided with a plurality of documents and generates M × N learning data s _mn for use in the scoring model generation device according to claim 1 or 2 ,
Word string generation means for generating one or more word strings including words included in the document for each given document;
Learning data generating means including each of the generated word strings and a label indicating a document used when generating the word strings as a query and a reference, and a set of the query and the reference as the learning data, respectively. Data generator. The learning data generation device according to claim 3 ,
Further comprising individual language model generation means for generating a probabilistic language model for each of the documents using each of the given documents;
The word string generating means generates the word string based on the probabilistic language model;
Learning data generation device. The learning data generation device according to claim 4 ,
A comprehensive language model generating means for generating probabilistic language models for all the documents using all the given documents;
The word string generation means generates the word string based on two probabilistic language models;
Learning data generation device. A scoring system including the scoring model generation device according to claim 1 and the learning data generation device according to any one of claims 3 to 5 ,
And a search device that performs a document search using the scoring model learned by the scoring model generation device using the learning data generated by the learning data generation device ,
Search system.   A scoring model generation method for learning a scoring model in document search based on statistical model learning using a scoring model generation device,
  M × N pieces of learning data s, where M is the number of documents and N is the number of queries generated from one document. _mn Receive
  The learning data s _mn Feature vector f from query q and document d included in _{q, d} Extract
  Parameter vector Φ and feature vector f _{q, d} The parameter vector Φ is learned so that the inner product of and takes a positive value when the document d is a related document of the query q and a negative value when the document d is not a related document of the query q. ,
  Scoring model generation method.   A scoring model generation method according to claim 7,
  || x || ² ₂ Is a L2-norm, c is a constant, information indicating whether the document d is a related document of the query q is r (d, q), and a parameter vector Φ that minimizes the following expression ₊₁ And Φ _-1 Seeking

The parameter vector Φ is obtained by the following equation:
Φ = Φ ₊₁ −Φ _-1
  Scoring model generation method. A learning data generation method for generating M × N learning data s _mn to be used in the scoring model generation method according to claim 7 or 8 using a learning data generation device given a plurality of documents. And
A word string generation step for generating one or more word strings including words included in the document for each given document;
A learning data generation step in which each of the generated word strings and a label indicating the document used when generating the word strings is a query and a reference, and a set of the query and the reference is the learning data. Data generation method. The learning data generation method according to claim 9 , wherein
Further comprising: generating an individual language model using each given document to generate a probabilistic language model for each document;
In the word string generation step, the word string is generated based on the probabilistic language model.
Learning data generation method. The learning data generation method according to claim 10 ,
A comprehensive language model generating step of generating a probabilistic language model for all the documents using all the given documents;
In the word string generation step, the word string is generated based on two probabilistic language models.
Learning data generation method. A scoring model generation method according to claim 7 or 8, and a learning data generation method according to any one of claims 9 to 11 ,
Further, the document search is performed using the scoring model learned by the learning data generation method using the learning data generated by the learning data generation method .
retrieval method.   The program for functioning a computer as a scoring model production | generation apparatus of Claim 1 or 2.   A program for causing a computer to function as the learning data generation device according to any one of claims 3 to 5.   A program for causing a computer to function as a search device included in the search system according to claim 6.

Images