JP2006065387A - Text sentence search device, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a text sentence search device, a text sentence search method, and a text sentence search program for searching for a text sentence with high precision while taking semantic content of the text sentence into consideration. <P>SOLUTION: A word sectioning/word class giving part 14 sections a query text sentence into words and gives each of them a word class. A syntax/meaning analysis part 16 analyzes a semantic chunk, a center word, and a case of the query text sentence based on the word sectioning/word class giving data. A feature quantity generation part 18 sets a semantic chunk weight and a center word weight and generates semantic vector feature quantity of the query text sentence by using these weights, the word sectioning/word class giving data, a DF value of the text query text sentence among DF values of the words stored in a statistic data storage part 20, and the like. A similarity calculation part 22 calculates similarity between the semantic vector feature quantity of the query text sentence and that of a search object text sentence stored in a database storage part 24 and stores the result in a storage part 26. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a text sentence search device, a search method, and a text sentence search program for searching for a text sentence using, for example, a computer, and more particularly to a text sentence search device and a text sentence search method in consideration of the semantic content of a text sentence. And a text sentence search program.

  With the development of information processing technology and the Internet, various information resources are increasing faster than expected. In such a flood of information, how to extract and provide necessary information to the user is a big issue.

  In addition, in example sentence-based machine translation, it is necessary to retrieve an example sentence most similar to the text to be translated from the example sentence database at high speed and with high accuracy. If an example sentence that closely resembles the translation target text sentence is found, the translation sentence of the translation target text sentence can be easily generated using the translation of the example sentence. Therefore, the search accuracy depends on the accuracy of translation, and high-speed high-speed search technology is one of important technologies for machine translation.

There are (1) a search method using a search keyword and (2) a vector space method (Vector Space Model: VSM) using a statistical technique as main methods of information search using a computer. In the vector space method VSM, a text document is expressed by a vector feature amount, for example, a TF ^* IDF (Term Frequency ^* Inverted Document Frequency) value is used as a vector feature amount, and the similarity between the vector feature amounts is used. Find the similarity between texts.

  However, the vector space method is a technique based on statistical theory, and is not a search method based on the semantic content of text. Therefore, there is a limit to the accuracy of search, and there is a problem that it cannot cope with example sentence-based machine translation.

  Since the semantic information of the text sentence represents the content of the text sentence and is not affected by the syntax structure or the form of the sentence, research on information comparison or retrieval using the semantic information has been conducted. For example, Non-Patent Document 1 describes a method in which semantic relations of text are expressed by case grammar, and semantic relations are compared by a tree matching method. However, there is a problem that the search accuracy of this method is lower than that of the vector space method.

  Non-Patent Document 2 describes a method in which a partial relationship matching method and a vector space method are merged. However, this method has a problem that the search accuracy is low although the search coverage is superior to the vector space method.

  Non-Patent Document 3 describes a search method using a causal relationship between text sentences. However, this method is basically an improved method of keyword search, and has a problem that it cannot cope with high-precision text sentence search required for example sentence-based machine translation.

  Further, in Patent Document 1, a plurality of types of words having synonym relations are regarded as one type of word, and vector data (updated vector data) including the appearance frequency of the words is created, and search is performed using this vector data. A technique for calculating the similarity between a key document and a search target document is described. According to this technique, even when words having the same meaning but different notations are mixed in one document, or even when words included in each document to be compared have the same meaning but different notations, High similarity document search can be realized.

  In Patent Document 2, the input character string is divided and assigned with morpheme information, and the dependency relation between phrases is analyzed based on the morpheme information, and the sentence structure is determined from the analysis result. , A technique for extracting an index from this sentence structure, assigning the importance of the index, and determining the similarity between the input document and the stored document from the similarity of the index and the similarity of the dependency relationship is described. Yes.

Japanese Patent Application Laid-Open Publication No. 2003-259542 discloses a Japanese language document that is syntactically analyzed / semantically analyzed / context analyzed using a knowledge base, expressed in a deep structure of a certain normalized propositional format, Check the semantics of the deep structure of the sentence and the theme, check the semantics of the theme that conforms to the semantic collation and the deep structure under the theme related to the theme, and create a document that has the deep structure that conforms to the semantic collation. The technology to output is described.
Lu, X. (1990). "An application of case relations to document retrieval (Doctoral dissertation, University of Western Ontario, 1990)". Dissertation Abstracts International, 52-10, 3464A Liu, GZ (1997). "Semantic vector space model: Implementation and evaluation". Journal of the American Society for Information Science, 48 (5), 395-417 Khoo, Christopher Soo-Guan (1997). "The use of relation matching in information retrieval". Electronic Journal ISSN 1058-6768 Japanese Patent Application Laid-Open No. 11-110395 Japanese Patent Laid-Open No. 3-172966 Japanese Unexamined Patent Publication No. 1-221624

  However, the techniques described in Patent Documents 1 to 3 have a problem that a text sentence may not be searched with high accuracy in consideration of the meaning content.

  The present invention has been made in consideration of the above facts, a text sentence search device, a text sentence search method, and a text sentence search capable of searching a text sentence with high precision in consideration of the semantic content of the text sentence. The purpose is to obtain a program.

  In order to achieve the above object, a text sentence search device according to claim 1, which divides a question text sentence into words, assigns parts of speech to the separated words, and generates word separation / part of speech provision data. And a syntactic / semantic analyzing unit that analyzes a semantic chunk included in the question text sentence, a central word of the semantic chunk, and a case of the semantic chunk based on the word segmenting / part of speech providing unit, Based on the appearance frequency data storage means in which appearance frequency data representing the frequency of appearance in the text sentence set to be searched for each word appearing in the text sentence set to be searched is stored in advance, and the case of the semantic chunk Setting means for setting at least one of a semantic chunk weight and a central word weight of a word included in the semantic chunk Calculating the weight of each word included in the question text sentence based on the appearance frequency data for the word included in the question text sentence and at least one of the semantic chunk weight and the central word weight. The feature quantity generating means for generating the semantic vector feature quantity of the question text sentence, the semantic vector feature quantity storing means for storing in advance the semantic vector feature quantity of each text sentence of the text sentence set to be searched, and the question text A similarity calculation means for calculating a similarity between the question text sentence and the text sentence to be searched based on a semantic vector feature quantity of the sentence and a semantic vector feature quantity of the text sentence to be searched; It is characterized by.

  According to this invention, the word segmentation / part-of-speech giving means segmentes the question text sentence into words, and gives part-of-speech such as a noun, verb, particle, etc. to each segmented word.

  The syntax / semantic analysis unit analyzes the semantic chunk, the central word of the semantic chunk, and the case of the semantic chunk included in the question text sentence based on the word segmentation / part of speech assignment data generated by the word segmentation / part of speech assignment unit. . Here, the semantic chunk is a collection of words constituting a single meaning. The central word is a word that plays an important role as semantic content in the semantic chunk. One or more central words exist in the semantic chunk. The case represents a semantic case of a word, such as an operation subject or a predicate target.

  The appearance frequency data storage means stores in advance appearance frequency data calculated for each of the words appearing in a plurality of text sentences that are comparison targets of the text sentence set to be searched, that is, the question text sentence. The appearance frequency data is data representing the frequency with which words appearing in the text sentence set to be searched appear in the text sentence set.

  The setting means sets at least one of a semantic chunk weight and a central word weight of a word included in the semantic chunk based on the case of the semantic chunk analyzed by the syntax / semantic analysis means. Specifically, for example, cases are classified into a plurality of classes according to importance, and semantic chunk weights and central word weights are set according to the importance of the cases.

  The feature quantity generation means reads appearance frequency data of each word included in the question text sentence from the appearance frequency data storage means, and the read appearance frequency data and the semantic chunk weight of the word included in the semantic chunk set by the setting means And the weights of the words included in the question text sentence are calculated based on at least one of the central word weights. The meaning vector feature amount of the question text sentence is constituted by the weight of each word thus calculated.

  The semantic vector feature amount storage means stores in advance the semantic vector feature amount of each text sentence in the set of text sentences to be searched. As the semantic vector feature amount of each text sentence in the set of text sentences to be searched, those generated in the same manner as generating the semantic vector feature quantity of the question text sentence can be used.

  The similarity calculation unit is configured to calculate a question based on the meaning vector feature amount of the question text sentence generated by the feature amount generation unit and the meaning vector feature amount of the text sentence to be searched stored in the meaning vector feature amount storage unit. The similarity between the text sentence and the text sentence to be searched is calculated.

  As described above, according to the present invention, the semantic chunk feature weight and the central word weight set in accordance with the significance level of the semantic chunk are taken into consideration to generate the semantic vector feature amount, so that the semantic content is appropriately considered. Similarity calculation can be performed, and a text sentence similar to the question text sentence can be searched with high accuracy.

  In addition, as described in claim 2, the feature amount generation unit uses the weight of a word included in the question text sentence as a first parameter based on a frequency with which the word appears in the question text sentence. It can be calculated by multiplying an addition value obtained by adding at least one of the semantic chunk weight and the central word weight by a second parameter based on the frequency of occurrence of the word in the text sentence set to be searched.

  In addition, as described in claim 3, the feature amount generation unit uses the weight of a word included in the question text sentence as a first parameter based on the frequency with which the word appears in the question text sentence. Calculating by multiplying a multiplication value obtained by multiplying a second parameter based on a frequency at which a word appears in the search target text sentence set by a coefficient including at least one of the semantic chunk weight and the central word weight. it can.

  In addition, as described in claim 4, the similarity calculation unit searches for a matching word among a word included in the question text sentence and a word included in the search target text sentence, and the searched word The degree of similarity can be calculated by adding a value obtained by multiplying each of the weights for all the matched words.

  Further, as described in claim 5, the similarity calculation unit searches for a word that matches between a word included in the question text sentence and a word included in the search target text sentence, and the searched word The degree of similarity can be calculated by calculating the distance of the respective weights and adding the calculated distance to all the matched words.

  According to a sixth aspect of the present invention, there is provided a text sentence search method for segmenting a question text sentence into words, adding part of speech to the segmented word, generating word segmentation / part of speech provision data, and based on the word segmentation / part of speech provision data. Analyzing the semantic chunk included in the question text sentence, the central word of the semantic chunk, and the case of the semantic chunk, and based on the case of the semantic chunk, the semantic chunk weight of the word included in the semantic chunk and At least one of the central word weights is set and included in the question text sentence out of the appearance frequency data representing the frequency of appearance in the search target text sentence set for each word appearing in the search target text sentence set The appearance frequency data for a word, and at least one of the semantic chunk weight and the central word weight; Based on the above, the semantic vector feature amount of the question text sentence is generated by calculating the weight of each word included in the question text sentence, and the meaning vector feature amount of the question text sentence and the meaning of the text sentence to be searched The similarity between the question text sentence and the search target text sentence is calculated based on a vector feature amount.

  According to the present invention, since the semantic vector feature amount is generated in consideration of the semantic chunk weight and the central word weight set in accordance with the importance of the meaning of the semantic chunk, the similarity calculation considering the semantic content appropriately It is possible to search for a text sentence similar to the question text sentence with high accuracy.

  The text sentence search program of the invention according to claim 7 divides a question text sentence into words, assigns parts of speech to the divided words, generates word separation / part of speech provision data, and the word separation / part of speech provision data. Analyzing the semantic chunk included in the question text sentence, the central word of the semantic chunk, and the case of the semantic chunk, and based on the case of the semantic chunk, the word of the word included in the semantic chunk Of the appearance frequency data representing the frequency of appearing in the search target text sentence set for each of the words appearing in the search target text sentence set, the step of setting at least one of the semantic chunk weight and the central word weight, The appearance frequency data for the words included in the question text sentence and the semantic chunk Generating a semantic vector feature amount of the question text sentence by calculating a weight of each word included in the question text sentence based on at least one of the first and the central word weights; and the question text sentence Calculating a similarity between the question text sentence and the text sentence to be searched based on the meaning vector feature quantity of the text and the semantic vector feature quantity of the text sentence to be searched. It is characterized by making it.

  According to the present invention, since the semantic vector feature amount is generated in consideration of the semantic chunk weight and the central word weight set in accordance with the importance of the meaning of the semantic chunk, the similarity calculation considering the semantic content appropriately It is possible to implement a computer that can search a text sentence similar to the question text sentence with high accuracy.

  The text sentence search device according to claim 8 performs a linguistic semantic analysis of a question text sentence, sets a semantic importance of each word of the question text sentence, and analyzes the linguistic semantic analysis. A feature amount generation unit that generates a feature amount of the question text sentence using a result and a vector space method; and a similarity between the feature amount of the question text sentence generated by the feature amount generation unit and the feature amount of the search target text sentence A similarity calculation unit that calculates a degree; and a search result extraction unit that extracts a text sentence as a search result from a set of search target text sentences based on the calculation result of the similarity.

  According to the present invention, the language semantic analysis of the question text sentence is performed, the semantic importance of each word of the question text sentence is set, the analysis result of the language semantic analysis and the feature of the question text sentence using the vector space method The amount of the generated query text sentence and the feature quantity of the search target text sentence are calculated, and based on the similarity calculation result, the text that is the search result from the search target text sentence set Since the sentence is extracted, it is possible to perform the similarity calculation considering the semantic content appropriately, and it is possible to search the text sentence similar to the question text sentence with high accuracy.

  In addition, as described in claim 9, the semantic analysis unit can perform language semantic analysis based on case grammar.

  The similarity calculation unit can calculate the similarity using at least one of an inner product, a cosine function, and a Euclidean distance.

  In addition, as described in claim 11, the feature quantity generation unit can calculate a feature quantity obtained by using the vector space method by the following equation.

TFi x log (N / ni + c)
Here, TFi is the number of times the word i appears in the text sentence, N is the total number of text sentences in the search target text sentence set, ni is the total number of text sentences including the word i in the search target text sentence set, c is a constant, c ≧ 0.01 is set.

  In addition, as described in claim 12, the semantic analysis unit extracts each semantic case in the text sentence and assigns a different weight to the semantic case based on the importance of the extracted semantic case. And a word having a central role in the text sentence, and a configuration having at least one of a central word analysis unit for assigning different weights to the central word based on the importance of the extracted central word; can do.

  In this case, as described in claim 13, the feature amount generation unit uses the semantic case weight obtained by the semantic case analysis unit and the central word weight obtained by the central word analysis unit. The feature amount obtained by using the vector space method can be corrected by the following equation.

(TFi + Chunk_Weight + Head_Weight) × log (N / ni + c)
Where TFi is the number of times the word i appears in the text sentence, Chunk_Weight is the semantic weight to which the word i belongs, Head_Weight is the central word weight (used when the word i is the central word), and N is the search target text sentence The total number of text sentences in the set, ni is the total number of text sentences including the word i in the search target text sentence set, c is a constant, and c ≧ 0.01.

  In addition, as described in claim 14, the feature amount generation unit uses the weight of the semantic case obtained by the semantic case analysis unit to calculate a feature amount obtained by using the vector space method according to the following equation: You may make it correct.

(TFi + Chunk_Weight) × log (N / ni + c)
In addition, as described in claim 15, the feature quantity generation unit corrects the feature quantity obtained by using the vector space method according to the following equation, using the central word weight obtained by the central word analysis unit. You may make it do.

(TFi + Head_Weight) × log (N / ni + c)
In addition, as described in claim 16, the feature value generation unit uses the semantic case weight obtained by the semantic case analysis unit and the central word weight obtained by the central word analysis unit, You may make it correct the feature-value obtained using the said vector space method by following Formula.
(TFi × log (N / ni + c)) × (Chunk_Weight + Head_Weight) / 2
In addition, as described in claim 17, the feature value generation unit uses a weight of the semantic case obtained by the semantic case analysis unit to calculate a feature value obtained using the vector space method according to the following equation: You may make it correct.

TFi x log (N / ni + c) x Chunk_Weight
In addition, as described in claim 18, the feature value generation unit corrects the feature value obtained by using the vector space method according to the following expression, using the central word weight obtained by the central word analysis unit. You may make it do.

  TFi × log (N / ni + c) × Head_Weight

  According to the present invention, there is an effect that a text sentence can be searched with high accuracy in consideration of the semantic content of the text sentence.

  Hereinafter, embodiments of the present invention will be described.

  FIG. 1 is a block diagram showing a schematic configuration of a text sentence search apparatus 10 considering semantic information according to the present invention. As shown in FIG. 1, the text sentence search device 10 includes an external storage device 12, a word segmentation / part of speech assignment unit 14, a syntax / semantic analysis unit 16, a feature amount generation unit 18, a statistical data storage unit 20, and a similarity calculation unit. 22, a database storage unit 24, a storage unit 26, and memories 28, 30, 32, and 34.

  The external storage device 12 stores, for example, question text sentence data input by a user. One question text sentence data stored in the external storage device 12 is stored in the memory 28.

  The word segmentation / part-of-speech assigning unit 14 isolates the words of the question text sentence data stored in the memory 28, assigns the part of speech to the separated words, and stores the result in the memory 30 as the word segmentation / part-of-speech assignment data. .

  The syntax / semantic analysis unit 16 reads word segmentation / part-of-speech assignment data of the question text sentence stored in the memory 30, and performs syntax analysis and semantic analysis of the question text sentence based on the word segmentation / part-of-speech assignment data. The syntax / semantic analysis data as a result is stored in the memory 32.

  The statistical data storage unit 20 stores the number of text sentences including the words (DF) for all words that appear in the text sentence set from a set of search target text sentences (search target text sentence set) prepared in advance. : Document Frequency) (DF value) is stored as appearance frequency data. That is, the DF value represents the frequency that the word appears in the text sentence set. The statistical data storage unit 20 also stores in advance natural number ID data of collectable word sets. That is, a unique natural number ID (word ID) is assigned to each word, and the DF value of the word assigned the word ID is stored in the statistical data storage unit 20. Note that it is preferable to store the synonym word with the same ID. As a result, words having different notations but the same meaning are treated equally, and the text sentence search accuracy can be improved.

  The feature quantity generation unit 18 reads the word segmentation / part of speech data of the question text sentence stored in the memory 30, and among the word DF values stored in the statistical data storage unit 20, the question text stored in the memory 30. Read the DF value of the word of the sentence. Then, using the read word segmentation / part-of-speech assignment data, the DF value of the word of the question text sentence, and the syntax / semantic analysis data stored in the memory 32, a semantic vector feature quantity of the question text sentence is generated, and the data is obtained. Store in the memory 34.

  The database storage unit 24 stores in advance semantic vector feature values calculated in advance for all text sentences in the set of text sentences to be searched.

  The similarity calculation unit 22 calculates the similarity (distance) between the semantic vector feature amount of the question text sentence stored in the memory 34 and the semantic vector feature amount of the text sentence to be searched stored in the database storage unit 24. ), And the calculated similarity is stored in the storage unit 26. Thereby, it is possible to display a text sentence to be searched that has the highest similarity to the question text sentence, or to display a list of text sentences to be searched in descending order of similarity.

  The text sentence search apparatus 10 configured as described above can be applied to an information terminal apparatus 40 as shown in FIG. 2, for example.

  The information terminal device 40 includes a hard disk 42, a keyboard 44, a display 46, and a processor unit 48.

  The hard disk 42 stores the data of the question text sentence input from the keyboard 44, various calculation results such as the similarity between the question text sentence calculated by the processor unit 48 and the text sentence to be searched, various software, and the like. The The hard disk 42 is also used as a storage space necessary for calculation. In addition, you may use not only the hard disk 42 but another external storage device.

  The keyboard 44 is an input device for a user to input a text sentence and instruct various operations. Other input devices such as a mouse may be provided.

  The display 46 is an output device for displaying a message to the user, text text data, similarity calculation results, and the like. Other output devices such as a printer may be provided.

  The processor unit 48 performs actual processing according to software stored in the hard disk 42. Specifically, the processor unit 48 can be configured by a computer system such as a microprocessor or a personal computer.

  The word segmentation / part-of-speech adding unit 14, the syntax / semantic analysis unit 16, the feature amount generation unit 18, and the similarity calculation unit 22 can be configured by various modules operating on the processor unit 48.

  Next, a specific operation of the text sentence search apparatus 10 will be described.

  The external storage device 12 stores data of the question text sentence input by the user. One text sentence data is read from the external storage device 12 and stored in the memory 28.

  The word segmentation / part-of-speech assigning unit 14 isolates each word constituting the text sentence, assigns the part-of-speech for each word, and stores it in the memory 30 as word segmentation / part-of-speech provision data. Note that publicly known methods can be used for the method of segmenting words and assigning parts of speech. For example, a word segmentation / part-of-speech assignment tool developed by Tsinghua University in China may be used, or another analysis tool may be used.

  As an example, when the Chinese text sentence 50 shown in FIG. 3 is input to the word segmentation / part-of-speech adding unit 14, word segmentation / part-of-speech provision data 52 is output. In addition, the Chinese text sentence 50 means that “Australian general Ferriman won a gold medal in the women ’s 400-meter final on September 25, 2000, Beijing time Monday afternoon”.

  The word segmentation / part-of-speech assignment data 52 includes a word part in which the part-of-speech symbol representing a part-of-speech such as a noun, a verb, or a particle is “/ ○” (○ is an alphabet representing an arbitrary part-of-speech). Data given to the end of the word. For example, the part of speech symbol represented by “/ nr” represents a noun. Note that the word segmentation / part-of-speech giving data 52 is not limited to the format shown in FIG.

  The syntax / semantic analysis unit 16 includes at least one of a semantic case analysis unit and a central word analysis unit, reads word segmentation / part-of-speech assignment data of a text sentence stored in the memory 30, and syntax / semantic analysis of the question text sentence To set the semantic significance of each word. Specifically, the semantic chunk, the central word, and the case (semantic case) constituting the question text sentence are analyzed, and the analysis result is stored in the memory 32 as syntax / semantic analysis data. Note that publicly known methods can be used as methods for syntax analysis and semantic analysis. For example, a syntax / semantic analysis tool developed by Tsinghua University in China may be used, or another analysis tool may be used.

  A text sentence is generally composed of a plurality of words, and one or a plurality of words that play an important role are included in the plurality of words. Here, a word that plays an important role in a text sentence is called a central word (Head).

  As an example, when a Chinese text sentence 54 shown in FIG. 4 is input to the syntax / semantic analysis unit 16, syntax / semantic analysis data 56 is output. The Chinese text sentence 54 has the following meaning.

  The syntax / semantic analysis data 56 is in a format in which each semantic chunk analyzed from the text sentence 54 is separated by “[]” and the central word is expressed in a gossic font. In addition, the alphabet given at the head in parentheses represents syntax information (chunk name), and the alphabet given after the parentheses represents the case of the semantic chunk.

  The alphabet that expresses syntax information (chunk name) is defined as follows.

S (Subject) ... Subject chunk P (Predicate) ... Predicate chunk O (Object) ... Acceptance chunk D ... Adverb chunk C (Complement) ... Complement chunk V ... Predicate or predicate chunk One sentence about the meaning of a semantic chunk , O1 (subject of predicate), and V (motion) are included in S1, S2,..., O1 depending on the order of appearance from the left to the right of the sentence. , O2,..., V1, V2,.

  The syntax / semantic analysis data 56 is not limited to the format shown in FIG. 4, but may be any other format as long as the semantic chunk, the central word, and the case can be discriminated.

  The feature quantity generation unit 18 generates a semantic vector feature quantity of the question text sentence using the word segmentation / part of speech assignment data, the DF value of the word of the question text sentence, and the syntax / semantic analysis data, and stores the data in the memory 34. Store.

  Next, the semantic vector feature amount will be described. Prior to that, the conventional vector feature amount will be described.

  A text sentence can be expressed by a vector V (f1, f2, f3,..., Fs). Here, s is the total number of words. Further, fi is a weight (feature amount) of a word (hereinafter referred to as a word i) having a word ID that is an i-th largest word among the words that appear in the text sentence and is included in the text sentence. expressed.

fi = TFi × log (N / ni) (1)
TFi (first parameter) is the frequency of the word i in the text sentence, that is, the number of words i included in the text sentence. Therefore, TFi tends to increase as the length of the text sentence increases. log (N / ni) is a so-called IDF (second parameter), N is the total number of text sentences included in the set of text sentences to be searched stored in the database storage unit 24, and ni is the DF of the word i. Represents a value. That is, the IDF becomes smaller as the number of text sentences including the word i is larger, and becomes larger as the number of text sentences including the word i is smaller. Accordingly, the IDF represents the importance of the word i for the target text sentence.

  The dimension number s of V (f1, f2, f3,..., Fs) is the total number of words that appear in the text sentence. About the vector feature-value of each text sentence, the number of dimensions may differ, respectively, and the word corresponding to each dimension may differ.

  Further, as described above, each word i is given a natural number ID, and in order to calculate the similarity between text sentences based on the vector feature amount, this ID is associated with the weight fi, The feature amount is expressed by V (ID1, f1, ID2, f2, ID3, f3,..., Ids, fs). Note that ID1, ID2,... Are arranged in ascending order of ID numbers.

  The feature quantity generation unit 18 reads the DF value of each word of the text sentence recorded in the memory 28 from the statistical data storage unit 20, and uses the syntax / semantic analysis data stored in the memory 32 to A semantic vector feature quantity Sv (ID1, f1, ID2, f2, ID3, f3,..., Ids, fs) is created.

  Next, a case representing a semantic relationship will be described.

  FIG. 5 shows a case representing the semantic relationship and a classification table 58 in which the cases are classified according to importance. In the example of FIG. 5, the cases are classified from the first class to the fifth class, and the cases classified into the first class have the highest importance. The first class includes semantic cases of proper nouns such as main actors.

  For example, since the case information of the first semantic chunk of the syntax / semantic analysis data 56 shown in FIG. 4 is “S”, the case class of this semantic chunk is the first class. Similarly, since the case information of the second semantic chunk is “H”, the case class of this semantic chunk is also the first class. In this way, the case of each semantic chunk is classified according to importance.

  Next, a method for generating the semantic vector feature amount Sv (ID1, f1, ID2, f2, ID3, f3,..., Ids, fs) will be described.

  If the weight of the central word is increased with respect to the vector feature amount of the question text sentence, the vector feature amount easily reflects the meaning of the question text sentence. Therefore, the feature quantity generation unit 18 generates the semantic vector feature quantity of the question text sentence using the frequency of words appearing in the question text sentence, the semantic chunk, and the central word. Hereinafter, three semantic vector feature value generation methods will be described.

  First, the first semantic vector feature value generation method will be described. In this method, first, the semantic chunk weight Chunk_Weight is set as shown in FIG. 6 according to the case class. Here, a is a constant (a ≧ 0). That is, the higher the case class is, the larger the semantic chunk weight Chunk_Weight becomes. The semantic chunk weight may be defined by other methods.

  Further, the central word weight Head_Weight is set as shown in FIG. 7 according to the case class. Here, b is a constant (b ≧ 0). That is, the higher the case class, the larger the central word weight Head_Weight. The central word weight may be defined by other methods.

  Then, the weight fi of the word i constituting the semantic vector feature amount Sv (ID1, f1, ID2, f2, ID3, f3,..., IDs, fs) of the text sentence is obtained by the following equations.

fi = (TFi + Chunk_Weight + Head_Weight) × log (N / ni + c) (2)
Here, N is the total number of text sentences in the text sentence set as described above. TFi is the frequency of the word i that appears in the text sentence, and ni is the total number of text sentences that include the word i in the search target text set. c is a constant and is set to c ≧ 0.01. The central word weight Head_Weight is used only when the word i is the central word. If the word i is not the central word, Head_Weight = 0.

  The difference between the expression (2) in the first semantic vector feature value generation method and the conventional expression (1) is that the sum of the semantic chunk weight and the central word weight is added to TFi. For this reason, the value of the weight fi changes according to the class of the case, and a semantic vector feature amount that appropriately reflects the meaning of the text sentence as compared with the conventional case is generated. Note that the first semantic vector feature value generation method is less influenced by the semantic chunk weight and the central word weight than the second and third semantic vector feature value generation methods described below. It is suitable for simple sentence search, and can be said to be a method suitable for machine translation and the like.

  Next, a second semantic vector feature value generation method will be described. In this method, as in the first semantic vector feature value generation method, the semantic chunk weight Chunk_Weight is set as shown in FIG. 6 according to the case class, but the constant a is set to a ≧ 1.

  The central word weight Head_Weight is also set as shown in FIG. 7 according to the case class, as in the first semantic vector feature generation method, but the constant b is set to b ≧ 1.

  Then, the weight fi of the word i constituting the semantic vector feature amount Sv (ID1, f1, ID2, f2, ID3, f3,..., IDs, fs) of the text sentence is obtained by the following equations.

fi = (TFi × log (N / ni + c)) × (Chunk_Weight + Head_Weight) / 2 (3)
Here, the constant c is set to c ≧ 0.01, as in the first semantic vector feature value generation method. When the word i is not the central word, Head_Weight = Chunk_Weight is set.

  In the second semantic vector feature value generation method, the equation (3) differs from the conventional equation (1) in that the semantic chunk weight and the central word weight are different from the fi obtained by the equation (1). The value obtained by dividing the sum of the two by 2, that is, the average value of the semantic chunk weight and the central word weight is multiplied. Further, in the second semantic vector feature value generation method, the constants a and b are both set to a value of 1 or more. Therefore, the fi obtained by the conventional equation (1) is large depending on the class of the case. It tends to be a value. Therefore, the importance of the semantic content is reflected more strongly, and a semantic vector feature amount that appropriately reflects the meaning of the text sentence as compared with the conventional case is generated.

  Next, a third semantic vector feature value generation method will be described. In this method, as in the first semantic vector feature value generation method, the semantic chunk weight Chunk_Weight is set as shown in FIG. 6 according to the case class, but the constant a is set to a ≧ 1.

  The central word weight Head_Weight is also set as shown in FIG. 7 according to the case class, as in the first semantic vector feature generation method, but the constant b is set to b ≧ 1.

  Then, the weight fi of the word i constituting the semantic vector feature amount Sv (ID1, f1, ID2, f2, ID3, f3,..., IDs, fs) of the text sentence is obtained by the following equations.

fi = (TFi × log (N / ni + c)) × Chunk_Weight × Head_Weight (4)
Here, the constant c is set to c ≧ 0.01, as in the first semantic vector feature value generation method. When the word i is not the central word, Head_Weight = 1 is set.

  The point (4) in the third meaning vector feature generation method differs from the above point (1) in that the meaning chunk weight and the central word weight are different from the fi obtained in the above point (1). The value multiplied by is multiplied. In the third semantic vector feature value generation method, both constants a and b are set to a value of 1 or more, so that fi is large in accordance with the class of class obtained from the conventional equation (1). It tends to be a value. Therefore, the importance of the semantic content is reflected more strongly, and a semantic vector feature amount that appropriately reflects the meaning of the text sentence as compared with the conventional case is generated. When comparing the third semantic vector feature value generation method and the second semantic vector feature value generation method, the fi obtained by the above equation (4) is more than the fi obtained by the above equation (3). It tends to be a large value.

  As described above, the database storage unit 24 stores in advance the semantic vector feature quantities of each text sentence in the text sentence set to be searched. These semantic vector feature quantities are the above-described semantic vector feature quantities. Using the generation method, it can be generated in the same way as the semantic vector feature amount of the question text sentence.

  Next, a specific similarity calculation method in the similarity calculation unit 22 will be described.

  The similarity calculator 22 calculates the similarity of the text sentence according to any of the following formulas (5) to (7).

Here, s1 and s2 represent the number of dimensions of the semantic vector feature quantity of the text sentence S1 and the text sentence S2, respectively, and f1 _k and f2 _h are weights of the respective dimension of the semantic vector feature quantity of the text sentence S1 and the text sentence S2, respectively. Represents. ID1 _k and ID2 _h represent word IDs of each dimension of the semantic vector feature amount of the text sentence S1 and the text sentence S2, respectively.

  First, the first similarity calculation by the above equation (5) will be specifically described with reference to the flowchart shown in FIG.

  In step 100, the subscript k for the text sentence S1, the subscript h for the text sentence S2, and the similarity Sim (S1, S2) are initialized. That is, k = 1, h = 1, and Sim (S1, S2) = 0.

In step 102, it is determined whether or not the word IDs of the text sentence S1 and the text sentence S2 are the same. That is, it is determined whether or not ID1 _k = ID2 _h . If the word IDs are the same, the process proceeds to step 104. If the word IDs are not the same, the process proceeds to step 108.

In step 104, k and h are respectively incremented, and a multiplication value of weights f1 _k and f2 _h is added to the current similarity Sim (S1, S2), and this is added to a new similarity Sim (S1, S2). And That is, the k = k + 1, h = h + 1, Sim (S1, S2) = Sim (S1, S2) + f1 k × f2 h.

  In step 106, it is determined whether k is greater than the dimension number s1, and whether h is greater than the dimension number s2, and if k is greater than the dimension number s1 or h is greater than the dimension number s2, The routine ends, otherwise, the process returns to step S2 to repeat the same processing as described above.

At step 108, it is determined whether a word ID1 _k text statement S1 is smaller than the word ID2 _h text sentence S2. That is, it is determined whether or not the relationship of ID1 _k <ID2 _h is satisfied. If ID1 _k <ID2 _h , the process proceeds to step 110; otherwise, that is, if ID1 _k > ID2 _h , the process proceeds to step 114.

  In step 110, k is incremented and the routine proceeds to step 112. In step 112, it is determined whether or not k is larger than the number of dimensions s1, and if k is larger than the number of dimensions s1, this routine is terminated. If k is less than or equal to the number of dimensions s1, the process goes to step 102. Return and repeat the same process as above.

  On the other hand, in step 114, h is incremented and the routine proceeds to step 116. In step 116, it is determined whether or not h is larger than the number of dimensions s2. If h is larger than the number of dimensions s2, this routine is ended. If h is smaller than or equal to the number of dimensions s2, the process proceeds to step 102. Return and repeat the same process as above.

  As described above, in this routine, by comparing the word ID of the word included in the text sentence S1 with the word ID included in the text sentence S2, the words included in both text sentences are searched, and both text sentences are searched. Multiply the weights of the words contained in, and add the multiplied values sequentially. As described above, since ID1, ID2,... Of the semantic vector feature amount Sv (ID1, f1, ID2, f2, ID3, f3,..., IDs, fs) are arranged in ascending order of ID, FIG. By using a simple flow as shown in Fig. 5, it is possible to quickly search for words included in both text sentences.

  Next, the second similarity calculation by the above equation (6) will be specifically described with reference to the flowchart shown in FIG.

In step 200, the denominator of the above equation (6) is calculated. That is, the square root of the product of the sum of the squares of the weights f1 _k and the sum of the squares of the weights f2 _h is obtained.

  In step 202, the subscript k for the text sentence S1, the subscript h for the text sentence S2, and the similarity Sim (S1, S2) are initialized. That is, k = 1, h = 1, and Sim (S1, S2) = 0.

In step 204, it is determined whether or not the word IDs of the text sentence S1 and the text sentence S2 are the same. That is, it is determined whether or not ID1 _k = ID2 _h . If the word IDs are the same, the process proceeds to step 206. If the word IDs are not the same, the process proceeds to step 212.

In step 206, k and h are incremented, and a multiplication value of weights f1 _k and f2 _h is added to the current similarity Sim (S1, S2), and this is added to a new similarity Sim (S1, S2). And That is, the k = k + 1, h = h + 1, Sim (S1, S2) = Sim (S1, S2) + f1 k × f2 h.

  In step 208, it is determined whether or not k is larger than the dimension number s1, and whether or not h is larger than the dimension number s2. If k is larger than the dimension number s1 or h is larger than the dimension number s2, Otherwise, the process proceeds to 210. Otherwise, the process returns to step 204 and the same processing as described above is repeated.

At step 212, it is determined whether a word ID1 _k text statement S1 is smaller than the word ID2 _h text sentence S2. That is, it is determined whether or not the relationship of ID1 _k <ID2 _h is satisfied. If ID1 _k <ID2 _h , the process proceeds to step 214; otherwise, that is, if ID1 _k > ID2 _h , the process proceeds to step 218.

  In step 214, k is incremented and the routine proceeds to step 216. In step 216, it is determined whether or not k is larger than the number of dimensions s1. If k is larger than the number of dimensions s1, the process proceeds to step 210. If k is less than or equal to the number of dimensions s1, the process proceeds to step 204. Return and repeat the same process as above.

  On the other hand, at step 218, h is incremented and the routine proceeds to step 220. In step 220, it is determined whether or not h is larger than the number of dimensions s2. If h is larger than the number of dimensions s2, the process proceeds to step 210. If h is less than or equal to the number of dimensions s2, the process proceeds to step 204. Return and repeat the same process as above.

  In step 210, the current similarity Sim (S1, S2) is divided by W obtained in step 200, and this is used as the final similarity Sim (S1, S2). That is, Sim (S1, S2) = Sim (S1, S2) / W.

  Thus, the similarity obtained in this routine differs only in that the similarity obtained in the routine shown in FIG. 8 is divided by W. The similarity Sim (S1, S2) obtained by the above equations (5) and (6) is higher as the value is higher.

  Next, the third similarity calculation according to the equation (7) will be specifically described with reference to the flowchart shown in FIG.

  In step 300, a subscript k for the text sentence S1, a subscript h for the text sentence S2, and a variable W for Euclidean distance calculation are initialized. That is, k = 1, h = 1, and W = 0.

In step 302, it is determined whether or not the word IDs of the text sentence S1 and the text sentence S2 are the same. That is, it is determined whether or not ID1 _k = ID2 _h . If the word IDs are the same, the process proceeds to step 304. If the word IDs are not the same, the process proceeds to step 310.

In step 304, k and h are incremented, and a value obtained by squaring the difference between the weights f1 _k and f2 _h is added to the current variable W, and this is used as a new variable W. That is, k = k + 1, h = h + 1, and W = W + (f1 _k −f2 _h ) ² .

  In step 306, it is determined whether k is greater than the dimension number s1, and whether h is greater than the dimension number s2. If k is greater than the dimension number s1 or h is greater than the dimension number s2, step 306 is performed. If not, the process returns to step 302 to repeat the same processing as described above.

At step 310, it is determined whether a word ID1 _k text statement S1 is smaller than the word ID2 _h text sentence S2. That is, it is determined whether or not the relationship of ID1 _k <ID2 _h is satisfied. If ID1 _k <ID2 _h , the process proceeds to step 312; otherwise, that is, if ID1 _k > ID2 _h , the process proceeds to step 316.

In step 312, k is incremented and a value _obtained by squaring the weight f1 _k is added to the current variable W, and this is set as a new variable W. That is, k = k + 1 and W = W + (f1 _k ) ² .

  In step 314, it is determined whether or not k is larger than the number of dimensions s1, and if k is larger than the number of dimensions s1, the process proceeds to step 319. If k is less than the number of dimensions s1, the process proceeds to step 302. Return and repeat the same process as above.

  In step 319, the variable W is calculated by the following equation, and the process proceeds to step 308.

On the other hand, in step 316, the increments of h, by adding the value obtained by squaring the weight f2 _h the current variable W, make this a new variable W. That is, h = h + 1 and W = W + (f2 _h ) ² .

  In step 318, it is determined whether h is larger than the dimension number s2. If h is larger than the dimension number s2, the process proceeds to step 320. If h is less than the dimension number s2, the process proceeds to step 302. Return and repeat the same process as above.

  In step 320, the variable W is calculated by the following equation, and the process proceeds to step 308.

  In step 308, the current square root of W is calculated, and this is used as the final similarity measure Dist (S1, S2).

Thus, in this routine, the Euclidean distance between the weights f1 _k and f2 _h is calculated as a similarity measure. That is, the similarity scale Dist (S1, S2) is lower as the value is higher (the distance is longer). This third similarity measure calculation method is considered to be effective in the case where the semantic vector feature value is generated by the above equations (3) and (4). As described above, in the above expressions (3) and (4), the importance of the meaning content is more strongly reflected than in the expression (2), so this is also strongly reflected in the calculation of the Euclidean distance. This is because the difference is greatly reflected in the difference in similarity.

  Next, a description will be given of a result of obtaining the similarity between the text sentence A and the text sentence B shown below by the text sentence search device 10.

  Text Sentence A and Text Sentence B as described above both have the important content of “The Women's 400-meter Final, Australia's famous general Ferriman has won a gold medal.” .

  First, the results of analyzing the text sentence A and the text sentence B by the word segmentation / part-of-speech adding unit 14 and the syntax / semantic analysis unit 16 are as follows.

  Further, as a result of obtaining the semantic vector feature amount Av (IDa1, fa1, IDa2, fa2,..., IDas, fas) of the text sentence A by using the above equation (5) by the feature amount generation unit 18, The word ai and its weight fai are shown in FIG.

  Similarly, as a result of obtaining the semantic vector feature quantity Bv (IDb1, fb1, IDb2, fb2,..., IDbs, fbs) of the text sentence B using the above equation (5) by the feature quantity generation unit 18, each dimension is obtained. The word bi and its weight fbi are shown in FIG. The constants a, b, and c in the above formula (5) were set to a = 0, b = 0, and c = 0, respectively.

  Then, the similarity calculation unit 22 calculates the similarity Sim (Av, Bv) from the semantic vector feature amount of the text sentence A and the semantic vector feature amount of the text sentence B using the above equation (3). Av, Bv) = 0.769. On the other hand, when the similarity was obtained by the conventional vector space method, Sim (Av, Bv) = 0.611.

  As described above, the similarity calculated by the method according to the present invention is larger than the similarity calculated by the conventional vector space method. This indicates that the method according to the present invention can appropriately calculate the similarity between the text sentence A and the text sentence B having similar semantic contents, and can be said to have higher accuracy than the conventional vector space method. .

  In this embodiment, the case where the present invention is applied when calculating the similarity of a Chinese text sentence has been described. However, the present invention is applied to other languages such as Japanese and English. Applicable.

It is a block diagram which shows schematic structure of a text sentence search apparatus. It is a block diagram which shows an example of the apparatus with which a text sentence search apparatus is applied. It is a figure which shows the analysis result of the text sentence by a word segmentation / part of speech provision part. It is a figure which shows the analysis result of the text sentence by a syntax and a semantic analysis part. It is a figure which shows classification of a case class. It is a figure which shows the relationship between a case class and a semantic chunk weight. It is a figure which shows the relationship between a case class and a center word weight. It is a flowchart which shows the flow of a process of 1st similarity calculation. It is a flowchart which shows the flow of a process of 2nd similarity calculation. It is a flowchart which shows the flow of a process of 3rd similarity calculation. It is a figure which shows the word and weight of each dimension of the semantic vector feature-value of the text sentence A. It is a figure which shows the word and weight of each dimension of the semantic vector feature-value of the text sentence B. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Text sentence search device 12 External storage device 14 Word segmentation / part-of-speech assignment unit 16 Syntax / semantic analysis unit 18 Feature quantity generation unit 20 Statistical data storage unit 22 Similarity calculation unit 24 Database storage unit 26 Storage units 28, 30, 32, 34 memory

Claims (18)

The question text sentence is segmented into words, part of speech is given to the segmented words, word segmentation / part of speech provision means for generating word segmentation / part of speech provision data,
A syntax / semantic analysis means for analyzing a semantic chunk included in the question text sentence, a central word of the semantic chunk, and a case of the semantic chunk, based on the word segmentation / part of speech assignment data,
Appearance frequency data storage means in which appearance frequency data representing the frequency of appearance in the search target text sentence set for each word appearing in the search target text sentence set is stored;
Setting means for setting at least one of a semantic chunk weight and a central word weight of a word included in the semantic chunk based on the case of the semantic chunk;
By calculating the weight of each word included in the question text sentence based on the appearance frequency data for the word included in the question text sentence and at least one of the semantic chunk weight and the central word weight, respectively. Feature quantity generating means for generating a semantic vector feature quantity of the question text sentence;
Semantic vector feature value storage means for storing in advance the semantic vector feature value of each text sentence in the set of text sentences to be searched;
Similarity calculating means for calculating the similarity between the question text sentence and the text sentence to be searched based on the meaning vector feature quantity of the question text sentence and the meaning vector feature quantity of the text sentence to be searched;
A text sentence search device characterized by comprising:   The feature quantity generation means uses a weight of a word included in the question text sentence as a first parameter based on a frequency with which the word appears in the question text sentence, and at least one of the semantic chunk weight and the central word weight. The text sentence search apparatus according to claim 1, wherein the value is calculated by multiplying an addition value obtained by adding a second parameter based on a frequency of occurrence of the word in the text sentence set to be searched.   The feature amount generation means uses a weight of a word included in the question text sentence as a first parameter based on a frequency with which the word appears in the question text sentence, and the word appears in the set of text sentences to be searched. The text sentence search according to claim 1, wherein a multiplication value obtained by multiplying a second parameter based on a frequency of performing the calculation is multiplied by a coefficient including at least one of the semantic chunk weight and the central word weight. apparatus.   The similarity calculation means searches for a matching word among words included in the question text sentence and a word included in the text sentence to be searched, and matches the values obtained by multiplying the weights of the searched words. The text sentence search apparatus according to claim 1, wherein the similarity is calculated by adding all the words that have been added.   The similarity calculation means searches for a matching word among words included in the question text sentence and a word included in the search target text sentence, calculates a distance between weights of the searched words, The text sentence search apparatus according to claim 1, wherein the similarity is calculated by adding the calculated distances to all the matched words. The question text is segmented into words, part of speech is given to the segmented words, and word segmentation / part of speech data is generated,
Based on the word segmentation / part-of-speech assignment data, the semantic chunk included in the question text sentence, the central word of the semantic chunk, and the case of the semantic chunk are analyzed,
Based on the case of the semantic chunk, at least one of a semantic chunk weight and a central word weight of a word included in the semantic chunk is set,
Of the appearance frequency data representing the frequency of appearance in the search target text sentence set for each word appearing in the search target text sentence set, the appearance frequency data for the words included in the question text sentence, Generating a semantic vector feature amount of the question text sentence by calculating a weight of each word included in the question text sentence based on at least one of the semantic chunk weight and the central word weight;
Calculating a similarity between the question text sentence and the text sentence to be searched based on the semantic vector feature quantity of the question text sentence and the semantic vector feature quantity of the text sentence to be searched;
The text sentence search method characterized by this. Cutting the question text sentence into words, adding part of speech to the cut word, and generating word separation / part of speech data;
Analyzing the semantic chunk included in the question text sentence, the central word of the semantic chunk, and the case of the semantic chunk based on the word segmentation / part of speech assignment data;
Setting at least one of a semantic chunk weight and a central word weight of a word included in the semantic chunk based on the case of the semantic chunk;
Of the appearance frequency data representing the frequency of appearance in the search target text sentence set for each word appearing in the search target text sentence set, the appearance frequency data for the words included in the question text sentence, Generating a semantic vector feature quantity of the question text sentence by calculating a weight of each word included in the question text sentence based on at least one of a semantic chunk weight and the central word weight;
Calculating the similarity between the question text sentence and the search target text sentence based on the semantic vector feature quantity of the question text sentence and the semantic vector feature quantity of the text sentence to be searched;
A text sentence search program for causing a computer to execute a process including: A semantic analysis unit that performs language semantic analysis of the question text sentence and sets the semantic importance of each word of the question text sentence;
A feature amount generation unit that generates a feature amount of the question text sentence using an analysis result of the language semantic analysis and a vector space method;
A similarity calculation unit that calculates the similarity between the feature amount of the question text sentence generated by the feature amount generation unit and the feature amount of the search target text sentence;
A search result extraction unit that extracts a text sentence as a search result from a set of search target text sentences based on the calculation result of the similarity;
A text sentence search device comprising:   9. The text sentence retrieval apparatus according to claim 8, wherein the semantic analysis unit performs language semantic analysis based on case grammar.   The text sentence search apparatus according to claim 8 or 9, wherein the similarity calculation unit calculates the similarity using at least one of an inner product, a cosine function, and a Euclidean distance. 11. The text sentence search device according to claim 8, wherein the feature quantity generation unit calculates a feature quantity obtained by using the vector space method by the following equation.
TFi x log (N / ni + c)
Here, TFi is the number of times the word i appears in the text sentence, N is the total number of text sentences in the search target text sentence set, ni is the total number of text sentences including the word i in the search target text sentence set, c is a constant, c ≧ 0.01 is set.   The semantic analysis unit extracts each semantic case in the text sentence and, based on the importance of the extracted semantic case, assigns a different weight to the semantic case, and plays a central role in the text sentence. 12. The method according to claim 9, further comprising at least one of a central word analysis unit that extracts a responsible word and assigns a different weight to the central word based on the importance of the extracted central word. The text sentence search device according to any one of claims. The feature quantity generation unit is obtained using the vector space method according to the following equation, using the semantic case weight obtained by the semantic case analysis unit and the central word weight obtained by the central word analysis unit. 13. The text sentence search apparatus according to claim 12, wherein the feature quantity to be corrected is corrected.
(TFi + Chunk_Weight + Head_Weight) × log (N / ni + c)
Where TFi is the number of times the word i appears in the text sentence, Chunk_Weight is the semantic weight to which the word i belongs, Head_Weight is the central word weight (used when the word i is the central word), and N is the search target text sentence The total number of text sentences in the set, ni is the total number of text sentences including the word i in the search target text sentence set, c is a constant, and c ≧ 0.01. 13. The feature quantity generation unit corrects a feature quantity obtained by using the vector space method according to the following equation, using the semantic case weight obtained by the semantic case analysis unit. Text sentence search device.
(TFi + Chunk_Weight) × log (N / ni + c)
Here, TFi is the number of times the word i appears in the text sentence, Chunk_Weight is the semantic weight to which the word i belongs, N is the total number of text sentences in the search target text sentence set, and ni is the word i in the search target text sentence set. , C is a constant, and c ≧ 0.01 is set. The feature quantity generation unit corrects a feature quantity obtained by using the vector space method according to the following equation, using the central word weight obtained by the central word analysis unit. Text sentence search device.
(TFi + Head_Weight) × log (N / ni + c)
Where TFi is the number of times the word i appears in the text sentence, Head_Weight is the central word weight (used when the word i is the central word), N is the total number of text sentences in the search target text sentence set, and ni is the search target text The total number of text sentences including the word i in the sentence set, c is a constant, and c ≧ 0.01. The feature quantity generation unit is obtained using the vector space method according to the following equation, using the semantic case weight obtained by the semantic case analysis unit and the central word weight obtained by the central word analysis unit. 13. The text sentence search apparatus according to claim 12, wherein the feature quantity to be corrected is corrected.
(TFi × log (N / ni + c)) × (Chunk_Weight + Head_Weight) / 2
Where TFi is the number of times the word i appears in the text sentence, Chunk_Weight is the semantic weight to which the word i belongs, Head_Weight is the central word weight (used when the word i is the central word), and N is the search target text sentence The total number of text sentences in the set, ni is the total number of text sentences including the word i in the search target text sentence set, c is a constant, and c ≧ 0.01. 13. The feature quantity generation unit corrects a feature quantity obtained by using the vector space method according to the following equation, using the semantic case weight obtained by the semantic case analysis unit. Text sentence search device.
TFi x log (N / ni + c) x Chunk_Weight
Here, TFi is the number of times the word i appears in the text sentence, Chunk_Weight is the semantic weight to which the word i belongs, N is the total number of text sentences in the search target text sentence set, and ni is the word i in the search target text sentence set. , C is a constant, and c ≧ 0.01 is set. The feature quantity generation unit corrects a feature quantity obtained by using the vector space method according to the following equation, using the central word weight obtained by the central word analysis unit. Text sentence search device.
TFi × log (N / ni + c) × Head_Weight
Where TFi is the number of times the word i appears in the text sentence, Head_Weight is the central word weight (used when the word i is the central word), N is the total number of text sentences in the search target text sentence set, and ni is the search target text The total number of text sentences including the word i in the sentence set, c is a constant, and c ≧ 0.01.

Images