JP2009223463A - Synonymy determination apparatus, method therefor, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To highly accurately determine synonymy of character string representation accompanied by addition of a typical character string, notation transformation with reading thereof stored, abbreviation, or the like. <P>SOLUTION: With a synonym candidate pair generating section 1, input text is analyzed and processed, synonym candidate representation is extracted from the text based on a result of the analysis and the corresponding analysis result is added thereto, and after normalizing notation and reading of the synonym candidate representation by using an inverse transformation rule 3 and a syllable normalization rule 4, respective synonym candidate representations are combined to generate a synonym candidate pair composed of a pair of the synonym candidate representations. With a synonymy determination section 2, a syllable similarity table 5 and an abbreviation determination model 6 are used to determine whether the synonym candidate representations in the synonym candidate pair is synonymous, and outputs the synonym candidate pair as a synonym pair if they are synonymous. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for determining the synonymity of a character string expression included in a text, and more specifically, a synonym complement expression that is a character string expression as a synonym complement is extracted from the text and a pair of synonym complements is extracted. The present invention relates to a technique for generating a synonym complement pair composed of expressions and determining whether or not synonym complement expressions in the synonym complement pair are synonymous (whether or not they indicate the same information).

  The synonym determination technique is used for detecting spelling errors in a text editor, creating a synonym dictionary by aggregating synonyms obtained by the technique, incorporating the dictionary into a search device, and using it for query expansion, etc. There is a usage.

  “PlayStation”, “PlayStation”, and “PlayStation” appearing in the specification and drawings are registered trademarks, and “Apple Inc.”, “Payeonjung”, “Takuya Kimura”, “Miki Ando”, “ “Megumi Yasu” and “Harry Potter” are names (names) of prominent companies, people, characters, etc., but this application is an invention of language processing, and the characters “(registered trademark)” are inserted, The meaning will change if the notation is changed.

  When expressing any one thing / event using a language, it is possible to select a variety of expressions, so the same information in the text consisting of a character string representing the language has multiple different expressions It exists in (character string representation). In addition, as the amount of text and the number of people who create it increase, the variation of expression for one piece of information increases. Therefore, in order to collect the same information in the text without omission, a synonymity determination method for determining whether or not two character string expressions indicate the same information is necessary. The granularity of the “character string expression indicating the same information” may vary from the document unit to the word unit, but the synonymity determination in the present invention is performed on the basis of nouns and compound noun units.

  There are roughly two conventional methods for performing synonymity determination in the above-mentioned noun and compound noun unit granularity, one is an identification method, and the other is a generation method. The identification method is a method of extracting a synonym complement expression from an arbitrary text to generate a synonym complement pair and determining whether or not they are synonymous (for example, see Non-Patent Document 1). The generation method is a method of generating all expressions that can be considered as synonyms for a certain character string expression. In some cases, the existence is confirmed using Web or the like after generation (see, for example, Non-Patent Document 2).

  Both methods create synonym complement pairs and narrow them down to the correct answer (perform synonym determination). The range of synonyms that can be acquired is the pair generation method, and the accuracy is pair generation and refinement. Depends on both methods. First, if a variety of synonym candidate pairs such as character types, abbreviations, and notation fluctuations are generated, the coverage is widened, but the accuracy is lowered. To improve accuracy, limit the types of synonyms at the time of pair generation, narrow down the collection to more probable synonym complement pairs, or use a variety of narrowing (synonymity judgment) techniques. It is necessary to adopt a technique that can determine with high accuracy even for synonyms.

  As a method of collecting synonym complements in the identification method, use the similarity of notation with the one using meta information such as table structure, tags and symbols, special expressions (XX) XX) Then, there are some which are limited to specific types such as abbreviations and katakana notation and are collected by pattern matching with the features (Non-patent Document 1). In both methods, certain accuracy is ensured by providing constraints such as the type of meta information and notation when generating pairs.

The generation method is a method of generating all expressions that can be considered as synonyms for a certain text expression, and the existence may be confirmed using the Web or the like after generation (for example, Non-Patent Document 2). In the generation method, synonym complement pairs are generated using heuristic rules, probability models, and the like, and the synonym complements that can be generated are limited to specific types such as abbreviations and katakana different notations.
Hiroyuki Sakai, Shigeru Masuyama “Automatic Acquisition of Correspondence between Nouns and Abbreviations from Corpus” Proceedings of the 9th Annual Conference of the Language Processing Society of Japan, 2003, pp. 226-229 Norifumi Murayama, Manabu Okuyama “Abbreviations Automatic Estimation Using Noisy-channel model” Proceedings of the 12th Annual Conference of the Language Processing Society of Japan, 2006, pp. 763-766 Tsunehito Seki, Kazutaka Shimada, Tsutomu Endo “Synonym Extraction Using Table Structure” Proceedings of the 11th Annual Conference of the Language Processing Society of Japan, 2005, C1-6

  In the conventional identification method, the method of collecting synonym complement pairs using meta information cannot be used as synonym complement pairs except for the notation written in a special description method, and it can also be specified by other methods There is a problem that the range that can be acquired is limited to the type because it is specialized in the type (abbreviation and katakana different notation), and the cover range is narrow.

  In addition, in the generation method, as described above, there are cases where a synonym complement pair is generated using a heuristic rule and a probability model, but the former is expensive and has a narrow coverage. In the latter case, there was a problem that the accuracy was extremely low.

  If synonyms are more likely to be synonymous as the notation is similar, such as fluctuations in notation, and if synonyms cannot be measured only by the similarity of the notation, such as abbreviations, then both of these properties are further The reason why the coverage of the conventional method is narrow is that the method has to be specialized for a specific type of synonym.

  However, paying attention to the generation process of synonyms with various types, there are three main types: (a) addition of a regular character string, (b) notation conversion by storing readings, and (c) omission. The cause can be narrowed down. The diversity of synonyms is increased when the three types occur individually and sometimes simultaneously (seven patterns of a, b, c, a + b, a + c, b + c, and a + b + c).

  The character string added in (a) is a fixed expression such as an affix expression such as “Chan” or “Tee” or a specific symbol. Therefore, if the added standard character string is deleted, it becomes the same as the notation before generation. In (b), “PlayStation” is converted to “PlayStation” and “PlayStation”, and the notation is converted while the readings are stored. Therefore, if readings are given to both synonyms, the readings are the same or very similar. In (c), characters are deleted while preserving the order of characters as in “United Nations” as in “United Nations”, so long characters include short ones, and there are some rules regarding which characters are deleted. There is sex.

  Therefore, first, the standard character string added in (a) is deleted to normalize the notation, and the notation converted in (b) is given a normalization, resulting in the notation. Or, if the readings are the same or very similar, it is determined to be synonymous. And if it is determined in the order of determining whether omission of (c) has occurred after absorbing the fluctuations generated in (a) and (b), all synonyms with diversity can be determined. It becomes possible to widen the cover range.

  The present invention has been made in view of the above problems. At the time of collecting synonym supplements, the synonym で is used regardless of the type of synonyms in all the nouns in one text (a group of sentences including at least one sentence). In order to expand the range in which synonyms can be determined to almost all kinds of generated synonym-complement pairs, normalization is performed from both notation and reading.

  When generating synonym complement pairs, synonym complement pairs are generated for all nouns, so synonym complement pairs can be generated without depending on the text description method. In addition, it prevents the generation of a synonym complement pair that exists across the text due to the restriction within one text but has similar notation but is irrelevant.

  At the time of synonym determination, each generated synonym complement pair is normalized from both notation and reading, and the synonym complement pair is conditionally determined and the synonym complement pair is separated by type. Then, by performing the determination, it is possible to apply a synonym determination method suitable for the type of the synonym complement pair and to improve accuracy.

  In addition, since the synonym determination method of the present invention is compatible with almost all types of synonyms (abbreviations and katakana different notations), synonym determination is possible regardless of the synonym complement collection method. It can also be used in combination with existing synonym complement collection techniques.

  When text is input, synonymous complement pairs are generated from the nouns and compound nouns contained in the text, and the notation and reading of each word in the pair are normalized. In the process, the same notation or very similar reading is used. What is determined to be synonymous, and what is not determined to be synonymous and is in an inclusive relationship is determined using a classifier to determine whether it is a synonym.

  According to the present invention, synonymity of diverse character string expressions, including (a) addition of a regular character string, (b) notational conversion by storing readings, or (c) omission, etc. Can be determined with high accuracy.

  Hereinafter, the present invention will be described in detail with reference to the illustrated embodiments.

  FIG. 1 shows an example of an embodiment of a synonym determination device according to the present invention. The synonym determination device includes a synonym candidate pair generation unit 1, a synonym determination unit 2, an inverse conversion rule storage unit 3, and a syllable. It consists of a normalization rule storage unit 4, a syllable similarity table storage unit 5, an omission determination model storage unit 6, an analysis processing result table storage unit 7, and a synonym candidate pair list storage unit 8.

  The synonym complement generating unit 1 is a text input directly from a keyboard or the like (not shown), or read from a storage medium or input from another device or the like via a communication medium, in this case, one text (at least one text) Text data corresponding to a set of sentences including one sentence) is used as a processing unit, and analysis processing such as well-known morphological analysis and proper expression extraction is performed, and synonym complement expressions are extracted from the text based on the analysis results In addition, the corresponding synonym complement expression and the corresponding analysis result are stored in the analysis processing result table storage unit 7, and the inverse conversion rule stored in the inverse conversion rule storage unit 3 for each synonym complement expression Is used to normalize the notation, normalize readings using the syllable normalization rules stored in the syllable normalization rule storage unit 4, and analyze the results. It is stored in the Buru storage unit 7. Thereafter, the synonym complement expressions stored in the analysis processing result table storage unit 7 are combined together to create a synonym complement pair, and the synonym candidate pair list storage unit 8 stores the same.

  The synonym determination unit 2 takes out the synonym complement pair from the synonym candidate pair list storage unit 8 and stores whether or not the synonym complement expressions in the synonym complement pair are synonymous. Analysis result and normalization processing result corresponding to each synonym complement expression in the synonym complement pair stored in the unit 7, syllable similarity table and omission determination stored in the syllable similarity table storage unit 5 The abbreviation determination model stored in the model storage unit 6 is used for determination as described below, and if it is synonymous, the synonym complement pair is output as a synonym pair, which is output as a synonym candidate pair list storage unit The synonym pair list is output in a similar manner for all the synonym candidate pairs stored in 8.

  The determination of whether or not synonyms are made by first determining whether or not the normalized notation is exactly the same, and if it is not determined to be synonymous here, it is determined whether or not the normalized readings are similar, that is, Using the syllable similarity table stored in the syllable similarity table storage unit 5, the similarity of reading after normalization is obtained, and it is determined whether or not the obtained similarity is equal to or greater than a predetermined value. If the notation or reading is in the inclusive relationship, the determination is made based on whether or not the abbreviation model is stored using the abbreviation determination model stored in the abbreviation determination model storage unit 6.

  Inverse conversion rule storage unit 3 deletes rules that delete prefixes and suffixes that are generally inserted when using nouns, and repetitive expressions that are used to create nicknames The reverse conversion rule for normalizing the notation of the synonym complement expression including at least the rule to perform and normalizing the reading in accordance with the normalization of the notation is stored.

  The syllable normalization rule storage unit 4 is applied to the vowel continuation, long sound, and prompting of the reading of the synonym complement expression, so that the unit of reading length (mora and syllable) that differs between Japanese and foreign words, colloquial It stores syllable normalization rules for at least normalizing fluctuations in expression and transliteration.

  The syllable similarity table storage unit 5 uses synonym complement expressions in a synonym complement pair with “a syllable pair with different notation but similar reading” as a key and “distance (similarity)” as a value. A syllable similarity table for obtaining the similarity of readings of

  The abbreviation determination model storage unit 6 includes a model for determining whether two words generated in advance by machine learning have an abbreviation relationship, and the synonym complement expressions in the synonym complement pair are abbreviations. An omission determination model for determining whether or not there is a relationship is stored.

  Hereafter, each part mentioned above is demonstrated in detail. It should be noted that in the following description, the storage contents of the storage units 3 to 8 may be described with the reference numerals of the storage units as they are.

[Synonym complement generation unit 1]
FIG. 2 shows the details of the synonym complement generation unit 1, which includes an analysis processing unit 11, a normalization processing unit 12, and a pair generation unit 13. In the synonym complement generation unit 1, one text is input, and the analysis processing unit 11 performs text analysis processing such as morphological analysis and proper expression extraction, and cuts out the synonym complement expression based on the results. Then, after normalization processing unit 12 normalizes the synonym complement expression, pair generation unit 13 generates a synonym complement pair. In the present embodiment, a case where a specific expression is targeted as a synonymous complement expression will be described as an example.

  The analysis processing unit 11 analyzes the text by using a morphological analysis technique and a specific expression extraction technique established as well-known techniques, and extracts synonym complement expressions. In the morphological analysis, information such as morpheme (notation), reading, part of speech (including proper expression class), etc. is given to the text. At this time, simple normalization such as unification of half-width / full-width characters in the text is also completed. For the reading assignment, the most likely one may be used, or N-best may be used. In addition, if the notation is an unknown word such as an alphabet and the reading is not correctly given only by morphological analysis, the reading is given again. Regarding the provision of readings, a method of correctly estimating readings of unknown words such as alphabets (for example, Japanese Patent Application Laid-Open No. 2001-142877 (name of invention: alphabetic character / Japanese reading correspondence device and method and alphabetic word transliteration device and method) As well as a recording medium on which the processing program is recorded).

  The synonym complement expression is cut out by the unique expression extraction technique, and written to the synonym complement expression column and the analysis result column of the analysis processing result table 7 together with the analysis result (information thereof) such as notation, reading, and part of speech. However, when a synonym complement expression having exactly the same notation already exists in the table 7, writing is not performed so that records do not overlap. At this time, the morpheme segmentation information is stored in notation, reading, and part of speech using, for example, a symbol “/”. An example of the analysis processing result table 7 is shown in FIG. However, at this time, only the synonym complement expression column and the analysis result column in the analysis processing result table 7 are filled, and the others are empty.

  Here, for the sake of explanation, the cut-out target is a specific expression, but nouns and compound nouns may be used as synonym complement expressions using morphological analysis results. As a result, the created analysis processing result table 7 can have records corresponding to the number of different synonym complement expressions existing in one text.

  The flow of processing in the normalization processing unit 12 will be described with reference to FIG. The normalization processing unit 12 normalizes the extracted synonym complement expression from notation and reading. The input is a list of analysis result columns of all records in the analysis processing result table 7 created by the analysis processing unit 11, and the following processing is repeated for each record in the list. When all the records have been processed, the processing in the normalization processing unit 12 ends.

  (Step s12-1) If the notation of the analysis result column in the analysis processing result table 7 is alphabet, uppercase / lowercase letters are unified to uppercase, and the same column is overwritten. Proceed to step s12-2.

  (Step s12-2) The notation rule and the reading rule (details will be described later) of the reverse conversion rule stored in the reverse conversion rule storage unit 3 are applied to the notation and the reading of the analysis result column in the analysis processing result table 7, respectively. The result is written to the analysis processing result table 7. For the export destination, out of the inverse conversion rules, the application result of the notation rule is the notation normalization column, and the application result of the reading rule is the notation + reading normalization column. The result column notation is written as it is to the notation normalization column, and the analysis result column reading is written as it is to the notation + reading normalization column.) Proceed to step s12-3.

  (Step s12-3) The syllable normalization stored in the syllable normalization rule storage unit 4 for the reading of the analysis result column of the analysis processing result table 7 and the reading of the notation + reading normalization column written in step s12-3 Apply the conversion rule (details will be described later), and write the result to the analysis processing result table 7. The result of normalizing the reading of the analysis result column is written to the reading normalization column, and the result of normalizing the reading after reading + normalization is overwritten to the notation + reading normalization column (There are no rules to apply) In this case, the reading of the analysis result column is read as it is and written to the normalization column, and the notation + reading normalization column is left as it is (not overwritten)).

  The synonym complement pair creation unit 13 creates a synonym complement pair by combining all the records in the analysis processing result table 7 that has been processed by the normalization processing unit 12 in a brute force manner. The complementary expression ID (candidate ID) is stored in the synonym candidate pair list storage unit 8 as a synonym complement pair list. An example of the synonym complement pair list is shown in FIG. With respect to the pair creation method, if the cover range is not regarded as important, it is possible to use the present invention thereafter even using a method other than brute force, for example, using meta information.

[Synonymity determination unit 2]
FIG. 6 shows details of the synonymity determination unit 2, which includes a notation similarity determination unit 21, a reading similarity determination unit 22, and an omission determination unit 23. FIG. 7 shows the flow of processing in the synonymity determination unit 2.

  The synonym determination unit 2 receives the analysis processing result table 7 and the synonym complement pair list 8 as input, and the synonym complement pair by the notation similarity determination unit 21, reading similarity determination unit 22, and omission determination unit 23. It is determined whether the word complement expressions are synonymous, and a synonym pair list is output.

  That is, the notation similarity determination unit 21 determines synonymity from the normalized notation (step s21), and the reading similarity determination unit 22 determines synonymity from the normalized reading using the syllable similarity table 5 ( In step s22), the omission determination unit 23 determines whether or not the abbreviation word uses the omission determination model 6 from both normalized notation and reading, and determines synonymity (step s23). The process of steps s21 to s23 is repeated for each record of the synonym complement pair list 8, and the synonym complement expressions in the synonym complement pair are synonymous when determined to be synonymous in any process. (Step s24), and the process proceeds to the next synonym complement pair process. Pairs that have not been determined to be synonymous until the end are not recognized as synonyms (step s25). Information about the notation, reading, part of speech, etc. of the synonym complement expression of the synonym complement pair necessary for the processing is the corresponding in the analysis processing result table 7 using the candidate ID of the synonym complement pair list 8 Browse information. When the processing of all the records in the synonym complement pair list 8 is finished, the synonym complement pair recognized as a synonym is output as a synonym pair list.

  The flow of processing in the notation similarity determination unit 21 will be described with reference to FIG. The input is one record of the synonym complement pair list 8. Here, the determination is made by looking at the normalized notation of each synonym complement expression in the synonym complement pair.

  (Step s21-1) When the notation normalization column notation of each synonym complement expression in the synonym complement pair is exactly the same, go to step s21-2, otherwise the notation similarity determination unit 21 Is finished, and the process proceeds to the reading similarity determination unit 22.

  (Step s21-2) It is determined to be synonymous, and the process in the synonymity determining unit 2 is terminated.

  The flow of processing in the reading similarity determination unit 22 will be described with reference to FIG. The record of the synonym complement pair which was not determined to be synonymous by the notation similarity determination unit 21 is input. Here, the determination is made by looking at each reading of the synonym complement expression in the synonym complement pair. There are three readings for one synonym complement expression: reading of the analysis result column, reading of the reading normalization column, and reading of the notation + reading normalization column. Therefore, the following processing is repeated three times for each, If the synonym is determined to be synonymous in any one of the processes, the synonym is recognized and the process in the synonym determination unit 2 is terminated. If not, the process proceeds to the process in the omission determination unit 23. At this time, the reading of the analysis result column, the reading of the reading normalization column, and the reading of the notation + reading normalization column are simply described as “reading”. In the following processing, the morpheme delimiter information (“/”) is ignored during matching.

  (Step s22-1) If the readings of the synonym complement expressions in the synonym complement pair are exactly the same, go to Step s22-5, otherwise go to Step s22-2.

  (Step s22-2) The number of syllables of reading of each synonym complement expression in the synonym complement pair is counted, and if they are the same, the process proceeds to step s22-3. Otherwise, the process ends (if the number of repetitions of the process in the reading similarity determination unit 22 is 2 or less, the process returns to step s22-1 to move to the process for the next reading of the synonym complement pair. If so, the process in the reading similarity determination unit 22 ends.)

  (Step s22-3) The distance between syllables having the same syllable position but different readings in the reading of each synonym complement expression in the synonym complement pair is used using the syllable similarity table 5 (details will be described later). Ask. When there are many syllables having the same syllable position but different readings, the sum of distances between syllables different between pairs is used. Proceed to step s22-4.

  (Step s22-4) If the sum of the distances is smaller than a preset threshold value, the process proceeds to Step s22-5. In other cases, the process ends (if the number of repetitions of the process in the reading similarity determination unit 22 is 2 or less, the process returns to step s22-1 and proceeds to the process of the next reading of the synonym complement pair. If so, the process in the reading similarity determination unit 22 ends.)

  (Step s22-5) The synonymity determination unit 2 ends the process of determining synonymity.

  The flow of processing in the omission determination unit 23 will be described with reference to FIG. The record of the synonym complement pair which was not determined to be synonymous by the reading similarity determination unit 22 is input. Here, for synonym complement pairs, the following processing is performed for each of the four patterns of notation of analysis result column, notation of notation normalization column, notation of reading normalization column, notation + reading of normalization column. Repeatedly, if it is determined to be synonymous in any one of the processes, it is recognized as a synonym, otherwise it is determined as not a synonym and the process in the synonym determination unit 2 is terminated. At this time, the notation of the analysis result column and the notation normalization column are simply described as “notation”, and the reading normalization column reading and the notation + reading normalization column reading are simply described as “reading”. In the following processing, the morpheme delimiter information (“/”) is ignored during matching.

  (Step s23-1) When notation is targeted, when notation is read, when reading is in an inclusive relationship, it progresses to step s23-2. In other cases, the process ends (if the number of repetitions of the process in the abbreviation determination unit 23 is 3 or less, the process proceeds to the next notation or reading of the synonym complement pair. The processing at 23 is terminated.)

  (Step s23-2) Alignment is performed using a DP matching method (Richard E. Bellman, “Dynamic Programming”, 1957) or the like. Proceed to step s23-3.

  (Step s23-3) The feature for applying to the classifier based on the difference between before and after the omission, with the longer number of characters (number of syllables in the case of reading) of each pair before omission and after the shorter one is omitted. The details are described later. Proceed to step s23-4.

  (Step s23-4) Whether the synonym complement expressions in the synonym complement pairs are abbreviations using the abbreviation determination model 6 which is the model of the feature and classifier extracted in Step s23-3. If it is determined that it is an abbreviation, the process proceeds to step s25-5. In other cases, the process ends (if the number of repetitions of the process in the omission determination unit 23 is 3 or less, the process proceeds to the next notation or reading process of the synonym complement pair. The processing at 23 is terminated.)

  (Step s <b> 23-5) The synonym determination unit 2 terminates the process of determining synonymity.

As the feature used in the above-described (step s23-3), when extraction is performed for each synonym complement expression (analysis result column notation or notation normalization column notation) in the synonym complement pair ,
<Synonym complement expression before and after omission>
-Before omission: morpheme number, number of characters, part of speech, proper expression class, character type-After omission: morpheme number, number of characters, part of speech, proper expression class, character type <Feature of morpheme unit>
・ If the whole morpheme is omitted: part of speech, notation, number of characters, character type, position information (whether it is the beginning or end or middle), whether the first morpheme remains ・ If the whole morpheme remains: part of speech, notation, number of characters, character type , Position information (whether it is the beginning, the end, or the middle), whether the morpheme at the end was omitted <Characteristic features in character units>
・ If omitted in character units: part of speech, notation, character type, position information (whether it is first or last or middle), or whether the first character in the notation is omitted ・ If left in character units: part of speech, notation, character type, Uses position information (whether it is the beginning, end, or middle) and whether the first character remains in the morpheme. However, in addition to those listed here, morphological analysis information, context information, and the like may be used. At this time, information such as part of speech and notation used uses information in the analysis processing result table 7.

  In addition, when extracting features for the reading of each synonym complement expression in the synonym complement pair (reading normalization column reading or notation + reading normalization column reading), the feature examples described above In the above, the reading feature is used for reading, the syllable number is used for the character feature, and the number of syllables when the position information feature is counted by syllable is used. Also, the character type features are unified with “Katakana”.

[Inverse conversion rule (storage unit) 3]
FIG. 11 shows an example of the reverse conversion rule 3, which is used when creating a nickname or a rule for deleting a prefix form such as a prefix or suffix that is generally inserted when using a noun. Heuristic description of rules for deleting repeated expressions. All rules applicable to a set of synonym candidate pairs apply. This rule is a rule that deletes a typical character string that is supposed to be inserted for a more general notation in a synonym complement expression, and corrects the reading along with the deletion of the character string. Therefore, it is different from the rule for creating an abbreviation (in the present invention, the abbreviation is not determined using the inverse transformation rule, but is determined using the classifier in the abbreviation determination unit). Therefore, the rule can be easily written using a regular expression or the like, and the rule is to delete a character string inserted in common with many synonyms. Note that the regular expression in FIG. 11 is described as being written in Perl. Therefore, when using other expressions, the method is followed.

  In the reverse conversion rule 3, a rule that applies to notation and a rule that applies to reading are paired. If a regular expression for notation is not applicable, the paired reading regular expression is not applied. However, on the contrary, if the regular expression for reading cannot be applied or does not exist, only the regular expression for notation may be applied (the reading may not change although the notation does not change, but the notation changes) Even if the reading doesn't change). The reverse conversion rules can be arbitrarily created and registered in addition to those shown in FIG. 11, and for example, “chan” or “Mr.” can be added as a suffix deletion rule.

[Syllable normalization rule (storage unit) 4]
FIG. 12 shows an example of the syllable normalization rule 4, in order to normalize the reading length unit (mora and syllable), colloquial expression, fluctuation in transliteration, etc. Consists of rules that apply to continuous vowels, long tones, and prompting sounds of synonym complement expressions. The rules are applied in the order of rule numbers, and all applicable rules are applied.

  For synonymity determination by notation conversion, the distance between syllables having the same position but different readings is used. At that time, the syllable length is the same between synonyms by notation conversion. However, the unit of reading differs between Japanese words and mora, and foreign words and syllables. Furthermore, when giving Japanese readings to other languages (transliteration), there was a problem that the number of syllables was not the same between foreign words. Examples thereof include the following cases.

・ Japanese, colloquial "Yuko", "Yuko"
When counted by the number of mora, it is 3 mora, but when counted by the number of syllables, it becomes 3 syllables and 2 syllables.

・ Foreign words "spaghetti", "spaghetti", "spaghetti"
If counted by the number of syllables, all three are 4 syllables, but if counted by the number of mora, they become 4 mora, 5 mora, and 6 mora.

・ Japanese language reading "Wimbledon", "Wimbledon" in other languages
When counted by the number of syllables, 7 syllables and 6 syllables, and even when counted by the number of mora, they become 7 mora and 6 mora.

  On the other hand, by using the syllable normalization rule as shown in FIG. 12, the unit of the reading length can be unified so that all can be counted by the number of syllables, and the alignment can be performed in (step s22-3). .

[Syllable similarity table (storage unit) 5]
FIG. 13 (a) shows a procedure for creating a syllable similarity table, and FIG. 13 (b) shows an example of a syllable similarity table. The syllable similarity table 5 is a syllable pair whose key: notation is different but reading is similar. , Value: distance (similarity). As shown in FIG. 13A, this table 5 collects words having the same standard notation and different pronunciation (reading) from the morphological analysis dictionary, and applies the syllable normalization rule 4 in the same manner as the reading normalization processing unit 12. Then, the unit of the reading length is unified into syllables, and alignment is performed when the number of syllables is equal. Then, syllable pairs with the same syllable position but different readings are extracted and counted, and the value obtained by dividing the number of occurrences of syllable pairs by the sum of the number of occurrences of each syllable constituting the syllable pair is the distance between syllables (similarity). To create.

[Omission determination model (storage unit) 6]
The abbreviation determination model 6 is a model for determining whether or not two words are in an abbreviation relationship. The notation and reading of each synonym complement expression in the synonym complement pair to be determined, morphological analysis information It consists of an identification function that uses the alignment information and the like as input, and binary-determines whether or not they are synonymous. As an identification function, for example, V.I. The discriminant function of Support Vector Machine (SVM) described in Vapnik, “The nature of statistical learning theory”, Springer, 1995, is used. Learning and determining with SVM. Here, although SVM was mentioned as a learning algorithm, you may utilize other learning algorithms, such as a decision tree and the maximum entropy method.

  An example of specific processing in the above-described embodiment will be described in detail. Here, “0.9” is used as the threshold value used in the reading similarity determination unit 22 of the synonymity determination unit 2, and SVM is used as the classifier of the omission determination model 4. First, processing performed by the synonym complement generation unit 1 and the synonymity determination unit 2 will be described, and then detailed application examples of the inverse transformation rule 3 and the syllable normalization rule 4, the syllable similarity table 5 and the omission determination model 6 An example of creating will be described.

[I] Processes performed by the synonym complement generation unit 1 and the synonym determination unit 2 [synonym complement generation unit 1]
If the input text to the analysis processing unit 11 is as shown in FIG. 14, synonym complement expressions such as “Apple” and “eva” are extracted. Although the expression “Apple” appears twice in the input text, it is written once in the analysis processing result table 7. The following description will be made on the assumption that the analysis of one text has been completed as shown in FIG.

  The input to the normalization processing unit 12 is a list of synonym complement expression columns of all records in the analysis processing result table 7 shown in FIG. 15, and the following processing is repeated for each record in the list.

  First, processing is started from “Apple Inc.” of ID1.

  (Step s12-1) Since the notation “Apple / Company” in the analysis result column is not an alphabet, the process proceeds directly to Step s12-2 (example of processing here; record ID 2: eva is converted to uppercase EVA and overwritten) To do.)

  (Step s12-2) The notation regular expression and the reading regular expression of the reverse conversion rule 3 (details will be described later) are applied to the analysis result column notation “Apple / Company” and the analysis result column reading “Apple / Sha”, respectively. Then, “Apple” and “Apple” respectively. The result is written to the notation normalization column and the notation + reading normalization column of the analysis processing result table 7, and the process proceeds to step s12-3.

  (Step s12-3) The syllable normalization rule 4 (details will be described later) is applied to the analysis result column reading “Apple / Sha” and the notation + reading normalization column reading “Apple” written in Step s12-2. Apply. The former is “apple / sha” and is written to the reading normalization column. The latter becomes “apple” and is overwritten to the notation + reading normalization column.

  The above processing is repeated in the same manner after ID2 (in FIG. 15, ID14 is displayed). The result is shown in FIG.

Here, in (step s12-2), the reverse conversion rule 3 is applied because of the records in the processing result table 7 shown in FIG. 3 (the morpheme delimiter “/” is not necessary from the following example) Omitted as far as possible).
・ Record ID 1: Apple (→ notation: Apple, notation + reading: Apple)
-Record ID 10: Shokotan (→ Notation: Choco, Notation + Reading: Choco)
・ Record ID 12: Yong (→ Notation: Yon, Notation + Reading: Yon)
Record ID 14: Mikiti (→ notation: Miki, notation + reading: Miki)
There are four.

In addition, the syllable normalization rule 4 is applied in (step s12-3) among the records in the processing result table 7 shown in FIG.
・ Record ID1: Apple Inc. (→ Reading: Apulcia, Notation + Reading: Apple)
・ Record ID2: Shoko (→ reading: sho, notation + reading: shoko)
・ Record ID4: Hakkeijima Sea Paradise (→ Reading: Hakeijimashi Paradise, Notation + Reading: Hakeijimashi Paradise)
・ Record ID8: Apple (→ reading: apple, notation + reading: apple)
-Record ID 11: Sea Para (→ Reading: Shipara, Notation + Reading: Shipara)
These are five.

  The synonym complement pair creation unit 13 creates a synonym complement pair for all the records in the analysis processing result table 7 created in the normalization processing unit 12 shown in FIG. 3, and the synonym complement pair shown in FIG. Listing 8 is output.

[Synonymity determination unit 2]
The input to the synonym determination unit 2 is the synonym complement pair list 8 shown in FIG. 5 and the analysis processing result table 7 shown in FIG. Hereinafter, the processing from the notation similarity determination unit 21 to the omission determination unit 23 is repeated for each record in the list 8 of FIG. 5, and the synonym complement pair is synonymous when it is determined to be synonymous in some process. Acknowledge and move on to processing of the next synonym complement pair. Pairs that have not been determined to be synonymous until the end are not recognized as synonyms. When all the records have been processed, the synonym complement pairs recognized as synonyms are output as a synonym pair list.

  First, processing is started from record ID 1 (“Apple”, “EVA”) in the synonym pair list 8.

  When the notation similarity determination unit 21 refers to the analysis processing result table 7 from the supplementary IDs 1 and 2 corresponding to the record ID 1 of the synonym pair list 8, the notation normalization column notation is “Apple” and “EVA”. Therefore, the processing in the notation similarity determination unit 21 is terminated, and the process proceeds to the reading similarity determination unit 22.

  The reading similarity determination unit 22 refers to the reading of the processing result table 7 from the supplementary IDs 1 and 2 corresponding to the record ID 1 of the synonym pair list 8 and performs the determination. The reading of this synonym complement pair is the reading of the analysis result column: “Applesha”, “Eva”, the reading normalization column: “Aprsha”, Eve ”, the notation + reading normalization column reading:“ It will be “apple” and “eva”. The following process is repeated three times for each.

  First, the analysis result column readings: “Applesha” and “Eva” are processed.

  (Step s22-1) Since the reading of the synonym complement pair is different, the process proceeds to Step s22-2.

  (Step s22-2) When the number of syllables to be read is counted, “Applesha” is different in four syllables and “Eva” is different in two syllables.

  The process returns to step s22-1 and repeats the processing “Apulcia”, “Eve”, “Apple”, and “Eve”, but the number of syllables differs from each other and is not synonymous.

  The omission determination unit 23 determines the following four types of information regarding the pair of record ID 1 in the synonym pair list 8 with reference to the analysis processing table 7. Analysis result column notation: “Apple”, “EVA”, notation normalization column notation: “Apple”, “EVA”, reading normalization column reading: “Aprsha”, “Eva”, notation + reading normalization Reading of the conversion column: There are 4 patterns of “apple” and “eva”. The following processing is repeated for each.

  First, the processing starts from the notation of the analysis result column: “Apple”, “EVA”.

  (Step s23-1) Since the notation is not in an inclusive relationship, the process proceeds to the next iteration. Subsequently, the processing of “Apple”, “EVA”, “Apulcia”, “Eva”, “Apple”, “Eva” is performed, but since all are not in an inclusion relationship, the processing in the omission determination unit 23 is terminated.

  Since the record ID 1 in the synonym pair list 8 has never been determined to be synonymous during the process of the synonym determination unit 2, the record ID 1 is not recognized as a synonym pair and the process proceeds to the next record, that is, ID2.

  Here, the record ID 2 (“Apple Inc.”, “Hakkeijima Sea Paradise”) and the record ID 3 (“Apple Inc.”, “Shoko”) are not synonymous with the record ID 1, so in the list 8 of FIG. Of the record ID7, ID22, ID45, ID63, ID78, ID90, and ID121 that are finally determined to be synonymous, the synonymity determination unit 2 narrows down to representative patterns ID7, ID22, ID45, ID63, and ID121. Processing will be described.

★ Record ID 7 ("Apple company", "Apple")
[Notation similarity determination unit 21]
(Step s21-1) When the analysis processing result table 7 is referred to from the supplementary IDs 1 and 8 of the synonym candidate pair list 8, the notation of the notation normalization column is “Apple” and “Apple”, which are exactly the same. Therefore, the process proceeds to step s21-2.

  (Step s21-2) It is determined to be synonymous, and the process in the synonymity determining unit 2 is terminated.

★ Record ID 22 ("EVA", "EVA")
[Notation similarity determination unit 21]
(Step s21-1) When the analysis processing result table 7 is referred to from the complement IDs 2 and 9 of the synonym complement pair list 8, the notation similarity determination is made because the notation of the notation normalization column differs between “EVA” and “EVA”. The process of the unit 21 is terminated, and the process proceeds to the reading similarity determination unit 22.

[Reading similarity determination unit 22]
Refer to the analysis processing result table 7 from the complements ID2 and ID9 of the synonym supplementary pair list 8. Reading the analysis result column: “Eva”, “Eva”
・ Reading of normalization column: “Eva”, “Eva”
・ Notation + Reading normalization column reading: "Eva", "Eva"
And repeat the process.

  First, the analysis result column is read: “eva” and “eva” are processed.

  (Step s22-1) Since the reading is different, the process proceeds to Step s22-2.

  (Step s22-2) The number of syllables to be read is counted, and since both are two syllables, the process proceeds to Step s22-3.

  (Step s22-3) The syllables having the same syllable position but different readings are “B” and “V”, and the distance is obtained as 0.87 from the syllable similarity table 4 (FIG. 13B: details will be described later). . Since this synonym complement pair has only one pair with the same syllable position and different readings, the distance between the pairs is 0.87. Proceed to step s22-4.

  (Step s22-4) Since the total sum of the distances is 0.87 and is smaller than the preset threshold value 0.9, the process proceeds to Step s22-5.

  (Step s22-5) The synonymity determination unit 2 ends the process of determining synonymity.

★ Record ID 45 (“Shoko”, “Shokotan”)
[Notation similarity determination unit 21]
(Step s21-1) When the analysis processing result table 7 is referred to from the supplementary IDs 3 and 10 of the synonym candidate pair list 8, the notation similarity determination unit is different because the notation of the notation normalization column differs between “Shoko” and “Shoko”. The process of No. 21 is terminated, and the process proceeds to the reading similarity determination unit 22.

[Reading similarity determination unit 22]
Refer to the analysis processing result table 7 from the supplementary IDs 3 and 10 of the synonym candidate pair list 8. Reading of the analysis result column: “shoko”, “shokotan”
・ Reading of reading normalization column: “Shoko”, “Shokotan”
・ Notation + Reading normalization column readings: “Shoko”, “Shoko”
And repeat the process.

  First, the analysis result column is read: “shoko” and “shokotan” are processed.

  (Step s22-1) Since the reading is different, the process proceeds to Step s22-2.

  (Step s22-2) The number of syllables to be read is counted, and “Shoko” is different in 3 syllables, and “Shochotan” is different in 4 syllables.

  Next, the reading normalization column readings: “chocolate” and “chocotan” are processed.

  (Step s22-1) Since the reading is different, the process proceeds to Step s22-2.

  (Step s22-2) The number of reading syllables is counted. Since “chocolate” is different in two syllables and “chocotan” is different in four syllables, the process proceeds to the next repetition process.

  Next, the notation + reading normalization column readings: “chocolate” and “chocolate” are processed.

  (Step s22-1) Since readings are equal, the flow proceeds to Step s22-5.

  (Step s22-5) The synonymity determination unit 2 ends the process of determining synonymity.

★ Record ID 63 ("Hakkeijima Sea Paradise", "Sea Para")
[Notation similarity determination unit 21]
(Step s21-1) When referring to the analysis processing result table 7 from the complement IDs 4 and 11 of the synonym complement pair list 8, the notation of the notation normalization column is different between “Hakkeijima Sea Paradise” and “Sea Para”. Therefore, the processing of the notation similarity determination unit 21 is terminated, and the process proceeds to the reading similarity determination unit 22.

[Reading similarity determination unit 22]
Refer to the analysis processing result table 7 from the complement IDs 4 and 11 of the synonym compensation pair list 8. Reading of the analysis result column: “Hackage Masi Paradise”, “Sea Para”
-Reading normalization column readings: "Hakeijima paradise", "Shipara"
・ Notation + Reading normalization column readings: “Hakeijimashi Paradise”, “Shipara”
And repeat the process.

  As a result of repeating the processing in the same manner as described above for each of the three patterns, the number of syllables differs between the pairs and is not determined to be synonymous, so the processing proceeds to the omission determination unit 23.

[Omission determination unit 23]
Refer to the analysis processing result table 7 from the supplementary IDs 3 and 10 of the synonym candidate pair list 8 ・ Analysis result column notation: “Hakkeijima Sea Paradise”, “Sea Para”
・ Notation of notation normalization column: "Hakkeijima Sea Paradise", "Sea Para"
-Readings of columns after reading normalization: "Hakeijimashi Paradise", "Shipara"
・ Notation + Reading normalization column readings: "Hakeijima paradise", "Shipara"
Ask for.

  The following processing is repeated for each of the four patterns.

  First, the notation of analysis result columns: “Hakkeijima Sea Paradise” and “Sea Para” are processed.

  (Step s23-1) Since the notation is in an inclusive relationship, the process proceeds to Step s23-2.

  (Step s23-2) When alignment is performed by the DP matching method, the result is as shown in the upper left of FIG. Proceed to step s23-3.

  (Step s23-3) Feature extraction is performed as shown in step s23-3 in FIG. The expression before deletion is “Hakkeijima Sea Paradise”, the expression after deletion is “Seapara”, the remaining morpheme is “Sea”, the deleted morpheme is “Hakkeijima”, the deleted characters are “Da”, “ “I” and “su” and the remaining characters are “pa” and “la”. With respect to the six, the features such as the number of morphemes, the number of characters, the part of speech, etc. are extracted as shown on the right side of FIG.

  (Step s23-4) As a result of determining whether or not the synonym complement pair is in an abbreviation relationship using the approximate determination model 6 that is a classifier model, the process proceeds to step s23-5 because it is synonymous.

  (Step s <b> 23-5) The synonym determination unit 2 terminates the process of determining synonymity.

★ Record ID 121 ("Miki Ando", "Mikiti")
[Notation similarity determination unit 21]
(Step s21-1) When the analysis processing result table 7 is referred to from the supplementary IDs 7 and 14 in the synonym candidate pair list 8, the notation of the notation normalization column is “Miki Ando” and “Miki”, and the notations are different from each other. The process of the notation similarity determination unit 21 ends, and the process proceeds to the reading similarity determination unit 22.

[Reading similarity determination unit 22]
Refer to the analysis processing result table 7 from the supplementary IDs 7 and 14 of the synonym candidate pair list 8. Reading of the analysis result column: “Ando Miki”, “Mikiti”
-Reading normalization column readings: "Andumiki", "Mikiti"
・ Notation + Reading normalization column reading: “Ando Miki”, “Miki”
And repeat the process.

  As a result of repeating the processing in the same manner as described above for each of the three patterns, the number of syllables differs between the pairs and is not determined to be synonymous, so the processing proceeds to the omission determination unit 23.

[Omission determination unit 23]
Refer to the analysis processing result table 7 from the supplementary IDs 7 and 14 of the synonym candidate pair list 8. The notation of the analysis result column: “Miki Ando”, “Mikiti”
・ Notation of notation normalization columns: “Miki Ando”, “Miki”
-Readings of columns after reading normalization: "Andumiki", "Mikity"
・ Notation + Reading normalization column readings: "Ando Miki", "Miki"
Ask for.

  The following processing is repeated for each of the four patterns.

  First, the notation of the analysis result column: “Miki Ando” and “Mikiti” are processed.

  (Step s23-1) Since the notation is not in an inclusive relationship, the process proceeds to the next iterative process.

  The following “Miki Ando”, “Miki”, “Ando Miki”, and “Mikiti” are sequentially processed, but since they are not in an inclusive relationship, they are not determined to be synonymous. The final “Andumiki” and “Miki” processes are as follows.

  (Step s23-1) Since reading is in an inclusive relationship, the process proceeds to Step s23-2.

  Thereafter, (step s23-2) to (step s23-5) are performed in the same manner as in the case of the record ID 63 (“Hakkeijima Sea Paradise”, “Sea Para”).

  After completing the synonym determination of all the synonym complements in the synonym complement pair list 8 as described above, the record ID7, ID22, ID45, ID63, ID78, ID90, ID121 determined to be synonymous are synonymous pairs. Output as.

[II] Detailed application example of reverse conversion rule 3 and syllable normalization rule 4, example of creation of syllable similarity table 5 and omission determination model 6 With respect to reverse conversion rule 3, the prefixes shown in FIG. , Deletion of suffix, deletion of reading kana, deletion of repeated expression, and deletion of ellipsis.

  When deleting a prefix, for example, when applied to a target “notation: o / kichi, reading: o / kichi”, the prefix “o” is deleted and becomes “notation: kichi, reading: kichi”.

  In the deletion of the suffix, for example, when applied to a target “notation: Apple / Company, reading: Apple / Sha”, the suffix “Company” is deleted to become “notation: Apple, reading: Apple”.

  In the case of deletion of a reading kana, for example, when applied to a target of “notation: cheap / (/ easy /) / Megumi, reading: Yasu / Yasu / Megumi”, it becomes “notation: Yasu / Megumi, reading: Yasu / Megumi”.

  When deleting repeated expressions, for example, when applied to an object “notation: Kyung Kyung, reading: Kyung Kyung”, it becomes “notation: Kyung, reading: Kyung”.

  When the ellipsis is deleted, it is applied to the object "notation: Harry Potter 3 // Fire / No /-, reading: Harry Potter Sun // Hoono / No", and "Notation: Harry Potter 3 // Fire / : Harry Potter Sun // Hoono / No ”.

  Regarding the syllable normalization rule 4, four examples of “Yuko”, “Wimbledon”, “Jerusalem”, and “Swing” shown in FIG. 12 will be described in this embodiment.

★ "Yuko"
Since the same vowel continues in “Yu” and “U”, Rule 1 is applied, and the continuous vowel part becomes longer and becomes “Yuko”. Next, rule 5 is applied to the part that has become a long sound according to rule 1, resulting in “Yuko”.

★ "Wimbledon"
Since the vowel “U” and the roaring “i” of another type of vowel are continuous, Rule 2 is applied, resulting in “Wimbledon”.

★ "Jerusalem"
Since the vowel “I” and the roaring “e” of another kind of vowel are continuous, Rule 2 is applied, resulting in “Jerusalem”. Furthermore, since the vowels that meet rule 2 and the stuttering of the vowels match the conditions of rule 3, the vowel “i” is deleted and becomes “Jerusalem”.

★ "Swing"
Since the vowel “U” and the roar “i” of another kind of vowel are continuous, Rule 2 is applied, resulting in “Swing”. Further, as a result of applying the rule 2, the vowels are continuous with “Uui”, and the condition of the rule 4 is satisfied. Therefore, one continuous vowel “U” of the same type is deleted and becomes “swing”.

  A method of creating the syllable similarity table 5 will be described with an example using “artist” and its different notation shown in FIG. The morphological analysis dictionary supports different notations and notation fluctuations, and uses a dictionary having information on the reading and standard notation. First, the words “Artist”, “Artist”, and “Artist” with the same standard notation and different pronunciation are collected from the morphological analysis dictionary. Next, the syllable normalization rule 4 is applied to each expression, and alignment is performed with the syllable number matching. As a result, three pairs “te and tee”, “chi and ti”, “te and ti”, and “te and ti” having the same syllable position and different readings are obtained as “Atist”, “Attest”, and “Atist”. Similarly, pairs of different notation collection, normalization, and reading are collected from the morphological analysis dictionary using the standard notation as a key. When all the dictionaries have been processed, the number of pairs with the same syllable position and different readings and the number of syllables that appear are counted for each type. Then, a table is created by calculating the distance using the formula of FIG.

  As a method for creating the omission determination model 6, an example in which SVM is used as an identification function will be described. If possible, it is desirable that the text data used for learning is acquired from the same domain as the input text used during actual operation, and the analysis processing is performed using the same analyzer as used during actual operation. After the synonym candidate expression pair is created for the learning text by the same method as in the first embodiment, only those whose notation is in an inclusive relationship are extracted. Each entry is correctly answered whether it is synonymous or not. Then, the same features as the omission determination unit 23 are extracted, and the omission determination model is created by learning the parameters of the discrimination function. In this case, if the synonym candidate expression pair created by the synonym candidate pair generation unit 1 is learned using the data of the reading column, the notation normalization column, the reading normalization column, and the notation + reading normalization column, An omission determination model corresponding to the column of can be created.

  Note that the description of the inverse conversion rule storage unit, syllable normalization rule storage unit, syllable similarity table storage unit, omission determination model storage unit, analysis processing result table storage unit, synonym candidate pair list storage unit in the embodiment This is an expression based on the functional difference of what kind of data is stored, and does not mean that an individual storage unit (storage device) is necessary in terms of hardware. In addition, the present invention can be applied to a program for realizing the functions shown in the configuration diagrams of FIGS. 1, 2, and 6 or the flowcharts of FIGS. 4, 7 to 10 via a medium or a communication line in a known computer. It can also be realized by installing a program having the indicated procedure.

The schematic block diagram which shows an example of embodiment of the synonym determination apparatus of this invention Block diagram showing details of synonym complement generation unit Explanatory drawing which shows an example of an analysis process result table Process flow in normalization processing unit Explanatory drawing which shows an example of a synonym complement pair list Block diagram showing details of synonymity determination unit Flow chart of processing in synonymity determination unit Flow chart of processing in the notation similarity determination unit Flow chart of processing in the reading similarity determination unit Flow chart of processing in the omission determination unit Explanatory drawing which shows an example of a reverse conversion rule Explanatory drawing which shows an example of a syllable normalization rule Explanatory drawing which shows the preparation procedure of the syllable similarity table and an example thereof Explanatory drawing showing an example of input text Explanatory drawing which shows an example of the analysis process result table at the time of an analysis process end Explanatory drawing which shows an example of the feature extraction process in an omission determination part

Explanation of symbols

  1: synonym candidate pair generation unit, 2: synonymity determination unit, 3: inverse conversion rule storage unit, 4: syllable normalization rule storage unit, 5: syllable similarity table storage unit, 6: omission determination model storage unit, 7: analysis processing result table storage unit, 8: synonym candidate pair list storage unit, 11: analysis processing unit, 12: normalization processing unit, 13: pair generation unit, 21: notation similarity determination unit, 22: reading similarity determination Part, 23: omission determination part.

Claims (10)

A synonym complement expression that is a string expression as a synonym candidate is extracted from the text to generate a synonym complement pair, and whether or not the synonym complement expressions in the synonym complement pair are synonymous with each other. A synonymity judging device for judging,
An inverse conversion rule storage unit for storing an inverse conversion rule for normalizing the notation of the synonym complement expression;
A syllable normalization rule storage unit for storing a syllable normalization rule for normalizing the reading of the synonym complement expression;
A syllable similarity table storage unit for storing a syllable similarity table for obtaining a similarity of reading between synonym complement expressions in the synonym complement pair;
An abbreviation determination model storage unit that stores an abbreviation determination model for determining whether or not the synonym complement expressions in the synonym complement pair are in an abbreviation relationship;
The input text is analyzed, the synonym complement expression is extracted from the text based on the analysis result, the corresponding analysis result is given, and the reverse conversion rule and syllable normalization stored in the reverse conversion rule storage unit After synthesizing the synonym complement expression using the syllable normalization rule stored in the generalization rule storage unit and normalizing the reading, the synonym complement expression is combined and a pair of synonym complement expressions A synonym complement pair generating means for generating the following synonym complement pair;
Whether the synonym complement expressions in the synonym complement pair are synonymous using the syllable similarity table stored in the syllable similarity table storage unit and the abbreviation determination model stored in the abbreviation determination model storage unit. Synonymity determining means for determining and outputting the synonym complement pair as a synonym pair if it is synonymous. The synonymity determination device according to claim 1,
Normalize the notation, including at least rules for deleting affix forms that are generally inserted when using nouns, rules for deleting repeated expressions used when creating nicknames, and combine normalization of the notation Using the inverse transformation rule to normalize readings
The synonym complement pair generation means applies at least the affix form and the repeated expression to the notation in the analysis result of the synonym complement expression, deletes at least the affix form and the repeated expression, and normalizes to correct the reading along with the deletion A synonymity determination device characterized by performing processing. The synonymity determination device according to claim 1,
Using syllable normalization rules that at least normalize the reading length unit, colloquial expression, and transliteration fluctuation between Japanese and foreign words by applying it to continuous vowels, long sounds, and prompting sounds,
The synonym complement pair generation means applies the syllable normalization rule to the vowel continuation, long sound, and prompting of the notation that is normalized by the reading and inverse conversion rules in the analysis result of the synonym complement expression. The synonymity determination apparatus characterized by performing the normalization process which makes it. The synonymity determination device according to claim 1,
A syllable similarity table for finding the similarity of readings between synonym complement expressions in a synonym complement pair, using "syllabic pairs with different notations but similar readings" as keys and "distance" as a value Use
The synonym determination means uses the syllable similarity table to calculate the sum of distances between syllables having the same number of syllables, the same syllable position, and different readings in the reading of each synonym complement expression in the synonym candidate pair. The synonymity determination device characterized in that the synonymity is determined if the sum of the distances is smaller than a preset threshold value. A synonym complement expression that is a string expression as a synonym candidate is extracted from the text to generate a synonym complement pair, and whether or not the synonym complement expressions in the synonym complement pair are synonymous with each other. A synonymity judging method for judging,
An inverse conversion rule storage unit for storing an inverse conversion rule for normalizing the notation of the synonym complement expression;
A syllable normalization rule storage unit for storing a syllable normalization rule for normalizing the reading of the synonym complement expression;
A syllable similarity table storage unit for storing a syllable similarity table for obtaining a similarity of reading between synonym complement expressions in the synonym complement pair;
Using an abbreviation determination model storage unit that stores an abbreviation determination model for determining whether or not the synonym complement expressions in the synonym complement pair are in an abbreviation relationship,
The synonym complement pair generating means analyzes the input text, extracts a synonym complement expression from the text based on the analysis result, and gives a corresponding analysis result, and stores it in the inverse conversion rule storage unit. The synonym complement expression is combined after normalizing the notation and reading of the synonym complement expression using the stored inverse conversion rule and the syllable normalization rule stored in the syllable normalization rule storage unit Generating a synonym complement pair consisting of a pair of synonym complement expressions;
The synonym determination means uses the syllable similarity table stored in the syllable similarity table storage unit and the abbreviation determination model stored in the abbreviation determination model storage unit to synonym complement expressions in the synonym complement pair A synonym determination method comprising: determining whether or not is synonymous and outputting the synonym complement pair as a synonym pair if synonymous. In the synonymity determination method according to claim 5,
Normalize the notation, including at least rules for deleting affix forms that are generally inserted when using nouns, rules for deleting repeated expressions used when creating nicknames, and combine normalization of the notation Using the inverse transformation rule to normalize readings
The synonym complement pair generation process is
Applying the inverse transformation rule to the notation in the analysis result of the synonym complement expression to delete at least the affix form and the repeated expression, and performing a normalization process to correct the reading in accordance with the deletion The synonymity judging method. In the synonymity determination method according to claim 5,
Using syllable normalization rules that at least normalize the reading length unit, colloquial expression, and transliteration fluctuation between Japanese and foreign words by applying it to continuous vowels, long sounds, and prompting sounds,
The synonym complement pair generation process is
A step of performing a normalization process of normalizing the syllable normalization rules by applying the syllable normalization rules to the vowel continuation, the long sound, and the prompt sound of the notation that is normalized by the reading and reverse conversion rules in the analysis result of the synonym complement expression A synonymity judging method characterized by including. In the synonymity determination method according to claim 5,
A syllable similarity table for finding the similarity of readings between synonym complement expressions in a synonym complement pair, with "syllabic pairs with different notations but similar readings" as keys and "distance" as a value Use
The synonym determination process
The sum of distances between syllables having the same syllable position but different readings in the reading of each synonym complement expression in the synonym candidate pair is obtained using the syllable similarity table, and the sum of the distances is a preset threshold. The synonymity determination method characterized by including the process of determining that it is synonymous if smaller.   The program for functioning a computer as each means of the synonym determination apparatus in any one of Claims 1 thru | or 4.   A computer-readable recording medium on which the program according to claim 9 is recorded.

Images