JP2015064671A - Sentence normalization system, sentence normalization method, and sentence normalization program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To robustly normalize a sentence even when the sentence is hardly normalized robustly by syntax analysis.SOLUTION: A sentence normalization system 10 includes: an input part 11 which inputs a sentence; a morphological analysis part 12 which divides the sentence into a word string and estimates a part of speech of each divided word; a separation part 13 separates the divided word string into a content part including a content of the sentence and a sentence end based on the estimated part of speech of each word; a content word string extraction part 14 which extracts a content word string which is content information showing the content of the sentence from an independent word included in the content part; a semantic label string extraction part 15 which extracts a semantic label string which is function information showing a functional expression of the sentence from the sentence end; and a symbol string combining part 16 which combines the content word string and the semantic label string, and outputs it as a normalized expression of the sentence.

Description

  The present invention relates to a sentence normalization system, a sentence normalization method, and a sentence normalization program that normalize sentences described in a natural language or the like.

  Conventionally, a system that interprets the meaning of a sentence described in a natural language and performs analysis and response based on the interpreted meaning is known. For example, Patent Document 1 discloses a method in which input text is decomposed into word strings, a syntax analysis process is performed on a natural language to obtain a syntax tree, and semantic information is further extracted from the syntax tree to perform information retrieval. It is disclosed.

Japanese Patent No. 4738523

  Extraction of semantic information from sentences written in natural language is to normalize sentences to symbol strings that correspond one-to-one with the meaning of sentences by removing fluctuations on the surface that have little effect on the meaning of sentences. Equivalent to. In natural language, various expressions can be used to write almost the same meaning. Considering various paraphrasing such as active voice, passive voice, synonym, ending expression, etc., there are a huge number of variations of sentence fluctuations that can have the same meaning. A practical natural language processing system is required to robustly accept and process these many fluctuations. Therefore, the technology that removes the fluctuations described above and normalizes the sentence is very important in many applications that accept natural language such as information retrieval, dialogue, and translation.

  In the prior art disclosed in Patent Document 1 and the like, this sentence normalization is performed on the basis of syntax analysis processing. By syntactic analysis, the constituent elements of the sentence such as the subject, verb, and object can be clarified and analyzed.

  However, if the sentence is colloquial and has grammatical disruption or contains errors due to transmission means in speech recognition or the like, syntax analysis does not always work robustly.

  The present invention has been made in view of the above problems, and a sentence normalization system and sentence normalization that can robustly normalize a sentence that is difficult to normalize robustly by syntax analysis. An object of the present invention is to provide a normalization method and a sentence normalization program.

  In order to achieve the above object, a sentence normalization system according to the present invention divides a sentence input by the input means into a word string and estimates the part of speech of each divided word. Morphological analysis means, separation means for separating the word string divided by the morphological analysis means into a content part including the content of the sentence and a sentence end based on the estimated part of speech of each word, and separation means from the sentence Content information extracting means for extracting the content information indicating the content of the sentence from the independent words included in the separated content part, and function information indicating the functional expression of the sentence from the sentence end separated from the sentence by the separating means. Function information extracting means for extracting, content information extracted by the content information extracting means, and output means for outputting the function information extracted by the function information extracting means as a normalized expression of the sentence , Comprising a.

  In the sentence normalization system according to the present invention, a word string obtained from a sentence is separated into a content part and a sentence end, content information and function information are extracted from each, and these are output as a normalized expression of the sentence Is done. That is, in the sentence normalization system according to the present invention, normalization is performed from a word string that operates more fundamentally and robustly, without using a syntactic analysis that is likely to be affected by grammatical disruptions that often appear in colloquial sentences. Thereby, according to the sentence normalization system which concerns on this invention, it can normalize robustly also with respect to the sentence which is hard to normalize robustly by syntax analysis.

  The separating means may use a section from the word string up to the rearmost independent word as a content part and a section after the section as a sentence end. According to this configuration, the sentence can be appropriately and reliably separated into the content part and the sentence end, and as a result, normalization can be performed appropriately.

  The content information extraction means determines whether or not the word extracted as the content information includes a question word, or whether or not the last word among the words extracted as the content information is a noun corresponding to the question sentence It is also possible to determine whether the sentence is a question sentence and to determine the type of the question sentence based on the question word or the last word. According to this configuration, it is possible to appropriately normalize the question sentence.

  The output means may set the content information according to the order in which the independent words appear in the sentence, and sort and output the function information. It is considered that the meaning of the order with respect to the meaning is large for the content word sequence, and the meaning of the order with respect to the meaning of the meaning label sequence is small. Therefore, according to this configuration, appropriate normalization can be performed. In particular, since the semantic label string is sorted, normalization that enables comparison between sentences can be performed by ignoring the order of the semantic label string.

  Of the independent words included in the word string divided by the morphological analysis means, a preset independent word may be regarded as an attached word. Further, the target word may be regarded as an attached word according to the word immediately before the target word corresponding to the preset independent word, the part of speech of the previous word, or the utilization form of the previous word. According to these configurations, it is possible to appropriately handle words that should be attached words in normalization, and as a result, normalization can be performed appropriately.

  By the way, the present invention can be described as an invention of a sentence normalization system as described above, and can also be described as an invention of a sentence normalization method and a sentence normalization program as follows. This is substantially the same invention only in different categories and the like, and has the same operations and effects.

  That is, the sentence normalization method according to the present invention is a sentence normalization method that is an operation method of a sentence normalization system, and includes an input step for inputting a sentence, and a sentence input in the input step is divided into word strings. A morpheme analysis step for estimating the part of speech of each word divided together with the word string divided in the morpheme analysis step into a content part including the content of the sentence and a sentence end based on the estimated part of speech of each word From the independent step included in the content part separated from the sentence in the separation step, the content information extraction step for extracting the content information indicating the content of the sentence, and the end of the sentence separated from the sentence in the separation step, A function information extraction step for extracting function information indicating the function expression of the sentence, the content information extracted in the content information extraction step, and the function information extraction means; Including an output step of outputting the feature information extracted as a normalized representation of the statement I.

  The sentence normalization program according to the present invention includes a computer, an input means for inputting a sentence, a morpheme analyzing means for dividing a sentence input by the input means into a word string and estimating a part of speech of each divided word; , Separation means for separating the word string divided by the morphological analysis means into a content part including the content of the sentence and a sentence end based on the estimated part of speech of each word, and contents separated from the sentence by the separation means Content information extracting means for extracting content information indicating the content of the sentence from independent words included in the section, and functional information indicating function information indicating the functional expression of the sentence from the sentence end separated from the sentence by the separating means An output means for outputting the content information extracted by the extraction means, the content information extraction means, and the function information extracted by the function information extraction means as a normalized expression of a sentence; To to function.

  In the present invention, normalization is performed from a word sequence that operates more fundamentally and robustly, without using parsing that is likely to be affected by grammatical disruptions that often appear in colloquial sentences. Thus, according to the present invention, it is possible to perform normalization robustly even for sentences that are difficult to normalize robustly by syntax analysis.

It is a figure which shows the function structure of the sentence normalization system which concerns on embodiment of this invention. It is a figure which shows the example of isolation | separation of the sentence into a content part and a sentence end. It is the correspondence table which matched the notation of a word, and an identifier. 6 is a correspondence table in which notations and semantic labels are associated with each other. It is a table | surface which shows an input sentence and the normalization symbol string corresponding to the said input sentence. It is a figure which shows the hardware constitutions of the sentence normalization system which concerns on embodiment of this invention. It is a flowchart which shows the sentence normalization method which is a process performed with the sentence normalization system which concerns on embodiment of this invention. It is a figure which shows the structure of the sentence normalization program which concerns on embodiment of this invention with a recording medium.

  Hereinafter, embodiments of a sentence normalization system, a sentence normalization method, and a sentence normalization program according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 shows a sentence normalization system 10 according to this embodiment. The sentence normalization system 10 is a system that inputs a sentence and normalizes the input sentence. The result of normalization by the sentence normalization system 10 is used in a system that makes a response based on the sentence. For example, it is used in a system that automatically executes a function based on a speech recognition result sentence of a user's utterance. The input sentence (text) is, for example, a short sentence described in a natural language. Further, the input sentence may be a spoken sentence. However, the sentence to be normalized is not necessarily limited to the above, and any sentence may be targeted for normalization.

  Sentence normalization is based on a certain rule, removing fluctuations on the surface that have little effect on the meaning of the sentence, and changing it to a preset format such as a symbol string that has a one-to-one correspondence with the meaning of the sentence. It is. As described above, for example, sentences described in a natural language such as a colloquial sentence have various expressions even if they have the same sentence meaning. That is, there is a fluctuation on the surface layer in the sentence. Normalization is to make sentences having the same meaning in a uniform expression (normalized expression) so that the meaning indicated by the sentence can be easily used in a system in which the result of normalization is used. Specifically, the format to be normalized will be described later.

  Specifically, the sentence normalization system 10 is realized by a computer such as a server device or a PC (Personal Computer). The sentence normalization system 10 is not necessarily realized by a single computer, and may be realized by a computer system including a plurality of computers connected to each other. In addition, the sentence normalization system 10 may be connected to another device to input necessary information or to output generated information, and may be capable of transmitting and receiving information.

  Subsequently, the functional configuration of the sentence normalization system 10 will be described. As shown in FIG. 1, the sentence normalization system 10 includes an input unit 11, a morpheme analysis unit 12, a separation unit 13, a content word string extraction unit 14, a meaning label string extraction unit 15, and a symbol string combination unit 16. And is configured.

  The input unit 11 is input means for inputting a sentence to be normalized. Specifically, the input unit 11 inputs a sentence by receiving a sentence (text information) from an external device or module. Or the input part 11 inputs a sentence by receiving the input of a user's sentence. The input unit 11 outputs the input sentence to the morphological analysis unit 12.

  The morpheme analyzer 12 is a morpheme analyzer that divides the sentence input from the input unit 11 into word strings (morpheme strings) and estimates the part of speech of each divided word. The morphological analysis unit 12 may estimate not only the part of speech of each divided word but also their utilization form. As a method for realizing morphological analysis for dividing into word strings and estimating parts of speech and utilization forms, methods that have been widely known to those skilled in the art can be used.

  As an example, when the input sentence is “I ate my ice cream”, the word string and the part of speech of the morphological analysis result by the morphological analysis unit 12 are “I: noun”, “no: particle”, “ Ice: Noun, Eat: Verb, Ta: Auxiliary verb, Doshi: Auxiliary verb, U: Auxiliary verb. In addition, said description is "the word divided | segmented from the sentence: the estimated part of speech of the said word." The word strings are arranged in the order in which words appear (appear) in the sentence. The unit of word division does not affect the essence of processing, and may differ depending on the implementation of morphological analysis. In addition, verbs such as verbs may be converted to a prototype (end form). This conversion is a function generally provided in the morphological analysis process. In the present embodiment, the utilization word is handled as being converted into a prototype. The morphological analysis unit 12 outputs information indicating the word string divided from the sentence and the estimated part of speech of each word to the separation unit 13.

  Based on the word boundary obtained from the result of the morphological analysis and the estimated part of speech of each word, the separation unit 13 includes a content part including the content of the sentence, a sentence ending, It is the separation means which isolate | separates into. Specifically, the separation unit 13 first determines whether the word is an independent word or an adjunct word based on the part of speech of each word. For example, the determination is made according to general grammar. That is, the separation unit 13 determines words of part of speech such as nouns, verbs, adjectives, adjective verbs, adverbs, and emotional verbs that are preset parts of speech as independent words. Further, the separation unit 13 determines a word that has not been determined as an independent word as an attached word. Note that adverbs may or may not be included in self-supporting words by manner.

  Subsequently, the separation unit 13 includes a content word of a continuous word string from the beginning of the sentence to the last independent word. Further, the separating unit 13 sets a continuous word string from the word next to the content part to the end of the sentence as the end of the sentence. As a result, the end of the sentence is an interval of attached words that continues from the end of the sentence. As an exception, among the independent words included in the word string input from the morphological analysis unit 12, the preset words “good”, “good”, “thing”, “yes”, “not”, “done”, “respect” ( A self-supporting word that has a low meaning as a content word such as “OK” may be regarded as an attached word.

  Further, a part or all of these independent words, which are preset words, can be obtained by adding the word immediately before the target word corresponding to the preset word, the part of speech of the immediately preceding word, or the immediately preceding word. Whether or not the target word is regarded as an attached word may be determined according to the usage form (depending on). For example, the adjective “good” is permitted without the meaning of “good / bad”, which is the original meaning when the particles “te” and “de” are immediately preceded, such as “can I do it?” It is natural to interpret it as a functional expression that asks for it, but it is natural to interpret it as having a meaning of “good / bad” when there is no particle “te” or “de” immediately before, such as “it is good”. . The separation unit 13 stores in advance a rule according to the above for regarding an independent word as an attached word, and determines whether or not the independent word is regarded as an attached word based on the rule.

  Note that the determination that an independent word is regarded as an attached word may be made by the sentence normalization system 10 as a whole. For example, once the independent word is regarded as an auxiliary word in the separating unit 13, the independent word is treated as an auxiliary word in the subsequent processing. Alternatively, the determination is performed in any one of the function units (for example, the separation unit 13 and the content word string extraction unit 14) where the distinction between the independent words and the attached words is used, and the independent words are only obtained by the processing in the functional unit. It may be treated as an adjunct.

  FIG. 2 shows an example of sentence separation into a content part and a sentence end. As shown in Fig. 2, if the input sentence is "I would have eaten my ice", a continuous word string from the beginning of the sentence to the last independent word "eat: verb" The content part. Further, a continuous word string from “ta: auxiliary verb”, which is the next word of the independent word located at the back to the end of the sentence, is set as the end of the sentence. The separation unit 13 outputs the word string that is the content part by the above separation to the content word string extraction unit 14 and the word string that is the sentence end to the semantic label string extraction unit 15.

  The content word string extraction unit 14 is content information extraction means for extracting content information indicating the content of the input sentence from the independent words included in the content part input from the separation unit 13. Specifically, the content word string extraction unit 14 extracts, as content information, a content word string that is a word string indicating the content of the input sentence from the content part. The content word string extraction unit 14 extracts all words corresponding to independent words from the words included in the content part, and sets them as the content word string. The content word string may be arranged in the order in which the extracted words appear in the sentence.

  The content word string extraction unit 14 may replace a word that can be grouped as a synonym or synonym among the words included in the content word string with an identifier that uniquely indicates the group. For example, the words “I” and “I” may be considered synonymous with pronouns indicating the first person singular, so if any word is included in the content word string, <pronoun: first person singular> It may be replaced with an identifier. This replacement can be performed, for example, based on the correspondence table in which the correspondence table in which the word notations and identifiers shown in FIG. 3 are associated in advance is stored in the content word string extraction unit 14.

  Further, the content word string extraction unit 14 may perform the question sentence determination process by the method described below, although it is not essential. The content word string extraction unit 14 determines whether or not the input sentence is a question sentence by determining whether or not a question word set in advance is included in the content word string. The preset question words are, for example, words such as “what” and “when”. When the content word string extraction unit 14 determines that the question word set in advance is included in the content word string, it removes the question word from the content word string and determines that the input sentence is a question sentence. When the content word string extraction unit 14 determines that the question word set in advance is not included in the content word string, the content word string extraction unit 14 determines that the input sentence is not a question sentence.

  Further, when it is determined that the input sentence is a question sentence as described above, the content word string extraction unit 14 determines the type of the question sentence based on the question word. For example, “What” is “What”, “When” is “When”, and the interrogative sentence is 5W1H (Who (who), What (what), When (when)), Where (where), Why (why). , How (how)) is determined. The type of question sentence is associated with, for example, a preset question word. The above determination is made, for example, by preliminarily storing the correspondence table in which the notation of the interrogative word as shown in FIG. 3 is associated with the questionable word type and the questionable sentence type in the content word string extracting unit 14. This can be done based on the correspondence table.

  Alternatively, the content word string extraction unit 14 determines whether or not the input sentence is a question sentence by determining whether or not the last word in the content word string is a preset noun corresponding to the question sentence. judge. The noun corresponding to the question sentence is an abstract noun corresponding to any of 5W1H such as “time”, “length”, and “reason”. When the content word string extraction unit 14 determines that the last word in the content word string is a preset noun corresponding to the question sentence, the noun is removed from the content word string, and the input sentence is the question sentence. It is determined that When it is determined that the last word in the content word string is not a preset noun corresponding to the question sentence, the content word string extraction unit 14 determines that the input sentence is not a question sentence.

  When the content word string extraction unit 14 determines that the input sentence is a question sentence as described above, the content word string extraction unit 14 determines the type of the question sentence based on the last word. The question sentence type is associated with, for example, a preset noun corresponding to the question sentence. The type of question sentence is “When” for “time”, “HowMuch” for “length”, and “Why” for “reason”. For example, the above determination includes information (for example, the correspondence table shown in FIG. 3) that associates a notation of a preset noun corresponding to a question sentence in advance with a question word and a question sentence type. It can be stored based on the information stored in the word string extraction unit 14.

  A specific example of the above determination will be shown. If the extracted content word sequence is “rice” or “what”, “what” is a question word corresponding to the type of “What”, so it is removed from the content word sequence and the sentence is changed to the type of “What”. Judged as the corresponding question sentence. If the extracted content word string is “Mt. Fuji” or “Height”, the word “height”, which is the last word in the content word string, is an abstract noun corresponding to “HowMuch”. It is determined that the sentence is a question sentence corresponding to the type of “HowMuch”.

  As a result of the above processing, the content word string extraction unit 14 determines the content word string, the determination result of whether or not the sentence is a question sentence, and, if the sentence is a question sentence, three pieces of information of the question sentence type Get. The content word string extraction unit 14 outputs these pieces of information to the symbol string combination unit 16. However, it is not essential to determine whether or not the sentence is a question sentence, and to determine the type of question sentence when the sentence is a question sentence, and it is not essential to output information on these determinations and determination results.

  The meaning label string extraction unit 15 is a function information extraction unit that extracts function information indicating the function expression of the sentence from the sentence end input from the separation unit 13. The functional information indicating the functional expression of the sentence indicates what kind of intention is expressed as a whole sentence such as “request”, “denial”, “complete”, and “estimation”. In the present embodiment, the semantic label string extraction unit 15 extracts a semantic label string including a set of semantic labels as described below as function information. The semantic label string extraction unit 15 stores a correspondence table in which notations and semantic labels as shown in FIG. 4 are associated in advance, and extracts semantic labels based on the correspondence table. The semantic label string extraction unit 15 determines whether the notation of the continuous word string (including one word) included at the end of the sentence matches the notation of the stored correspondence table, and if they match, To the semantic label column. The collation between the end of the sentence and the notation of the correspondence table of FIG. 4 may be performed based on simple character string matching. Alternatively, instead of simple character string matching, the semantic label may be estimated in consideration of the immediately preceding word or its utilization form. For example, Japanese Patent Application Laid-Open No. 2011-145844 (predicate function expression normalization method, apparatus and program thereof, Patent Document 2) and Japanese Patent Application Laid-Open No. 2012-203584 (Predicate normalization apparatus, method and program, Patent Document 3). ).

  For example, in the case of the example sentence shown in FIG. 2, since “ta” corresponds to the meaning label “complete” and “decho” corresponds to the meaning label “estimation”, the meaning label string extraction unit 15 performs “completion” Are extracted as a semantic label string. The meaning label string extraction unit 15 outputs the extracted meaning label string to the symbol string combination unit 16.

  The symbol string combination unit 16 generates a normalized symbol string by concatenating the content word string input from the content word string extraction unit 14 and the meaning label string input from the meaning label string extraction unit 15, and generates a normalized symbol string. An output unit that outputs information indicating a normalized expression of a sentence based on a column. For example, the symbol string coupling unit 16 performs the output by transmitting the information to a system that makes a response based on the input sentence. Further, the information may be output to any other output destination.

  When the determination result that the sentence is a question sentence is input from the content word string extraction unit 14, the symbol string combination unit 16 adds “question” indicating that to the meaning label string. The symbol string combination unit 16 also adds information indicating the type of question sentence input from the content word string extraction unit 14 to the semantic label string. In the generation of the normalized symbol string, the symbol string coupling unit 16 treats the content word string as a string in which the content words are arranged in the order of appearance in the input sentence. That is, the symbol string coupling unit 16 assumes that the content information corresponds to the order in which the independent words appear in the sentence. Alternatively, the content words may be sorted on a constant basis according to the character string of the content words. This reference is, for example, in the order of 50 notes or alphabetical order. Sorting in the normalization procedure is equivalent to handling the content word string ignoring the order in the input sentence.

  Further, the symbol string coupling unit 16 may sort the meaning label string according to a character string of the meaning label according to a certain standard. This standard is, for example, the same as the above in the order of the Japanese syllabary or alphabetical order. Further, as a result of adding the question sentence determination result, duplication may occur in the semantic label string, and the duplication may be removed.

  With the above processing, when the question sentence determination process is performed in the content word string extraction unit 14, the normalized symbol string is the same regardless of the position of the question word in any position of the input sentence. . In addition, the question sentence using the question word and the question sentence not used, such as “when is the meeting” and “the time of the meeting”, are the same content word string “meeting”, meaning label string “question”, “When” Can be regarded as having the same meaning.

  FIG. 5 shows an input sentence and a normalized symbol string corresponding to the input sentence obtained by the symbol string coupling unit 16. In the example of the normalization symbol string shown in the normalization pattern column of the table of FIG. 5, the word string following the character string “Content” is the content word string, and the word string following the character string “Label” is the meaning label string. Show. The word string here also includes an identifier (for example, <pronoun: first person singular>) in which the word is replaced.

  As described above, the normalization symbol string is a set of two identifier arrays (word strings), which are a content word string and a semantic label string. The symbol string coupling unit 16 may expand the identifier array into a character string according to a certain rule as shown in the column of the normalization pattern in FIG. This character string can be regarded as a normalization pattern (normalized expression of the sentence) uniquely determined from the input sentence. The symbol string coupling unit 16 outputs the above normalization pattern. It should be noted that the normalized expression of the sentence is not necessarily a normalization pattern as shown in FIG. 5, and can take any form as long as it is information based on the normalization symbol string.

  If the normalization patterns obtained from a plurality of sentences are the same, they can be regarded as synonymous. For example, the sentence “I would have eaten my ice” and the sentence “I want to eat my ice” in FIG. 5 are not synonymous because the semantic label columns are different and the normalization patterns do not completely match. On the other hand, the sentence “when is the meeting” and the sentence “when is the meeting” are synonymous because the normalization patterns completely match. As described above, the meaning label strings are sorted, and the comparison between the normalization patterns is equivalent to the comparison between sets for the meaning labels. The synonym determination between a plurality of sentences shown in the above procedure is used, and for example, an application such as an interactive system that returns a robust response to a sentence input by a user is possible. The functional configuration of the sentence normalization system 10 has been described above.

  FIG. 6 shows a hardware configuration of the sentence normalization system 10 according to the present embodiment. As shown in FIG. 6, the sentence normalization system 10 includes a central processing unit (CPU) 101, a random access memory (RAM) 102 and a read only memory (ROM) 103 that are main storage devices, a communication module 104 for communication, In addition, the computer includes a computer including hardware such as an auxiliary storage device 105 such as a hard disk. The functions of the sentence normalization system 10 described above are exhibited when these components are operated by a program or the like. The above is the configuration of the sentence normalization system 10 according to the present embodiment.

  Subsequently, a sentence normalization method, which is an operation method of the sentence normalization system 10 according to the present embodiment and is a process executed by the sentence normalization system 10, will be described using the flowchart of FIG. 7. In this process, a sentence to be normalized is input by the input unit 11 (S01, input step). The input sentence to be normalized is output from the input unit 11 to the morpheme analysis unit 12. Subsequently, the morpheme analysis unit 12 performs morpheme analysis on the sentence input from the input unit 11, and the sentence is divided into word strings (morpheme strings), and the part of speech of each divided word is estimated. (S02, morphological analysis step). Information indicating the word string divided from the sentence and the estimated part of speech of each word is output from the morphological analysis unit 12 to the separation unit 13.

  Subsequently, the separation unit 13 separates the word string input from the morphological analysis unit 12 into a content part and a sentence end based on the estimated part of speech of each word (S03, separation step). The word string set as the content part is output from the separating part 13 to the content word string extracting part 14. The word string at the end of the sentence is output from the separation unit 13 to the semantic label string extraction unit 15.

  Subsequently, the independent word included in the content part input from the separation unit 13 is extracted by the content word string extraction unit 14 as a content word string that is content information (S04, content information extraction step). At this time, determination as to whether or not the sentence is a questionable sentence and determination of the type of questionable sentence may be performed based on the content part. The content word string obtained as described above, the determination result as to whether or not the sentence is a question sentence, and, if the sentence is a question sentence, three kinds of information of the question sentence type are symbolized from the contents word string extraction unit 14 It is output to the column coupling unit 16.

  On the other hand, the meaning label string extraction unit 15 extracts a meaning label string as function information from the sentence end inputted from the separation unit 13 (S05, function information extraction step). The extracted meaning label string is output from the meaning label string extraction unit 15 to the symbol string combination unit 16. Note that the processes of S04 and S05 are independent processes, and thus are not necessarily performed in the above order, and may be performed in parallel or in the reverse order.

  Subsequently, the symbol string combination unit 16 concatenates the content word string input from the content word string extraction unit 14 and the meaning label string input from the meaning label string extraction unit 15 to generate a normalized symbol string. (S06, output step). Subsequently, the symbol string coupling unit 16 outputs a normalized pattern (normalized expression of the sentence) based on the normalized symbol string by being transmitted to a system that makes a response based on the input sentence, for example ( S07, output step). The above is the sentence normalization method that is the process executed by the sentence normalization system 10 according to the present embodiment.

  As described above, in the present embodiment, a word string obtained from a sentence is separated into a content part and a sentence end, and a content word string that is content information and a meaning label string that is function information are extracted from each of them. The normalization pattern based on is output as a normalized representation of the sentence. In other words, in this embodiment, normalization based on simple and deterministic operations is performed from a more basic and robustly operating word string, without using parsing that is easily affected by grammatical disruptions that often appear in colloquial sentences. Is called. That is, normalization is robustly performed while avoiding sentence normalization errors due to the accuracy limit of parsing. Accordingly, the sentence normalization system according to the present invention can robustly normalize a sentence that is difficult to normalize robustly by syntactic analysis of a colloquial sentence or the like.

  Further, as described above, the separation of the content part and the sentence end may be performed by using a section from the word string to the rearmost independent word as the content part and the section and the following as the sentence end. According to this configuration, the sentence can be appropriately and reliably separated into the content part and the sentence end, and as a result, normalization can be performed appropriately.

  In addition, as in the present embodiment, question sentence determination and question sentence type determination may be performed based on the words in the content word string. According to this configuration, it is possible to appropriately normalize the question sentence.

  Further, the content word string may not be sorted as in the present embodiment, and the semantic label string may be sorted and normalized. It is considered that the meaning of the order with respect to the meaning is large for the content word sequence, and the meaning of the order with respect to the meaning of the meaning label sequence is small. Therefore, according to this configuration, appropriate normalization can be performed. In particular, since the semantic label string is sorted, normalization that enables comparison between sentences can be performed by ignoring the order of the semantic label string. However, it is not always necessary to sort the content word string and the semantic label string as described above. Depending on the assumed sentence and the system in which normalization is used, the content word string may be sorted or the semantic label string may not be sorted.

  Further, as described above, normalization may be performed by regarding a specific independent word as an attached word. According to these configurations, it is possible to appropriately handle words that should be attached words in normalization, and as a result, normalization can be performed appropriately.

  Subsequently, a sentence normalization program for causing a computer to execute the above-described series of sentence normalization systems 10 will be described. As shown in FIG. 8, the sentence normalization program 30 is inserted into a computer and accessed, or stored in a program storage area 21 formed on a recording medium 20 provided in the computer.

  The sentence normalization program 30 includes an input module 31, a morphological analysis module 32, a separation module 33, a content word string extraction module 34, a semantic label string extraction module 35, and a symbol string combination module 36. The The functions realized by executing the input module 31, the morphological analysis module 32, the separation module 33, the content word string extraction module 34, the semantic label string extraction module 35, and the symbol string combination module 36 are described above. The functions of the input unit 11, morphological analysis unit 12, separation unit 13, content word string extraction unit 14, semantic label string extraction unit 15, and symbol string combination unit 16 of the sentence normalization system 10 are similar to each other. It is.

  The sentence normalization program 30 may be partly or wholly transmitted via a transmission medium such as a communication line and received and recorded (including installation) by another device. Each module of the sentence normalization program 30 may be installed in any one of a plurality of computers instead of one computer. In that case, the series of sentence normalization programs 30 described above is performed by the computer system of the plurality of computers.

  DESCRIPTION OF SYMBOLS 10 ... Sentence normalization system, 11 ... Input part, 12 ... Morphological analysis part, 13 ... Separation part, 14 ... Content word string extraction part, 15 ... Semantic label string extraction part, 16 ... Symbol string coupling | bond part, 101 ... CPU, 102 ... RAM, 103 ... ROM, 104 ... communication module, 105 ... auxiliary storage device, 20 ... recording medium, 21 ... program storage area, 30 ... sentence normalization program, 31 ... input module, 32 ... morpheme analysis module, 33 ... Separation module 34... Content word string extraction module 35... Meaning label string extraction module 36.

Claims (8)

An input means for inputting a sentence;
A morpheme analyzing unit that divides a sentence input by the input unit into word strings and estimates a part of speech of each divided word;
Separating means for separating the word string divided by the morphological analysis means into a content part including the content of the sentence and a sentence end based on the estimated part of speech of each word;
Content information extracting means for extracting content information indicating the content of the sentence from independent words included in the content part separated from the sentence by the separating means;
Functional information extracting means for extracting functional information indicating the functional expression of the sentence from the sentence end separated from the sentence by the separating means;
Output means for outputting the content information extracted by the content information extraction means and the function information extracted by the function information extraction means as a normalized expression of the sentence;
A sentence normalization system.   2. The sentence normalization system according to claim 1, wherein the separating unit includes a section from the word string up to the rearmost independent word as a content part and a sentence end after the section.   The content information extraction means determines whether or not a word extracted as content information includes a question word, or whether or not the last word among the words extracted as content information is a noun corresponding to the question sentence. The sentence normalization system according to claim 1, wherein the sentence normalization system determines whether or not the sentence is a question sentence and determines a type of the question sentence based on the question word or the last word.   The sentence normalization according to any one of claims 1 to 3, wherein the output means sets the content information according to an order in which the independent words appear in the sentence, and sorts and outputs the function information. system.   The sentence normalization system as described in any one of Claims 1-4 which considers the preset independent word among the independent words contained in the word string divided | segmented by the said morpheme analysis means as an attached word.   The sentence according to claim 5, wherein the target word is regarded as an adjunct according to the word immediately before the target word corresponding to the preset independent word, the part of speech of the previous word, or the utilization form of the previous word. Normalization system. A sentence normalization method that is an operation method of a sentence normalization system,
An input step for entering a sentence;
Dividing the sentence input in the input step into word strings and estimating the part of speech of each divided word;
A separation step of separating the word string divided in the morphological analysis step into a content part including the content of the sentence and a sentence end based on the estimated part of speech of each word;
A content information extraction step for extracting content information indicating the content of the sentence from independent words included in the content part separated from the sentence in the separation step;
A function information extraction step for extracting function information indicating the function expression of the sentence from the sentence end separated from the sentence in the separation step;
An output step of outputting the content information extracted in the content information extraction step and the function information extracted by the function information extraction means as a normalized expression of the sentence;
Normalization method including Computer
An input means for inputting a sentence;
A morpheme analyzing unit that divides a sentence input by the input unit into word strings and estimates a part of speech of each divided word;
Separating means for separating the word string divided by the morphological analysis means into a content part including the content of the sentence and a sentence end based on the estimated part of speech of each word;
Content information extracting means for extracting content information indicating the content of the sentence from independent words included in the content part separated from the sentence by the separating means;
Functional information extracting means for extracting functional information indicating the functional expression of the sentence from the sentence end separated from the sentence by the separating means;
Output means for outputting the content information extracted by the content information extraction means and the function information extracted by the function information extraction means as a normalized expression of the sentence;
Sentence normalization program to function as.

Images