JP2003058538A - Sentence analysis method and sentence analyzer capable of utilizing this method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique which efficiently specifies the theme of sentences. SOLUTION: In a sentence processing system 10, a sentence inputted by a user is acquired by a sentence acceptance part 26 and is set as a noticed block by an object setting part 28, and a character string analysis part 34 extracts one or more words from the noticed block, and a set of the words is used to update sets of words extracted in the past, and thus a theme specifying part 42 specifies the theme of the sentence.

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to a sentence analysis method and apparatus. The present invention particularly relates to topic identification technology in dialogue.

[0002]

2. Description of the Related Art With the progress of advanced information technology in the general society,
Inputting sentences to an electronic terminal such as a personal computer (hereinafter referred to as “PC”) is a part of many people's lives. In the past, the main form of text input was to create business documents and academic papers with a word processor, but today, with the spread of the Internet, tools such as e-mail are indispensable for the communication of all people. Recently, mobile phones are also equipped with an e-mail function as standard. As a result, it is expected that the chances that the computer will process sentences input by humans will increase more and more.

[0003]

Here, when a computer processes a sentence, a lot of research has been done on techniques necessary for treating the sentence as a meaningful character string, not as simple text data. One of them is the research of the technique to identify the topic of the text. If these technologies enable computers to more accurately understand the content of text and the intentions of text creators, the use of computers as intelligent agents will dramatically increase.

However, the conventional approaches to topic identification often rely on the syntactic analysis and semantic analysis of the target sentence. In this case, in order to maintain a certain degree of accuracy, it is necessary to make a database of all possible text patterns and dialogue patterns, which inevitably becomes complicated. For example, Japanese Patent Laid-Open No. 63-106042 discloses a technique for searching for a break in a topic by paying attention to the meaning of pronouns and prepositions. In this case, a sentence pattern based on all pronouns and prepositions is used as a condition. It is necessary to register, and it is hard to say that the search process is efficient. Given the fact that many simple devices other than PCs such as mobile phones have text processing functions,
There are great expectations for the establishment of general-purpose and highly efficient text analysis technology.

The present inventor has made the present invention based on the above recognition, and an object thereof is to provide a technique for efficiently specifying a theme of a sentence.

[0006]

One aspect of the present invention relates to a sentence analysis method. This method acquires sentences that are input one after another over time, sets a block of newly acquired sentences as an input block each time there is an input, and decomposes the attention block to generate at least one or more words and phrases. The theme of the sentence set as the block of interest is identified based on the change including the temporal element between the extracted phrase and the phrase extracted in the past.

"Sentences that are input one after another over time" are, for example, sentences that a user of a PC continues to input when creating a document, sentences that a plurality of users mutually transmit via a network, and voice recognition. A character string such as a sentence generated as a result of the above is being continuously input to the present device as an analysis target. "A newly acquired block of sentences" means, for example, that each paragraph included in a series of sentences is used as a block unit, or utterance content transmitted at one time in a dialogue such as so-called chat is used as a block unit. A string of three or more words that form a significant chunk.

The "change including a temporal element" focuses on the fact that a change in a topic depends not only on a change in a person's thinking but also on a lapse of time. For example, except for the case where the topic is completely changed, human beings proceed with conversation while keeping a memory of the previous conversation contents, and it can be said that the memory fades over time. To put it the other way around, the conversation is proceeded while memorizing the past conversation contents that went back a while to some extent, and not necessarily only the immediately preceding conversation contents are memorized. By incorporating this point into the topic identification algorithm, it is possible to realize a technology that is closer to human consciousness.

The "sentence theme" is, for example, the utterance of the creator of the sentence at the time when the block of the sentence is input, such as a topic for each statement in a conversation, a topic for each paragraph in a paper, or a subject for each sentence in a document file. The main points of the sentence are as follows.

Another aspect of the present invention is a sentence analysis device. This device is a sentence receiving unit that receives a sentence input,
Each time there is an input, the target setting unit that sets the block of the newly input sentence as the attention block, the character string analysis unit that decomposes the attention block and extracts at least one or more words, and the extracted words and phrases And a theme specifying unit for specifying a theme of a sentence based on the theme specifying unit including a temporal element between the theme of the sentence set as the attention block and the theme of the sentence input in the past. Identify based on change.

The term "previously entered" as used herein mainly means "previously entered", but the "previously entered sentence theme" is "further previously entered sentence theme". Is sometimes reflected, and is expressed as "past" in the sense that it is included.

Yet another aspect of the present invention is a computer program. This program acquires the sentences that are input one after another over time, sets the newly acquired block of the sentence as the input block each time there is an input, and decomposes the block of interest into at least one block. A process for extracting the above words and phrases, and a process for identifying a theme of a sentence set as a block of interest based on a change including a temporal element between the extracted phrases and the words extracted in the past, To run.

Any combination of the above components,
It is also effective as an aspect of the present invention that the components and expressions of the present invention are mutually replaced among methods, devices, systems, computer programs, recording media storing computer programs, and the like.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION In this embodiment, a block of an input sentence is decomposed to extract a plurality of words and phrases, and a set of words and phrases that are the theme of the sentence is determined using these. The degree of importance is given to each of the plurality of words as a weighting, and the degree of importance is lowered with the passage of time.
This degree of importance corresponds to the size of the impression that the phrase has in the consciousness of the human being during the conversation, and is adjusted as the consciousness or memory fades over time. By updating such a set of phrases and importance each time a sentence is input, it is possible to proceed with the dialogue process while efficiently determining the latest “theme” in real time.

FIG. 1 is a functional block diagram showing the configuration of the text processing system in this embodiment. The text processing system 10 includes an input / output unit 12 and a text analysis unit 1.
Have 4. The input / output unit 12 processes input / output of texts with a user or input / output of texts via the Internet. The text analysis unit 14 analyzes the text input by the input / output unit 12 and outputs the result to the input / output unit 12. Text processing system 1
0 can be realized by an element such as a CPU of a computer in terms of hardware, and can be realized by a program having a data processing function in terms of software.
In this figure, the functional blocks realized by the cooperation are drawn. Therefore, these functional blocks can be realized in various ways depending on the combination of hardware and software. This sentence processing system 10 is P
In addition to C, a mobile phone, a PDA, etc., it can be realized as any electronic device or home electric appliance having a text processing function.

The input / output unit 12 has a communication section 20, a display section 22, a dialogue processing section 24, a language input processing section 30, and an application processing section 40. The language input processing unit 30 receives an input of a sentence from the user by keyboard input or voice recognition input, and sends the sentence to the interaction processing unit 24 in the form of text data. The communication unit 20 receives a sentence from another user via the Internet. Dialog processing unit 24
Displays a sentence exchanged between a plurality of users on the display unit 22 in an interactive manner and transmits the sentence input by the user of the present apparatus to another user via the communication unit 20. The application processing unit 40 will be described later.

The text analysis unit 14 includes a text reception unit 2
6, target setting unit 28, likelihood determination unit 32, character string analysis unit 3
4 and a theme specifying unit 42. Text reception section 26
Accepts a text input from the dialogue processing unit 24. This sentence is text data input to the dialogue processing unit 24 in the form of dialogue. The target setting unit 28 sets a block of a newly input sentence as a block of interest each time there is an input.

The character string analysis unit 34 includes a phrase extraction unit 36 and an importance level setting unit 38. The word / phrase extracting unit 36 decomposes the block of interest and extracts at least one or more words / phrases. The extraction of words may be performed by a general morphological analysis method. For example, it seems that the words "yesterday" and "sunny" are extracted from the sentence "Yesterday was sunny."

The importance setting section 38 associates each extracted word with the importance of the word in the sentence. This importance acts as a weight for each word. For example, "5" is given to the words "yesterday" and "sunny" as the degree of importance. The importance may be set based on at least one of the linguistic likelihood and the recognition likelihood. The linguistic likelihood is calculated by, for example, the phrase extraction unit 3
It is the linguistic certainty of each word or phrase in the morphological analysis according to 6, and may be judged by dependency or co-occurrence between words and the like. The linguistic likelihood may be the linguistic certainty of the conversion result in text conversion or kana-kanji conversion during speech recognition by the language input processing unit 30. The recognition likelihood indicates the certainty of the recognition in the speech recognition by the language input processing unit 30, for example, and may be affected by the sound volume or noise. The likelihood determining unit 32 may determine the linguistic likelihood and the recognition likelihood.

The theme specifying unit 42 is a block for specifying the theme of a sentence based on the words and phrases extracted by the character string analyzing unit 34, and includes a similarity determining unit 44 and an importance updating unit 4.
6, a theme determination unit 48, and a theme holding unit 50. This theme is based on the change including the temporal element between the themes of the sentences entered in the past, assuming that one or more words and the set of their importance are reflected in the theme. Specified. Each word / phrase reflected in the theme contributes to the topic identification for each sentence in the dialogue. For example, a set of phrases such as {yesterday (5), sunny (5)} and their importance is positioned as the theme of the sentence "It was sunny yesterday."

The theme holding unit 50 stores the newly specified latest theme, and is referred to as the "previous theme" when the new theme is specified when the next text is input. The theme holding unit 50 may be configured as a memory that stores only the latest theme, or may be configured as a database that cumulatively holds past themes. Each theme may be recorded as one record in the database.

The similarity determining unit 44 determines the similarity between the set of words in the target block and the set of words included in the previous theme. This similarity is a conceptual closeness between a set of words and phrases, and corresponds to, for example, the magnitude of change in a topic in a dialogue. Therefore, if the degree of similarity is high, it can be determined that the topic is continuing, and if the degree of similarity is low, it can be determined that the topic has changed significantly.

As the similarity, the number of intersection sets between two sets may be used. In this case, two sets A and B
Is similar to | A∩B | / | A∪B | = | A∩B | /
It is obtained by the formula of {| A | + | B | − | A∩B |}. Alternatively, the Levenshtein distance may be used as the similarity. In this case, the similarity between the two sets A and B is max
{| A |, | B |}-| A∩B | A distance between average vectors of two sets may be used as the similarity. This vector may be expressed by an n-dimensional space vector for a set having n words and phrases as elements.

When the degree of similarity is lower than a predetermined reference value, the importance updating unit 46 updates the theme by replacing the words included in the previous theme with the set of words in the target block. That is, when the theme of a sentence changes significantly, all the sets of words are exchanged.

When the degree of similarity is equal to or greater than a predetermined reference value, the importance update unit 46 updates the set of words and phrases included in the previous theme and their importance by using the set of words and phrases in the target block. To identify the latest theme. Specifically, the words in the target block and the words included in the previous theme are merged, and when a common word is included, the one with higher importance is used. For example, if the phrase and its importance in the block of interest are {yesterday (5), sunny (5)} and the phrases and importance in the past are {today (3), sunny (3)}, these are merged. The updated set is {yesterday (5), sunny (5), today (3)}.

The importance updating unit 46 attenuates the importance of a word or phrase when updating the word and the importance according to the time elapsed until the update. For example, if the newly specified theme is {Yesterday (5), Sunny (5)}, the importance will be reduced to {Yesterday (3), Sunny (3)} at the next theme identification,
This set is treated as the above-mentioned “set of words included in the previous theme and their importance”. If this importance falls below a predetermined minimum value, the phrase is excluded from the set. For example, in the case of {yesterday (3), fine (1)}, "fine (1)" is excluded when the condition "exclude importance level 1 or less" is set. That is, in the actual dialogue, it is judged that the impression of "clear" has almost disappeared in the consciousness of the speaker. Note that the degree of attenuation is arbitrary.

The theme determining section 48 determines the set of words and importance updated by the importance updating section 46 as the theme for the sentence set as the block of interest, and records this in the theme holding section 50.

The application processing unit 40 reflects the newly specified theme in the priority order of conversion candidates in the Kana-Kanji conversion by the language input processing unit 30. The application processing unit 40 reflects the newly specified theme in the priority order of the recognition candidates in the voice recognition processing by the language input processing unit 30.

The operation of the above configuration will be described below. FIG. 2 is a flowchart showing the operation of the text processing system 10 according to this embodiment. First, a new sentence is input (S10), and the sentence is set as a block of interest (S12). Extract words and phrases from the block of interest (S1
4), each importance is set (S16). The similarity is determined between the set of words in the block of interest and the set of words in the previous theme (S18). When the degree of similarity is lower than the predetermined reference value (S20N), all the words in the set are replaced (S26). When the similarity is equal to or higher than a predetermined reference value (S20Y), the importance of the word or phrase in the previous theme is attenuated (S22), and the words of each set are merged to update (S24).

The set of updated phrases and importance is determined as the theme for the sentence set as the block of interest, and this is recorded (S28). The new theme is used for dialogue processing and the like (S30). Until the end of these dialogues
The processing of S10 to S30 is repeated (S32N).

FIG. 3 shows a correspondence relationship among a block of interest, a phrase, importance, and similarity in a dialogue. In the figure, "A" and "B" are expressed in an interactive form.
The character 60 is a sentence input as the statement of “Mr. A” and is set as a block of interest at the time of input. The character 64 is a word or phrase that is an element of the set specified as the theme, and the number 66 is its importance. Number 6
2 is the degree of similarity determined with the theme of the previous speech.

[0032] from the target block of speech (2) are extracted phrase of "Hello", for common words and phrases between the theme of the speech (1), even after the annexation, the elements of the set to be the theme of "Hello "Only. The degree of importance is "5" because the original phrase "5" is reduced to "3" when transitioning from utterance (1) to utterance (2), but the same phrase is included in utterance (2). Will remain.

[0033] While the theme of the speech (3) that contains the phrase "Hello", the degree of importance is "3" and decay. Significance of the phrase of "Hello" is speaking down to (4), the further "1", it is excluded in speech (5). Note that the utterance (5) is illustrated to show that the importance of the phrase decreases while both AB are silent, but it need not be displayed on an actual PC screen or the like.

In the statement (6), since the similarity with the statement (5) is judged to be "0", all the words and phrases in the theme are exchanged. Similarly, in the utterance (14), all the words and phrases are exchanged, assuming that the similarity between the utterance (13) and the utterance (13) is lower than a predetermined value. Based on such theme conversion, a higher-level theme may be determined as a higher-level concept of each theme. As shown in the figure, the upper theme of the dialog surrounded by the line 70 is "greeting", the upper theme of the dialog surrounded by the line 72 is "weather", and the upper theme of the dialog surrounded by the line 74 is "TV". It is also possible to apply these higher-level themes to kana-kanji conversion and voice recognition processing.

FIG. 4 shows a process of updating a word and a degree of importance.
The table 80 shows the correspondence between the term and the importance in the previous theme. The table 82 shows the correspondence between the word and the importance after the importance of each word in the previous theme is attenuated. For example, the importance of "rain" is reduced from "5" to "3". The table 84 shows the correspondence between words and phrases in the block of interest and the degree of importance. The table 86 shows the correspondence between the word and the importance degree after the set of the word and the importance degree in the previous theme and the word and the importance degree set in the attention block are merged. The terms are sorted in descending order of importance. “Weather”, “forecast”, and “rain” are terms common to both sets, and a higher value is adopted as the degree of importance. For example, “weather” and “forecast” are “5” because the word in the block of interest is more important.
It is updated with "4", and "rain" is not updated as "3" because the word in the previous theme is more important.
The importance levels of “dark” and “probability” are “1” and are excluded as being below a predetermined minimum value. After this exclusion is made, the set of words and importance is determined as the latest theme,
Shown in table 88. The minimum value for exclusion and the number of words and phrases to be decided as the latest theme are arbitrary.

FIG. 5 shows the result of reflecting the theme in the kana-kanji conversion. For example, in (a), when the theme of the sentence input immediately before is composed of phrases such as "today, sunny, weather", the process of guessing from this theme and converting the illustrated input example into "rain" You may For example (b)
In, when the theme of the sentence entered immediately before is composed of words such as “today, gum, sweets”, even if the input example shown in the figure is converted to “candy” Good. Similar conversion may be reflected in voice recognition.

(Second Embodiment) FIG. 6 shows a screen of the interactive system. In the present embodiment, the dialogue system receives a message from the user and automatically generates and displays a response to the message. At this time, the theme of the user's statement is reflected in the generation of the reply by the dialogue system. In this system, the dialogue along the topic immediately before can be processed quickly and smoothly.

The present invention has been described above based on the embodiments. It should be understood by those skilled in the art that this embodiment is an exemplification, that various modifications can be made to the combinations of the respective constituent elements and the respective processing processes, and that such modifications are within the scope of the present invention. . Hereinafter, modified examples will be described.

The word / phrase extracting unit 36 may unify synonyms and controlled words into certain words / phrases when extracting words / phrases from the block of interest. Moreover, you may convert the phrase into the form of the stem,
The superordinate concept of the term may be further extracted and added.

The theme determining section 48 may add the superordinate concept of these terms to the set of terms updated by the importance updating section 46.

According to the present invention, the theme of a sentence can be specified relatively efficiently.

[Brief description of drawings]

FIG. 1 is a functional block diagram showing the configuration of a text processing system according to this embodiment.

FIG. 2 is a flowchart showing the operation of the text processing system in this embodiment.

FIG. 3 is a block of interest, a phrase, an importance level in a dialogue,
It is a figure which shows the corresponding relationship of and a similarity.

FIG. 4 is a diagram showing a process of updating a word and a degree of importance.

FIG. 5 is a diagram showing a result of reflecting a theme in kana-kanji conversion.

FIG. 6 is a diagram showing a screen of an interactive system.

[Explanation of symbols]

26 text receiving unit, 28 target setting unit, 34 character string analyzing unit, 40 applied processing unit, 42 theme specifying unit.

Claims (10)

[Claims] 1. A sentence that is input one after another over time is acquired, and a block of a sentence that is newly acquired each time the input is input is set as a block of interest, and the block of interest is decomposed into at least one or more blocks. The phrase, which is characterized by specifying the theme of the sentence set as the attention block based on the change including the temporal element between the extracted phrase and the phrase extracted in the past analysis method. 2. A sentence receiving unit that receives an input of a sentence, a target setting unit that sets a block of a sentence newly input as an attention block each time there is an input, and at least one of which is obtained by disassembling the attention block. A character string analysis unit that extracts the above words and phrases, and a theme identification unit that identifies the theme of the sentence based on the extracted words and phrases, wherein the theme identification unit is the sentence set as the block of interest. The text analysis device is characterized in that the above-mentioned theme is specified based on a change including a temporal element with respect to the theme of the previously input text. 3. The sentence receiving unit receives a sentence input in the form of a dialogue, and the theme specifying unit specifies the theme based on a phrase that contributes to topic identification for each sentence in the dialogue. The sentence analysis device according to claim 2, which is characterized in that. 4. The theme specifying unit updates the past set of words by using the set of words in the block of interest, on the assumption that the set of extracted words is reflected in the theme. The sentence analysis device according to claim 2, wherein the latest theme is specified by the above. 5. The character string analysis unit associates each of the extracted words and phrases with the degree of importance of the word or phrase in the sentence, and the theme specifying unit determines whether or not the extracted word or phrase and its degree of importance. On the assumption that the set is reflected in the theme, the latest theme is specified by updating the words and their importance in the past using the words and their importance in the attention block. Claim 2
Or the sentence analysis device described in 3. 6. The theme specifying unit, when updating the word and the degree of importance, attenuates the degree of importance of the word according to the time elapsed until the update, and when the degree of importance falls below a predetermined minimum value. The sentence analysis device according to claim 5, wherein the phrase is excluded from the set. 7. The theme specifying unit determines the degree of similarity between a set of words and phrases in the block of interest and a past set of words and phrases, and when the similarity is lower than a predetermined reference value, these sets are selected. The sentence analysis device according to any one of claims 4 to 6, wherein the update is performed by replacing the sentence. 8. The sentence analysis apparatus according to claim 2, further comprising an application processing unit that reflects the specified theme in a priority order of conversion candidates in Kana-Kanji conversion. 9. The sentence analysis device according to claim 2, further comprising an application processing unit that reflects the specified theme in a priority order of recognition candidates in a voice recognition process. 10. A process of acquiring sentences that are input one after another over time, a process of setting a block of a newly acquired sentence as an attention block each time there is an input, and decomposing the attention block. A process of extracting at least one or more words and phrases, and specifying the theme of the sentence set as the attention block based on a change including a temporal element between the extracted words and phrases and the words extracted in the past. A computer program characterized by causing a computer to execute a process.