JP2008083753A - Dialogue support device and its method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a dialogue support device for allowing accurate and smooth communications by detecting miscommunications in a dialogue between users. <P>SOLUTION: The dialogue support device has a voice recognition part 102, an ambiguous expression detecting part 103, an ambiguous expression subject detecting part 104, a DB speaker specifying part 105, a cognition range estimating part 106, a cognition range difference detecting part 107, an ambiguous expression word list 120, a speaker information DB 121, and a cognition range DB 122. The voice of a speaker is recognized, and if there is any ambiguous expression in the content of the speech, the cognition range DB 122 is searched for the cognition ranges of the speaker and the listener; if there is a difference between the cognition ranges of the speaker and the listener, a warning is displayed to notify the speaker and the listener of the possibility of miscommunications. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a dialogue support apparatus and method for detecting misunderstandings and consciousness shifts in ambiguous expressions in communication between a speaker and a listener.

  In recent years, the importance of remote communication such as video conferencing has been increasing against the background of speeding up of corporate activities and globalization.

In the future, it is expected that PCs (personal computers) will be used as remote communication terminals due to the spread of VoIP. When a PC is used, advanced processing such as voice recognition (see, for example, Patent Document 1) and image processing as well as mutual transfer of voices and images between users as in conventional TV conference terminals. Can be realized.
"Speech recognition using probabilistic models" by Seiichi Nakagawa IEICE ISBN4-88552-072-X

  Miscommunication is inevitable as long as the speaker and listener are different in remote communication. The sense of things is often different due to differences in culture, life, growth and background. Moreover, it is expected that opportunities for communication with foreign countries will increase in the future, and the possibility of miscommunication is expected to increase. When people with different senses of language nuances have conversations, there is a possibility that more serious consciousness / recognition shifts may occur.

  It is easily imagined that if such misunderstandings and misconceptions are left unattended in the discussion of the TV conference, the subsequent work will be hindered. However, if an ambiguous expression that is likely to cause consciousness dissatisfaction is confirmed every time it appears in the conversation, the conversation does not proceed slowly, and the efficiency is low and smooth communication cannot be achieved.

  In view of the above problems, the present invention provides a dialogue support apparatus and method that can determine the difference between a speaker and a listener due to the ambiguity even if the expression of the speaker is ambiguity.

  The present invention relates to a dialogue support apparatus that mediates communication between a speaker and a listener, and an ambiguous that causes misunderstandings and deviations of consciousness between the speaker and the listener, and a speech recognition unit that recognizes the speech of the speaker. An ambiguous expression word list storage unit that stores a list of expression words, and an ambiguous expression detection that compares the ambiguous expression word list with the speech recognition result and detects an ambiguous expression phrase including the ambiguous expression word from the speech recognition result. An ambiguous expression target detection unit that detects from the speech recognition result an ambiguous expression target whose interpretation of the listener is ambiguous by the ambiguous expression phrase, and a speaker that identifies the person information of the speaker and the person information of the listener A recognizing range storage unit that stores a recognizing range of the ambiguous expression for each person information with respect to a combination of the specifying unit and the ambiguous expression phrase and the ambiguous expression target; The ambiguous expression phrase related to the person information of the speaker stored in the recognition range storage unit and the recognition range of the speaker for the object of the ambiguous expression are obtained, and the ambiguous information about the person information of the listener stored in the recognition range storage unit A recognition range estimation unit for obtaining a recognition range of the listener with respect to an expression phrase and the ambiguous expression target is compared with a recognition range of the speaker and the listener to obtain a difference between the recognition range of the speaker and the listener. And a recognition range difference detection unit.

  According to the present invention, even if there is ambiguity in the expression of the speaker, the difference between the speaker and the listener due to the ambiguity can be obtained, and misunderstanding and misconception can be prevented.

  Hereinafter, a speech recognition apparatus according to an embodiment of the present invention will be described with reference to the drawings.

(First embodiment)
The dialogue support apparatus according to the first embodiment will be described below with reference to the configuration diagram shown in FIG. 1 and FIG. Note that the dialogue support apparatus of the present embodiment incorporates a voice recognition apparatus.

(1) Configuration of Dialogue Support Device As shown in FIG. 1, the dialogue support device includes a speech recognition unit 102, an ambiguous expression detection unit 103, an ambiguous expression target detection unit 104, a DB (data base) speaker identification unit 105, a recognition It has a range estimation unit 106, a recognition range difference detection unit 107, an ambiguous expression word list 120, a speaker information DB 121, and a recognition range DB 122. Each function of these units 102, 103, 104, 105, 106, 107, 108 can be realized by a program stored in a computer.

(1-1) Voice recognition unit 102
The speech recognition unit 102 recognizes speech speech 130 uttered by the speaker 100. Further, the speech recognition unit 102 notifies the ambiguous expression detection unit 103 of the recognition result 131 for the recognized speech 130 of the speaker.

(1-2) Ambiguous Expression Detection Unit 103
The ambiguous expression detection unit 103 first checks whether or not the word in the ambiguous expression word list 120 exists in the recognition result 131. If it is not found, nothing is done if there is no ambiguous expression in the statement. On the other hand, when an ambiguous expression word is found, an ambiguous expression phrase 132 including the ambiguous expression word is detected.

  Here, the ambiguous expression word list is stored in the DB of the ambiguous expression detection unit 103 as a word list of adverbs, adjectives, and the like indicating a degree such as “slightly, slightly, a little more, too much”, for example. The ambiguous expression word may be a list of words representing different concepts on the same surface layer, such as “English second floor” and “American second floor”, in addition to the degree word. Further, the ambiguous expression detection unit 103 notifies the ambiguous expression target detection unit 104 of the phrase 132 including the ambiguous expression only when the ambiguous expression is included in the speech recognition result.

  The ambiguous phrase including the ambiguous expression word is, for example, an adjective phrase or an adverb phrase.

(1-3) Ambiguous Expression Object Detection Unit 104
The ambiguous expression target detection unit 104 detects an object of the ambiguous expression phrase 132 including the ambiguous expression detected by the ambiguous expression phrase. The ambiguous expression target detection unit 104 notifies the recognition range estimation unit 106 of the received ambiguous expression phrase 132 and a set 133 of the target of the phrase.

(1-4) DB speaker specifying unit 105
The DB speaker specifying unit 105 receives the speaker specifying information 134 for specifying the information of the speaker and the listener, searches the speaker information DB 121 to find out which person in the recognition range DB 122 corresponds to the speaker information, and the speaker information. 135, the listener information 136 is notified to the recognition range estimation unit 106.

  The conversation person specifying information 134 includes a person name and an ID number. The speaker information DB 121 holds a correspondence table between the conversation person specifying information and the person information in the recognition range DB 122. When detailed personal information is managed to the individual level in the recognition range DB 122, the person is specified from the interlocutor specifying information such as the name of the person. On the other hand, in the case where information by person is managed in rough units such as race or country in the recognition range DB 122, the race or country of origin is specified from the interlocutor specifying information.

(1-5) Recognition range estimation unit 106
In the recognition range estimation unit 106, the ambiguous expression phrase output from the ambiguous expression target unit 104, the target group 133 of the phrase, and the conversation person output from the DB speaker specifying unit 105 identifies which person in the recognition range DB 122. Recognizing ranges 137 and 138 for the vague expressions of the speaker and the listener are calculated from the person specifying information 135 and 136, and notified to the recognized range difference detecting unit 107.

(1-6) Recognition range difference detection unit 107
The recognition range difference detection unit 107 receives the respective recognition ranges 137 and 138 for the vague expression of the speaker and the listener, and determines whether there is a difference in the recognition range. The comparison method used in the recognition range difference detection unit 107 may be any conventional information comparison method. For example, when the recognition range is expressed as a numerical value, whether the difference between the two values is within a certain threshold, or when the recognition range is given as a statistical distribution, is the degree of overlap between the distributions calculated from the mean and variance? The difference is judged by the method of how.

  When the recognition range difference detection unit 107 determines that there is a difference between the recognition ranges 137 and 138 of the speaker and the listener, a warning is issued. Any conventional warning notification method may be used as a method of notifying the user of the warning. On the other hand, if it is determined that there is no difference, nothing is done.

(2) Operation Contents of Dialogue Supporting Device Hereinafter, the ambiguous expression detection unit 103, the ambiguous expression target detection unit 104, the recognition range estimation unit 106, and the recognition range difference detection unit 107 will be described with reference to the example of dialogue in FIG.

  FIG. 3 shows an example of dialogue between the person A and the person B. The first column in FIG. 3 is the dialogue number, the second column is the conversation, the third column is the ambiguous phrase and the target of the phrase, the fourth column is the ambiguous expression of the same line in the third column, and the person A for the target set The fifth column represents the recognition range of the person B.

  At the time of the dialogue 001, the ambiguous expression detection unit 103 first detects “slightly” and “a little” which are words in the ambiguous expression word list from the speech recognition result of the speaker A (step 601).

  Then, the phrase of the ambiguous expression word is detected as “slightly changed” and “slightly delayed”, respectively. The ambiguous expression detection unit 103 notifies the ambiguous expression target detection unit 104 of the ambiguous expression phrase (step 602).

  The ambiguous expression target detection unit 104 detects the target noun phrase of the ambiguous expression phrase. The most common method of detecting an object is a phrase dependency word by dependency analysis. When dependency analysis is performed on the dialogue 001 in FIG. 3, dependency relationships of “change slightly” → “specification”, “slightly delay” → “schedule” are detected. The ambiguous expression target detection unit 104 notifies the recognition range estimation unit 106 of the set 133 of the ambiguous expression phrase and the ambiguous expression target (step 603).

  Here, the method used when detecting an object by the ambiguous expression object detection unit 104 and the ambiguous expression object in the “person, ambiguous expression phrase, ambiguous expression object, and recognition area” stored in the recognition area DB 122 are the same method. It must be detected. That is, when the dependency analysis is used in the ambiguous expression target detection unit 104, it is necessary to use the word detected by the dependency analysis in the “ambiguous expression target” in the recognition range DB 122.

  On the other hand, the DB speaker specifying unit 105 specifies which person in the DB the speaker and listener are based on the conversation person specifying information 134. The information specifying the dialogue person includes a name and an ID number. The authentication information may be any kind of authentication information as long as it can specify which person in the recognition range DB 122 of the interaction person matches. The user may explicitly input when starting the system, or may automatically authenticate using a camera and face recognition. Further, the level specified here may be specified up to a personal name, or may be a generalized level such as country, hometown, affiliated company, and the like. What is necessary is just to match | combine with the person who most matches the person in the data in recognition range DB122. The speaker information 135 specifying the speaker and the listener information 136 specifying the listener are notified to the recognition range estimation unit 106.

  The recognition range estimation unit 106 searches the recognition range DB 122 by combining the ambiguous expression phrase and ambiguous expression target set 133 with the speaker information 135 and the listener information 136. In the example of FIG. 3, first, for the group “slightly changed, specifications”, the recognition range is <slightly changed, specifications, A> is “10-25%” 136, <slightly changed, specifications , B> results in “10-30%” 137. Similarly, “slightly delayed, schedule” results in <slightly delayed, schedule A> is “1-4 weeks”, <slightly delayed, schedule, B> is “1-2 weeks”. The recognition range estimation unit 106 notifies the recognition range difference comparison unit 107 of this result (step 604).

  The recognition range difference comparison unit 107 compares the speaker's recognition range 137 estimated by the recognition range estimation unit 106 with the listener's recognition range 138. For example, the comparison results are “slightly change, specification, AB, maximum 5% difference”, “slightly delay, schedule, AB, maximum 2 weeks”. In the example, the comparison result is expressed by the difference in the numerical value of the recognition range, but it is not limited to this method. For example, when the recognition range is given in the form of statistical information, it may be expressed in the form of an overlap degree of probability distributions, an average value, or a variance difference. This will be described later.

  The recognition range difference detection unit 107 detects the case where there is a difference based on the comparison result of the recognition range with respect to the determined expression of the speaker and listener indefinitely (steps 605 and 606). Here, “15% or more, period difference of 1 week or more, difference of money of 10,000 yen or more” is assumed as a criterion for determining that there is a difference. In this case, since the dialogue 001-a in the example of FIG. 3 “changes slightly, the specification, AB, maximum 5% difference” is “less than 15%”, “the misunderstanding occurs between the dialogues A and B. It is determined that no. On the other hand, since the dialogue 001-b “slightly delayed, schedule, AB, maximum 2 weeks” is “1 week or more”, it is determined that “misunderstanding occurs between the dialogues A and B”. Any known user interface may be used as a warning to be notified to the conversation person when the recognition range difference detection unit 107 detects the possibility of “misunderstanding”. For example, any method can be used as long as the user can be alerted by displaying a warning message or playing a warning voice.

  Each of the conversation persons A and B only needs to confirm whether or not there is a deviation in the recognition of the utterance contents only when the conversation support apparatus displays a warning. For example, during the dialogue shown in FIG. 3, the dialogue support apparatus displays warnings only for the three locations 001-b, 006-c, and 013-f, so that the wasteful confirmation work shown in FIG. 3 can be reduced. As shown in FIG. 4, a smooth dialogue can be performed.

(Second Embodiment)
The dialogue support apparatus according to the second embodiment will be described below with reference to FIGS. Note that the dialogue support apparatus of the present embodiment incorporates a speech translation apparatus. Note that the same parts as those in FIG.

  First, when the language used by the speaker and the language used in the system ambiguous expression word list 120, the recognition range DB 122, and the recognition range adjustment DB 123 are different, the speech recognition unit 102 recognizes the speaker's voice and then listens to the listener. It also performs language translation processing.

  The recognition range adjustment unit 109 adjusts the recognition range by referring to the recognition range adjustment DB 123 when comparing the recognition ranges 137 and 138 of the speaker and the listener estimated by the recognition range estimation unit 106. The cognitive range difference detection unit 107 is notified of the adjusted cognitive range information.

  The operation of the recognition range adjustment unit 109 will be described with reference to FIG.

  Fig. 5 shows an example of dialogue between Korean and Japanese. The first column is the conversation number, the second column is the conversation, the third column is the ambiguous phrase and the target of the phrase, the fourth column is the ambiguous expression of the same line in the third column, the Korean recognition range for the target group, the first The 5th column represents the recognition range of the Japanese.

  Here, the recognition range regarding the amount of money that appears in the dialogue 006 is managed in the recognition range DB 122 in units of “won (W)” for Koreans and “yen” for Japanese. This is to accurately represent the recognition range of each citizen. Although it is conceivable that the unit is stored in the recognition range DB 122 in a unit of one of the units (for example, “yen”), there is an inconvenience that it cannot be correctly followed when the exchange rate changes. Therefore, it is desirable to store <a little cheaper clothes, Korean, -10000W> and <a little cheaper clothes, Japanese, -3000 yen> in the recognition range DB 122, respectively. However, in this case, since the unit of the amount is different, the difference in the recognition range of the dialogue 006 cannot be simply compared as it is.

  Here, in the cognitive range adjustment DB 123, if a cognitive range adjustment formula of “Korean, Japanese, 0.12W = 1 yen” is given, <a little cheaper clothes, Korean, -10000W> is <a little cheaper, Clothes, Korean, -1200 yen>. Thereby, it can be correctly detected that there is a possibility of misunderstandings between Koreans and Japanese in the conversation 006.

(Example of change)
The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist thereof.

(First change example)
As a method for detecting an ambiguous expression target, there is a method in which N word strings before and after an ambiguous expression word (N is a natural number of 1 or more) other than the dependency in the above embodiment. This method has an advantage that an object of ambiguous expression can be easily detected without performing complicated analysis processing such as dependency analysis.

  However, in this case, N word strings before and after the ambiguous expression word are also used as the “ambiguous expression object” in the recognition range DB 122.

(Second modification)
Based on FIG. 9, a description will be given of a modification example in which the recognition ranges of Mr. A and Mr. B with respect to the <slightly changed, schedule> expression are statistically expressed.

  If it is just the image range, both Mr. A and Mr. B image 1 to 5 weeks.

  Therefore, when the recognition range is expressed by a simple numerical value, it is expressed as “1 to 5 weeks”, and it is determined that there is no difference.

  In the case of statistical expression, as shown in FIG. 9, A and B both count how many times x weeks are imaged for the expression <slightly changed, schedule>, and based on the frequency Create a statistical distribution. The statistical distribution difference comparison can be determined by the overlapping area S.

  Therefore, it is determined that there is a difference if the x weeks that both images strongly differ.

It is a block diagram which shows the dialog assistance apparatus concerning the 1st Embodiment of this invention. It is a block diagram which shows the dialog assistance apparatus concerning 2nd Embodiment. It is the example of an operation situation explaining operation of a 1st embodiment. It is the example of the operation | movement situation improved by 1st Embodiment. It is an operation | movement situation example explaining the operation | movement of 2nd Embodiment. 10 is a flowchart for explaining the operation procedure of an ambiguous expression detection unit 103, an ambiguous expression target detection unit 104, a DB speaker identification unit 105, a recognition range estimation unit 106, and a recognition range difference detection unit 107. It is a table | surface which shows the example of the group data of the ambiguous word of the ambiguous expression word list | wrist 120, and a part of speech. It is a figure of the table | surface which shows the example of quaternary data of the person of the recognition range DB122, an ambiguous expression phrase, an ambiguous expression object, and a recognition range. It is a figure which shows the example of the comparison method of the figure of the recognition range expressed with statistical information, and statistical distribution.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Speaker user 101 Listener user 102 Speech recognition part 103 Ambiguous expression detection part 104 Ambiguous expression object detection part 105 DB speaker identification part 106 Recognition range estimation part 107 Recognition range difference detection part 109 Recognition range adjustment part 120 Ambiguous expression word list 121 Talk Information DB
122 Recognition range DB122
123 Recognition range adjustment DB123

Claims (11)

In a dialogue support device that mediates communication between the speaker and the listener,
A voice recognition unit for recognizing the voice of the speaker;
An ambiguous expression word list storage unit that stores a list of ambiguous expression words that cause misunderstandings and misconceptions between the speaker and the listener;
An ambiguous expression detection unit that compares the ambiguous expression word list with the speech recognition result and detects an ambiguous expression phrase including the ambiguous expression word from the speech recognition result;
An ambiguous expression target detection unit for detecting an ambiguous expression target whose interpretation of the listener is ambiguous by the ambiguous expression phrase from the speech recognition result;
A speaker identification unit that identifies the person information of the speaker and the person information of the listener;
A recognition range storage unit that stores a recognition range of the ambiguous expression for each person information with respect to the combination of the ambiguous expression phrase and the ambiguous expression target;
The ambiguous expression phrase related to the person information of the speaker stored in the recognition range storage unit and the recognition range of the speaker with respect to the object of the ambiguous expression are obtained, and the ambiguous information about the person information of the listener stored in the recognition range storage unit A recognition range estimation unit for obtaining a recognition range of the listener for the expression phrase and the ambiguous expression target;
A recognition range difference detection unit that compares the speaker's recognition range with the listener's recognition range to determine a difference between the speaker and the listener's recognition range;
A dialogue support apparatus characterized by comprising: The dialogue support apparatus according to claim 1, wherein the ambiguous expression phrase is an adjective phrase or an adverb phrase. The dialogue support apparatus according to claim 1, wherein the ambiguous expression target is a dependency destination. The dialogue support apparatus according to claim 1, wherein the ambiguous expression target is a noun phrase. The dialogue support apparatus according to claim 1, wherein the object of ambiguous expression is a word string before and after an ambiguous expression phrase. The dialogue support apparatus according to claim 1, wherein the recognition range is represented by a numerical value. The dialogue support apparatus according to claim 1, wherein the recognition range is represented by statistical information. The dialogue support apparatus according to claim 1, further comprising a recognition range adjustment unit that adjusts the recognition range of the speaker and the recognition range of the listener based on one of the standards. The dialogue support apparatus according to claim 1, wherein the speaker or the listener is warned when the difference in the recognition range is not less than an arbitrary standard. In a dialogue support method that mediates communication between the speaker and the listener,
Recognizing the voice of the speaker,
Storing a list of ambiguous words that cause misunderstandings and misconceptions between the speaker and the listener;
Comparing the stored ambiguous expression word list with the speech recognition result to detect an ambiguous expression phrase including the ambiguous expression word from the speech recognition result;
Detecting an ambiguous expression target in which the interpretation of the listener is ambiguous by the ambiguous expression phrase from the speech recognition result;
Identifying the person information of the speaker and the person information of the listener,
For the combination of the ambiguous expression phrase and the ambiguous expression object, store the recognition range of the ambiguous expression for each person information,
The ambiguous expression phrase related to the stored person information of the speaker and the recognition range of the speaker for the object of the ambiguous expression are obtained, and the ambiguous expression phrase related to the person information of the listener stored in the recognition range storage unit and the ambiguous expression Obtaining the listener's perception range for the expression object;
A dialogue support method, comprising: comparing a recognition range of the speaker with a recognition range of the listener to obtain a difference between the recognition range of the speaker and the listener. In a dialogue support program that mediates communication between the speaker and the listener,
A voice recognition function for recognizing the voice of the speaker;
An ambiguous expression word list storage function that stores a list of ambiguous expression words that cause misunderstandings and misconceptions between the speaker and the listener;
An ambiguous expression detection function for comparing the ambiguous expression word list with the speech recognition result and detecting an ambiguous expression phrase including the ambiguous expression word from the speech recognition result;
An ambiguous expression object detection function for detecting an ambiguous expression object whose interpretation of the listener is ambiguous by the ambiguous expression phrase from the speech recognition result;
A speaker identification function for identifying the person information of the speaker and the person information of the listener;
A recognition range storage function for storing a recognition range of the ambiguous expression for each person information for the combination of the ambiguous expression phrase and the ambiguous expression target;
The ambiguous expression phrase related to the person information of the speaker stored in the recognition range storage function and the recognition range of the speaker for the object of the ambiguous expression are obtained, and the ambiguous information about the person information of the listener stored in the recognition range storage function A recognition range estimation function for obtaining a recognition range of the listener for the expression phrase and the ambiguous expression target;
A recognition range difference detection function for comparing the speaker's recognition range and the listener's recognition range to obtain a difference between the speaker and the listener's recognition range;
A dialogue support program characterized by

Images