JP2003131692A - Interactive device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make the device to be easily used by various types of people including children, adults, elderly people and persons who are not familiar with the system or feel uneasiness while using the system and to change output information in accordance with the needs of the user so that the system becomes easy to be used in accordance with the frequency of utilization and the degree of familiarity. SOLUTION: The interactive device is provided with a portion which successively controls output information of the system, a portion which halfway stops the output information of the system, a portion which changes the output information of the system and a table which holds output change patterns used to change the output information of the system.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a dialog device for performing a voice dialog with a user, and more particularly to a dialog device using a multimodal interface.

[0002]

2. Description of the Related Art Generally, when using an interactive interface between a user and a system, a response function suitable for the user and a learning function for the user, and a general system such as "guidance system", "secretary system", "reception system", etc. It is necessary to have a function for realizing task execution, an input means such as voice / gesture and its recognition function, and an output means using an anthropomorphic agent and its output function.

Conventionally, in order to realize such a function,
A voice input and a voice recognition function with a microphone, a face image input function using an image input with a camera, a dialogue management function with a rule base, a method of using a script of a dialogue flow prepared in advance are used. Such a system recognizes information from one or more input means and controls it integrally according to the progress of the corresponding task. The control results are output to the user using one or more output means.

The problem here is that there are various types of users who use the dialogue system, such as children, adults, and elderly people who are not immediately familiar with the system or who find it difficult to use. Is. In other words, the dialogue system is aimed at an unspecified number of users,
Depending on the user, the way to deal with it varies. Moreover, the operation method changes even for the same user depending on the frequency of use and familiarity. Therefore, the interactive system must be able to change the form of the input means / output means and the response speed according to the user's preference. Furthermore, it must provide a function that can be changed into a form that suits the user's taste according to the frequency of use and familiarity.

When such a function is lacking, the response of the system does not change to any user and always responds to a certain amount of time. Therefore, it is difficult to use, the frequency of use and familiarity inherent in the system. It will force users to use it. That is, it is natural that it is possible to provide a certain function and usability by using a voice, gesture, anthropomorphic interface, etc. in the dialogue system, and it is required to adapt to the user's usage pattern of the above problem. . When the dialog system makes some kind of reply to the user, it is important for the user interface to be appropriately communicated as a human interface that is easy to use from the viewpoint of the user.

[0006] As a conventional technique of the dialogue system, NT
Dialogue system "Computers that can exchange flexibly in spoken dialogue; Spoken dialogue system DUG-1" developed by T Communication Science Laboratories (JTT magazine 2
December 2000 issue Vol. 23, No. 12; Advanced Technology Co., Ltd.), a natural interaction is realized by using a technique called a sequential understanding type / sequential generation method. Here, when the system receives an input from the user during the output, the output is stopped and a new input from the user is dealt with.
That is, the processing flow is as shown in FIG.

In the prior art shown in FIG. 13 (a), while the dialogue system is outputting the content of "I have a meeting with the personnel manager of Hasegawa on Wednesday", the user is asked "What is Kawasaki?" The case is shown. However, "Wednesday", "Hasegawa Human Resources Manager", "Meetings", "Meeting with Kawasaki Manager",
It is assumed that the user is checking whether or not there is any input in the unit “4 pm to 5 pm”. Hereinafter, these units will be referred to as unit input information when inputting and unit output information when outputting.

In the conventional dialogue system, while the unit output information "Hasegawa Human Resources Manager" is being output,
If the user interrupts the input "What is Kawasaki?", The output "I have a meeting." Is canceled,
Creates an Azuchi "Yes". After that, after interpreting the interrupted user information, output is made for each unit output information such as "meeting with Kawasaki section manager" and "from 4 pm to 5 pm". There is no “yes” agreement inside, and “just having a meeting with manager Kawasaki”
There is also an example of outputting "4 pm to 5 pm".

FIG. 14 is a block diagram showing a conventional system, FIG. 15 is a flowchart showing the processing of the conventional system, and FIG. 16 is a diagram showing an example of an output pattern table. In FIG. 14, a user information input unit 1401 performs pre-recognition processing and recognition processing based on input information such as a voice signal and an image input by a user. In the recognition process, for example, in voice recognition, the recognition result is not output for each word or each user utterance unit, but is output even when the input signal is partially recognized. That is, waiting for the recognition time of the input signal,
The partial recognition result is output at a certain time interval or a certain range of time intervals so that the subsequent processing is not delayed.

As a speech recognition apparatus which outputs a partial recognition result at a certain time interval or a time interval within a certain range, many methods have been proposed and developed so far. Among them, for example, Japanese Unexamined Patent Publication (Kokai) No. 5-197389 discloses a "speech recognition device" capable of coping with utterances of meaningless words and natural utterances such as continuously uttering a plurality of sentences. Japanese Patent Laying-Open No. 3-191400 discloses a "speech recognition device" that obtains high speech recognition performance by using a discrete HMM (discrete hidden Markov model).

Further, as a gesture recognition device, several methods have been proposed and developed so far, and a typical method among them is, for example, Japanese Patent Laid-Open No. 6-1249.
As disclosed in Japanese Patent Publication No. 3, a computer that detects a gesture using DP (dynamic programming; dynamic programming) matching from a time-series movement of a mouse or a pen and executes a command corresponding to the gesture. A user interface "gesture recognition method and user interface method" has been proposed, and is disclosed in Japanese Patent Laid-Open No. Hei 1
In Japanese Patent Laid-Open No. 0-162151, a time difference image is acquired from a captured image of a gesture of a subject, binarized, and a feature vector for a feature pattern is acquired from a feature amount of the difference image using continuous DP. Recognition method ”has been proposed. Further, Japanese Patent Laid-Open No. 2000-149025 discloses a "gesture recognition device" which is suitable for recognizing a user's gesture whose position / orientation freely changes and for operating a CG (computer graphic) object in a virtual space. And method ”have been proposed.

The system output information generation unit 1402 generates the output information which is the response content of the system based on the partial or whole recognition result of the user input information received from the user information input unit 1401 and outputs the output pattern table 1404. Refer to the information in. When the user's input information includes a plurality of pieces, the recognition results are integrated by the system output information generation unit 1402. For example, in the "integrated recognition dialogue device" disclosed in Japanese Patent Laid-Open No. 8-234789 and the "multimodal information integrated analysis device" disclosed in Japanese Patent Laid-Open No. 9-114634, information in a multimodal dialogue database is stored. Originally, we proposed a method to handle inputs from multiple input channels in an integrated manner.

The system information output unit 1403 outputs the output information sent from the system output information generation unit 1402. Generally, when outputting system information using voice or anthropomorphic agent, an output device such as a speaker or a monitor is required. System output information generation unit 14
When the output stop command is transmitted from 02, the system information output unit 1403 immediately stops the output of the system information. The state in which the output stop command is transmitted means that the system is outputting information in the system information output unit 1403,
Input from the user is input to the user information input unit 1401,
This is a case where the system output information generation unit 1402 outputs information corresponding to the user input information.

A specific description will be given with reference to the flowchart shown in FIG. In step S1501, the user information input unit 1401 checks whether there is an input from the user. The input from the user is information from the user, which is a target of input by the system, suggesting voice, gesture, line of sight, and the like. The check continues until there is input from the user. If there is an input from the user, step S1
At 502, the input information is analyzed. This corresponds to voice recognition in the case of voice input or gesture recognition in the case of gesture input from a CCD camera or the like.

In step S1503, the system output information generation unit 1402 refers to the output pattern table 1404 to calculate unit output information according to the input information,
Prepared for output. For example, in the case of an interactive system in which a system input “Ohayo” is generated in response to a user input “Ohayo” in the reception guidance system, “Ohayo” output by the system corresponds to that. That is, the generation of the system's response to user input. These pieces of output information are output in step S150.
After the output completion check of No. 4, the system information output unit 1403 outputs the data for each unit in step S1505. These steps S1504 and S1505 take into consideration the case where the output contents are not output all at once but are output in a certain unit (sentence sentence, clause / phrase unit). In the example above, "Wednesday", "Hasegawa Human Resources Manager", "Meetings", "Meeting with Kawasaki Manager", "4 pm to 5 pm". . That is, it is output for each unit output information.

Information partially output (step S15)
When the output of all of 05) is completed, that is, when it is determined Yes in step S1504, the information input check from the user is performed again in step S1501. In step S1506, the user information input unit 1 is in the process of partially outputting information in step S1505.
At 401, it is determined whether or not there is information input from the user. In the above example, the interrupt input from the user "What is Kawasaki section manager?" If there is no interrupt input from the user, the process returns to step S1504, and it is determined again whether all output information has been output. If there is an input from the user, step S150
At 7, the system output information generation unit 1402 stops the output information. In the above example, this is equivalent to canceling the output "I have a meeting." afterwards,
Returning to step S1502, the information input by the user is analyzed and output is prepared (step S1503). In the above example, after outputting the "Yes" Azuchi,
"Meeting with Manager Kawasaki", "4 pm-5 pm
Until time. Is output.
The output of the "yes" azuchi depends on the design of the dialogue system. There are many methods, such as a system that does not perform the output of a hammer and an interrupt input from the user.

FIG. 16 shows the contents of the system output pattern table 1404 corresponding to the user's input information. For example, in T1601, it is indicated that the system returns a reply of "Good morning." To the user input "Good morning." T160
2, T1603, in the same way even T1604, the output of "Hello." With respect to the input of "Hello.", The output of "Good evening." With respect to the input of "Good evening.", To the input of "goodbye." On the other hand, it indicates that the output “Goodbye.” Is performed.

T1605 responds to the user's input information "Good night" by saying "Good night." At T1606 and T1607, the user's input is the same as "How are you?"
The output pattern is different, saying "I'm fine." This is how to build a dialogue system,
That is, it depends on how the response pattern is used. For example, when switching between "I'm fine." And "I'm not very well." Based on a random value, or on the basis of a certain variable (such as a large number of interactions), "It's possible to switch between "I'm fine." And "I'm not very well." Or to use both methods together. This depends on the design of the interactive system.

At T1608, the system responds to the user's input information "I'm tired.""Let's do our best." In T1609, the weather information is output in response to the user's input information “How is the weather?”. This [$ WEATHER] is a variable indicating the weather information at that time. For example, [sunny], [cloudy],
Of course, it is possible to output random information such as [rain], but usually, it is often the case that weather information at that time is acquired and output from the Internet. Some dialog systems do not have such a response example.
It depends on the design specifications of the dialogue system. T1610 indicates that the system responds when there is no user input for a while (NIL), "There is no reply." The time interval without user input depends on the design specifications of the interactive system. It may be fixed or variable.

Note that this table can also be expressed using the concept of state transition (automaton). In this case, generation of an advanced response, such as making a specific response (when the system output "Ohayo." Has already been made, the response "Good evening" is not returned) is made only at the time of a certain transition. Will be possible.
This is a general extension method and depends on the design method of the interactive system. The above is a typical process of the conventional interactive system.

[0021]

However, in such a conventional technique, when the multimodal input information such as voice and gesture is integrated and presented to the user, the system uses the input from the user as a reference, and the user uses the system. It is assumed that the response is based on the output of. That is, processing is performed in the cycle of input from the user, response from the system, input from the user, or response from the system, input from the user, response from the system as a unit, during the response from the system or from the user. It does not consider interrupting while inputting.

However, there is an introduction of a technique in which the output of the system is stopped when the user interrupts the input of an utterance during the output of the system. However, basically, just by stopping the output process of the unit output information, the flow of the dialogue changes (interrupts) by being interrupted (interrupted).
I do not consider it.

As described above, the prior art also describes a method of integrating multi-modal information and feeding back the result to the user through voice synthesis, gesture, etc. of the CG agent, but the utterance is changed. It is fixed in units and bunsetsu, and there is no description about the technique of changing it during utterance. Therefore, redundant information exchange (conversation by voice, gesture, etc.) may occur between the user and the system. In particular, it is difficult for a person who is new to the system, a child, an elderly person, or other unfamiliar person to operate the system.

The present invention has been made in view of the above points, and can be applied to children, adults, old people, etc. who use the dialogue system.
Make it easy for various types of people, such as people who are not familiar with the system immediately or people who find it difficult to use, and change the output information according to the user to make it easier to use depending on the frequency of use and familiarity. It is an object of the present invention to provide a dialogue device capable of performing.

In order to solve the above-mentioned problem, it is necessary to consider a mechanism of changing the utterance content during the utterance of the system. Specifically, a device that makes a response as shown in FIG. 13B is desirable. In other words, while the unit output information "Hasegawa Human Resources Manager" is being output, if the user is interrupted by the input "What is Kawasaki Manager?", Stop the output "There is a meeting." No, there is a meeting in order to make a natural response, including the result of interpreting the interrupted user information, "There is a meeting with Mr. Kawasaki," and "4 pm
From 05:00 to 5pm. It is desirable to change part of the unit output information, such as ". If you simply stop what the system was trying to output, the dialogue would be choppy, making it difficult to use and difficult to interact with.

If the user output is changed, the unit output information "There is a meeting" is changed,
If the processing time to change to the content "There is a meeting," is not in time, or if the output content "Meeting already" has already been output to the middle, that is, the position where the output stop position is assumed. When the deviation occurs, it is desirable that the unit output information can be changed again as shown in FIG. It is desirable to make changes such as "Meetings are held.", "Others,""Meetings with Manager Kawasaki" and "From 4 pm to 5 pm."

Such a change in the response to the user's interrupt input is often the first time the user uses the interactive system. In other words, when the system is first contacted with the system, the user does not know how to use the system and how to interact with the system. Therefore, the timing of utterance change between the system and the user should be smooth. I can't. In addition, for a user who is accustomed to using the conversation, the conversation becomes messy, and the user often ignores the exchange of the utterance change, such as suddenly giving a message. The present invention is effective in such a case.

[0028]

SUMMARY OF THE INVENTION The interaction device of the present invention comprises at least one of a user gesture or a user utterance.
One as input, means for generating output information for responding to a dialogue with the user for each unit output information based on the recognition result of the input, and generated output information for each unit output information. And a means for outputting to the user, the dialogue device performing a dialogue with the user, when a new input is made during the output of the unit output information to the user from the dialogue device, recognition of the new input It is characterized by comprising means for changing the unit output information based on the result. When the system receives an input from the user during the output of the unit output information, the unit output information is changed during the output so as to correspond to the new input from the user, which enables a natural response. Further, it is possible to provide an interactive device capable of changing output information according to various users.

Further, in the interactive apparatus of the present invention, the means for changing the unit output information to the user comprises means for stopping the output information on the way, and an output for changing the output information to the user. A table for holding change patterns is provided, and means for sequentially managing output information for the user is provided. The means for changing the output to the user is the recognition result of the new input and the output status of the unit output information. The unit output information is changed based on the following.

The means for stopping the output information on the way enables the information output by the system to be stopped at natural intervals. In addition, since a table holding an output change pattern for changing the output information for the user is provided, the stop position (time) when the output information of the system is set,
It becomes possible to retain the contents that change the output information of the system in a natural form. In addition, the means for sequentially managing the output information makes it possible to determine and manage how much the output information generated by the system is output at a certain time, and when the output information is stopped, how much is output. , You can determine what is not output.

In the interactive apparatus of the present invention, when the unit output information to the user is changed based on the recognition result of the input, the actual output is the output stop schedule calculated by the means for stopping the output information on the way. If the position has already passed, the means for sequentially managing the output information newly generates information on the actual stop position, and whether the difference between the actual stop position and the planned output stop position exceeds a threshold value. It is characterized in that the content of the unit output information is changed depending on whether or not the unit output information is changed.
Even if the estimated output stop position calculated by the dialogue device has already passed, by changing the unit output information as appropriate to the extent that it passed, the system can be immediately used by children, adults, old people, etc. who use the dialogue system. It is possible to provide an interactive interface that is easy for various types of people, such as people who are unfamiliar and people who find it difficult to use.

Further, the interaction method of the present invention includes the step of recognizing at least one of the user's gesture or the user's utterance as an input, and output information for responding to the interaction with the user based on the recognition result of the input. Is a unit of output information, and a step of outputting the generated output information for each unit of output information. If a new input is made during the output of information,
It is characterized by including a step of recognizing the new input and a step of changing and outputting the unit output information based on a recognition result of the new input.

Further, the method further comprises a step of sequentially managing output information for the user, and the step of changing and outputting the unit output information is based on the recognition result of the new input and the output status of the output information. It is characterized in that the output information is changed. The present invention can also be realized as a program for causing a computer to function as the dialog device of the present invention and a computer-readable recording medium recording the program.

[0034]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. (Embodiment 1) FIG. 1 is a block diagram showing a configuration of an interactive apparatus according to Embodiment 1 of the present invention. As shown in FIG. 1, the dialogue apparatus according to the present embodiment includes a means for sequentially managing the output information of the system, a means for temporarily stopping the output information of the system, a means for changing the output information of the system, and an output information of the system. It is characterized in that it is provided with a table holding an output change pattern for changing. Specifically, in this configuration, the system output information sequential management unit 105,
The system output information stopping unit 106, the system output information changing unit 107, and the output change pattern table 108 are provided. These correspond to the means for changing the unit output information described in claim 1. Also, the system output information changing unit 10
7 functions as a control unit of a unit that changes the unit output information.

Thus, in the dialog device having the function of recognizing at least one of the gesture of the user or the utterance of the user as an input, and in the course of outputting the unit output information from the dialog device to the user, in the dialog device. It is possible to recognize the input and change the output of the unit output information to the user based on the input recognition result.

FIG. 2 is a flow chart showing the processing of the dialogue apparatus of the present invention. The process in the dialog device of the present invention is
2 and FIG. 15, the output is stopped (S1507 in FIG. 15), the output is changed (step S208 in FIG. 2), and the output (step S20 in FIG. 2) is performed.
It is expressed by the difference with 9).

In the processing flow shown in FIG. 15 which is a conventional technique, when the system receives an input from the user while the unit output information is being output (step S1506), step S1.
At 507, the output of the unit output information is stopped. On the other hand, in the present invention, when the system inputs the user during the output of the unit output information (step S206),
The input is analyzed (step S207), and the output content of the unit output information is changed (step S208). The changed output content is output in step S209. Figure 2
S201 to S20 of the processing flow shown in FIG.
6 and steps S1501 to S1506 of the processing flow shown in FIG. 15 are the same. Note that step S
Details of 208 will be described below in the present embodiment.

In FIG. 1, the user information input unit 101 performs pre-recognition processing and recognition processing based on input information such as a voice signal and an image input by the user, and the recognition result is sent to the system output information generation unit 102. Output. Also,
This recognition processing outputs even if the input signal is partially recognized. The system output information generation unit 102 generates the output information, which is the response content of the system, for each unit output information based on the partial or entire recognition result of the user input information received from the user information input unit 101. . When the user's input information extends over a plurality of pieces, the respective recognition results are integrated by the system output information generation unit 102. For example, "Integrated weaving dialogue device" disclosed in Japanese Patent Laid-Open No. 8-234789, Japanese Patent Laid-Open No. 9-114
The "multimodal information integrated analysis device" disclosed in Japanese Patent No. 634 proposes a method of integrating and handling inputs from a plurality of input channels based on information in a multimodal dialogue database.

However, the present invention differs from these techniques in that the system output information generation unit 102 generates a new system response by inputting the change information of the system output information from the system output information change unit 107. The point of sending an output stop command to the system output information stop unit 106. By sending the stop command to the system output information stopping unit 106, the system output information generating unit 102 can receive new unit output information obtained by changing the unit output information output from the system from the system output information changing unit 107. It is possible to avoid output of redundant system information.

The system information output unit 103 outputs the output information sent from the system output information generation unit 102 for each unit output information. The state of this output information, that is, how much information has been output at a certain time or in a certain range of time, including the position of the unit output information, is sent to the system output information sequential management unit 105. Generally, when outputting system information using voice or anthropomorphic agent,
Output devices such as speakers and monitors are required. The same applies to the present invention. Further, the present invention is characterized in that it detects to what stage the information to be always output is actually output, and that the detected output information position is transmitted to the system output information sequential management unit 105. is there.

When a stop command is transmitted from the system output information sequential management unit 105, the system information output unit 103
Immediately stops outputting system information. Then, the stop position is transmitted again to the system output information sequential management unit 105. In the system output information sequential management unit 105, based on the output information of the system information output unit 103, up to which part of the unit output information the system output information is output, per unit time, or time within a certain range. The output information is managed at intervals.

The system output information sequential management unit 105 manages the output information position transmitted from the system information output unit 103 and, at the same time, transmits the output information position to the system output information stop unit 106. System output information stopping unit 106
When the stop command is sent from, the stop command is immediately transmitted to the system information output unit 103. The output information position and the stop command here extend to all types of system output information handled by the system output information generation unit 102. That is, control is performed for all output means such as voice, gesture, window in monitor screen, and the like.

The system output information stopping unit 106 stops the output of the system information based on the output stop command from the system output information generating unit 102. At this time, the stop command is not immediately transmitted to the system output information sequential management unit 105. Using the information of the output change pattern table 108,
If the calculated position becomes equal to the actual output position after the stoppable position is calculated, or if it is determined that the actual output position has been output before the calculated position. , The stop command is sent for the first time. For example, if it is calculated in FIG. 13 that “meeting is available” is a position where it can be stopped, even if “meeting is available” is output, it may be possible to change it as shown in FIG. 13 (b). The possibilities are dependent on the system design and performance. If the actual output position is less than the calculated position, the stop command is not issued and the system information continues to be output.

The system output information changing unit 107 searches the output change pattern table 108 based on the stop position information sent from the system output information stopping unit 106,
The output information to be changed is calculated. The calculated change information and stop position information are used again for the system output information generation unit 10
Sent to 2. Here, the change information includes not only the voice but also the gesture. For example, if you change "I have a meeting" to "I have a meeting,"
The movement of the mouth corresponding to the pronunciation of "ga" is also changed. The system output information stopping unit 106 knows the stop position information in advance, so the system information outputting unit 1
Even when the system information is being output at 03, the stop position information can be sent to the system output information changing unit 107. That is, the system information output unit 103, the system output information sequential management unit 105, and the system output information changing unit 107 can be operated in parallel, and the output information of the system is immediately output when the output information position reaches the output stop position. Can be changed.

The system output information generation unit 102 again generates new output information for each unit output information based on the change information and the stop position information sent from the system output information change unit 107. The result is sent to the system information output unit 103 as before. The system information output unit 103 outputs the output information sent from the system output information generation unit 102 for each unit output information.

The output change pattern table 108 holds output information such as a part or all of the output utterance contents and the change information at that time, and the change information according to whether or not the output can be stopped and the content of the stop. Search and generate. These pieces of information cover output information in general and include not only utterance information (voice) but also display information on gestures, monitor screens, and the like. When there are a plurality of output stoppable parts, it is possible to have a plurality of pieces of change information.
Further, when the data is output up to the position beyond the stoppable position, the contents of the unit output information can be changed depending on whether or not the threshold value is exceeded, as in the second embodiment. Next, the operation of the dialogue device having the above configuration will be specifically described.

FIG. 3 is a flow chart of processing in the system output information generation unit 102. The process here is
User input recognized by the method shown in "Voice recognition device" disclosed in Japanese Patent Laid-Open No. 5-197389 and "Gesture recognition method and user interface method" disclosed in Japanese Patent Laid-Open No. 6-12493. This is a part that generates a system response using information.

In step S301, the user information input unit 10
The user input information recognized from 1 is read. In step S302, system response generation is performed for each unit output information based on the user input information recognized in step S301 or based on the system response change information obtained in step S306. At this time, if the output stop determination is made in step S303, a stop command is transmitted to the system output information stop unit 106 in step S305. Then, in step S306, the system output information changing unit 107
The change information is read from.

However, if it is not determined that the output is stopped in step S303, the system output information is transmitted to the system information output unit 103 in step S304. Then, the process returns to step S301 again. If there is no system response change information sent from the system output information changing unit 107 in step S302, the system response is generated only with the information input by the user. The criterion for issuing the output stop command in step S303 is whether or not the user input information in step S301 is an interrupt input from the user.

FIG. 4 is a flow chart of processing in the system information output unit 103. In step S401, the system output information sent from the system output information generation unit 102 is read, and in step S402, the system output information sequential management unit 105 reads whether or not a stop command has been transmitted. If it is determined in step S403 that the stop command has been transmitted, the output information is immediately discarded (step S407), and the system output information is read again in step S401.

When it is determined that the stop command has not been transmitted, the position of the information that is actually output is sent to the system output information sequential management unit 105 in step S404, and then the speaker output in step S405. It is output using an output device such as a monitor. In step S406, it is determined whether or not all pieces of output information that have been output in pieces have been output. If output information still remains, the process returns to step S403 again, and whether a stop command has been transmitted. Is determined. If all are output,
Returning to step S401, the system output information is read again. That is, when the output information sent from the system output information generation unit 102 is continuously output, it is sequentially determined whether there is output information, whether a stop command has been sent, and whether all output information has been output. The processing flow is as follows.

FIG. 5 shows the system output information sequential management unit 10
6 is a flowchart of the process in 5. Step S5
In 01, the stop command sent from the system output information stop unit 106 is read. The output information stop position is also read. If the stop command has been transmitted (step S5)
02), and sends a stop command to the system information output unit 103 together with the output information stop position (step S506). Then, the process again returns to step S501. If it is determined in step S502 that the stop command has not been transmitted, the output information position transmitted from the system information output unit 103 is read in step S503. The information is managed as an output information position in step S504. This managed output information position is used in step S505, and the output information position is transmitted to the system output information stop unit 106. Then, the process again returns to step S501.

The main processing contents of the system output information sequential management unit 105 are management of output information actually output and transmission of a stop command. The system output information sequential management unit 105 does not generate the stop command by itself, and based on the transmission from the system output information stop unit 106, transmits the stop command together with the output information stop position to the system information output unit 103.

FIG. 6 is a flow chart of processing in the system output information stopping unit 106. Step S601
Then, the stop instruction transmitted from the system output information generation unit 102 is read. If it is determined in step S602 that the stop instruction has not been transmitted, step S608.
Then, the output information position transmitted from the system output information sequential management unit 105 is read. When it is determined that the stop command is transmitted, the position where the output information is stopped is calculated based on the current output information position in step S603. The calculation of the stop position is performed using the information held in the output change pattern table 108.

In step S604, the output information position currently being output is read, and in step S605, it is compared with the calculated actually stoppable information position. If the current output information position is less than the stop possible position, the process returns to step S604 again, and the system information output unit 103 continues to output the system information until the stop possible position is reached.

When the current output information position becomes the stoppable position, a stop command for actually stopping the output information is issued in step S606. The stop command is transmitted to the system output information sequential management unit 105, and from there, again,
After being transmitted to the system information output unit 103, the output information is actually stopped. After the output information stop command is issued in step S606, the stop position is transmitted to the system output information changing unit 107 in step S607.

Whether or not the stop position has been reached is determined, for example, when a part of the keywords of the output information becomes equal to the keyword corresponding to the stop possible position, or the voice waveform itself and the movement information of the gesture match or a certain threshold value. The judgment can be made in the case of the following range, that is, in the case of no coincidence and in an approximate state within a certain criterion.
This is already implemented as part of the recognition algorithm, and as a simple method, it can be easily done by comparing character strings (such as how many character strings match and how many characters match). It is possible.

FIG. 7 is a flow chart of processing in the system output information changing unit 107. Step S701
Then, the stop position information transmitted from the system output information stop unit 106 is read. When the reading of the stop position information is not successful (when the stop position information has not been transmitted), the process returns to step S701 again.

If the reading of the stop position information is successful in step S702 (if the stop position information is transmitted), the output change pattern table 1 is output in step S703.
08, the change information regarding the unit output information of the system is calculated. For example, in the case where the input is interrupted by the user while the unit output information “There is a meeting” is being output in FIG.
As shown in (b), change to “There is a meeting,”. Alternatively, as shown in FIG. 13 (c), it is possible to change "we have a meeting" and "others". The change information is transmitted to the system output information generation unit 102 in step S704. At this time, the system output information generation unit 102 reads the change information regarding the unit output information sent from the system output information change unit 107 in step S306. Then, step S30
Returning to 2, the system response is generated again. After the change information is transmitted in step S704, the process returns to step S701 again.

FIG. 8 shows the output change pattern table 108.
It is a figure which shows the example of. In FIG. 8, the item “stop information” on the left side shows information scheduled to be output by the system,
The item “change information” on the right side shows the contents to be transmitted to the system output information generation unit 102 after the output is stopped.

In the "stop information", items corresponding to the type of output information of the system are listed. It is also possible to determine the unit output information in units of this size, but here,
The explanation will continue with a unit with a slightly larger unit such as "There is a meeting." Further, in FIG. 8, it is assumed as an example of an implementation in which a voice / gesture is used as an output expression means. In T801, "Isn't it?"
When the output information "is used as the stop information, it is converted into the output information"". Here, not only the voice but also the movement of the mouth due to the difference in pronunciation may be changed, and in addition, the gesture of the limb may be changed. T8
Similarly, in 02, "?" Is converted into output information "Is it right?" Also in T803, "I am doing." Is changed to "I am doing."

T 804 indicates that the output information “I have it” is changed to the information “I have it”. This corresponds to the case shown in FIG. In T805, the output information of "I will investigate." Is converted into the output information of "I am investigating. Wait a while." This means that the output is not interrupted in the form of "inspection" and the information "Please wait a while" is added. The same applies to T806. Do not block the output information "I think.", Add the information "Yes.", And then change it to "I think."

T807 defines a pattern indicating a reaction such as a response to a call. That is, the output "How is it?" Is changed to "Yes. I understand." This means that the system responds to the input from the user before or during the output of "How is it?", Which is an example of the system output that properly responds to the input from the user.
Means to reply to "Is it okay?" This is an example to avoid outputting redundant questions to the user.

On the other hand, T809 has the content "#",
It means the change content of "other,". this"#"
Is for the case where the system tried to stop the output before or during the output of the response but the stop process or the response generation process was not in time. By adding, we intend to make the connection with the input information interrupted by the user a natural form. This corresponds to the case shown in FIG. T810 has a content of "*", which indicates that all the content matches. That is, when the stop command is transmitted, the output is immediately stopped and converted into the output information "I'm sorry." It is used when the stop information does not exist in the output change pattern table 108.

According to the dialog device described in this embodiment, the following effects can be obtained. By providing a unit for changing the unit output information based on the recognition result of a new input, when the system receives an input from the user during the output of the unit output information, the unit output information is changed during the output. By responding to new input from the user, a natural response is possible. Further, it is possible to provide an interactive device capable of changing output information according to various users.

By providing means for sequentially managing output information from the system such as utterances / operations of the system, it is possible to judge and manage how much output information generated by the system is output at a certain time. Become.
Then, when the output information is stopped, it is possible to determine how much is output and what is not output.

Further, by providing means for stopping the output information of the system on the way, it becomes possible to stop the information output by the system at natural intervals. A natural break is not only in the middle of a word, but between words
It is to divide at the interruption position (time) such as the stop of the utterance that is usually done in conversation between humans. Also, by providing a table that holds the output change pattern for changing the output information of the system, the stop position (time) when changing the output information of the system and the contents of changing the output information of the system can be set in a natural manner. It becomes possible to hold.

Further, the means for changing the output information of the system functions as the control portion of the means for changing the unit output information based on the recognition result of the new input, which has been performed by the system output information generating portion so far. The output information change process can be divided on the hardware, and the output position change pattern for changing the output position of the system and the stop position (time) of the means for sequentially managing the output information of the system It is possible to change the system output contents in any form.

As a result, it is possible to provide a dialog interface that is easy to use by various types of people, such as children, adults, and elderly people who use the dialog system, those who are not immediately familiar with the system, or those who find it difficult to use. Become.
Further, it becomes possible to provide an interactive system in which output information can be changed according to the user so that it can be easily used depending on the frequency of use and familiarity.

The dialogue apparatus of the present invention is not limited to the above-mentioned illustrated example, and various changes can be made without departing from the scope of the present invention. For example, the system output information sequential management unit 105 may be omitted and the system output information stopping unit 106 may be provided with that function, or the information of the output change pattern table 108 may be stored in the system output information stopping unit 106 and the system output information changing unit 107. It is also possible to have each of them. In that case, the table information may be the same information or different information having the same purpose. However, each information needs to be consistent data so that the information of the system can be stopped or changed.

(Embodiment 2) In the present embodiment, when the actual output when changing the output of the unit output information to the user has already passed the scheduled output stop position calculated by the dialog device, A case where the output of the unit output information is changed will be described.

FIG. 9 is a block diagram showing the structure of the dialogue apparatus according to the second embodiment of the present invention. 9, the same parts as those in FIG. 1 are designated by the same reference numerals as those in FIG. 1 and their detailed description is omitted. In the dialog device shown in FIG. 9, the system output information stopping unit 901 to the system output information changing unit 9
02, that is, information on the actual stop position is newly added. With the addition of this transmission information, the processing flow in the system output information generation unit 903, the system output information stop unit 901, and the system output information change unit 902 is different from that in the first embodiment.

In the dialog device shown in FIG. 9, in the system output information stop unit 901, when the output information position to be stopped is different from the actually output information, the planned stop position and the actual stop position are set. It is changed so as to be transmitted to the system output information changing unit 902. At this time, the system output information generation unit 903 is additionally provided with a process of reading the actually stopped position and change information.

FIG. 10 shows the system output information generator 903.
3 is a flowchart showing the processing in FIG. Step S
At 1001, the input information of the user is read from the user information input unit 101. In step S1002, system response generation is performed for each unit output information based on the actual system output stop position information and the change information obtained in step S1006. At this time, if the output stop determination is made in step S1003, a stop command is transmitted to the system output information stop unit 901 in step S1005. Then, in step S1006, the system output information changing unit 90
The actual system output stop position information and change information are read from 2. However, if it is not determined that the output is stopped in step S1003, step S100
At 4, the system output information is transmitted to the system information output unit 103. Then, the process returns to step S1001 again.

FIG. 11 shows the system output information stopping unit 901.
3 is a flowchart showing the processing in FIG. Step S
At 1101, the stop instruction transmitted from the system output information generation unit 903 is read. If it is determined in step S1102 that the stop command has not been transmitted, the output information position transmitted from the system output information sequential management unit 105 is read in step S1109. When it is determined that the stop command is transmitted, step S11
At 03, the position where the output of the unit output information is stopped is calculated based on the current output information position. To calculate this stop position,
This is performed using the information held in the output change pattern table 108.

In step S1104, the output information position currently being output is read and compared with the actually stoppable information position calculated in step S1105. If the current output information position is less than the stoppable position, the process returns to step S1104 again until the stoppable position is reached.
The system information output unit 103 continues to output the system information. When the current output information position has reached the stoppable position, a stop command for actually stopping the output information is issued in step S1106. The stop command is transmitted to the system output information sequential management unit 105, is transmitted again to the system information output unit 103, and then the output of the unit output information is actually stopped.

After the output information stop command is issued in step S1106, the stoptable position (scheduled stop position) is output in step S1107, and the information of the actually stopped position is sent to the system output information changing unit 902 in step S1108. Sent. Then, the process returns to step 1101 again.

FIG. 12 shows the system output information changing unit 902.
3 is a flowchart showing the processing in FIG. Step S
At 1201, the stop position information (scheduled stop position; expressed as type A) transmitted from the system output information stop unit 901 is read. When the reading of the stop position information is not successful (when the stop position information is not transmitted), the process returns to step S1201 again. If the reading of the stop position information is successful (when the stop position information is transmitted) in step S1202, the stop position information (actual stop position; expressed as type B) is read in step S1203.

In step S1204, the planned stop position and the actual stop position are compared using the two stop position information (A and B). If the difference between the two stop positions at this time is larger than a certain threshold value, step S12.
At 06, change information (type X) is calculated. If it is not larger than a certain threshold, change information (type Y) is calculated in step S1205. For example, in the case where the input is interrupted by the user while the unit output information "There is a meeting" is being output in FIG. However, if the threshold value is exceeded,
As in (c), change to "I have a meeting." And "Other". It should be noted that it is possible to set a threshold for the gesture based on the degree of its approximation. afterwards,
The calculated change information is transmitted to the system output information generation unit 903 in step S1207. Then, the process returns to step 1201 again.

According to the dialog device of the present embodiment, information on the actual stop position is newly added. Therefore, when the actual stop position has already passed the scheduled output stop position calculated by the dialog device. Also, even in the case of, it is possible to change the contents of the unit output information as appropriate, and it is an easy-to-use dialogue interface for various types of people, such as children, adults, and elderly people who use the dialogue system, those who are not familiar with the system immediately or those who find it difficult to use. That is, it is possible to provide an interactive system in which output information can be changed according to the user so as to be easy to use according to the frequency of use and familiarity.

It should be noted that the computer or the interactive function can be realized by using software, and by installing the interactive device or the program realizing the interactive function in a computer or various devices, various interactive functions according to the present invention can be installed. A device or a service can be realized.
The invention related to each of the above devices is also realized as a method related invention, and the invention described above is a program for causing a computer to execute a corresponding procedure or means and a computer-readable recording medium recording the program. Also holds.

In the present invention, as the recording medium, the ROM itself of the dialog device may be a program medium, or a program reading device such as a CD-ROM drive is provided and the recording medium may be inserted therein. It may be a program medium such as a CD-ROM that can be read by. In any case, the stored program may be configured to be accessed and executed by the CPU, or the program may be read and the read program may be downloaded to a program storage area (not shown). The method in which the program is executed may be used. It is assumed that this download program is stored in advance in this dialog device.

Here, the program medium is a recording medium which can be separated from the main body of the dialogue device, and is a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a floppy (registered trademark) disk or a hard disk, or a C disk.
Disk system of optical disc such as D-ROM / MO / MD / DVD, card system such as IC card (including memory card) / optical card, mask ROM, EPROM, E
It may be a medium that fixedly carries the program, including a semiconductor memory such as an EPROM or a flash ROM.

Further, it may be a medium that carries the program in a fluid manner, such as downloading the program from a communication network. When the program is downloaded from the communication network in this way, the download program may be stored in the dialogue device body in advance, or may be installed from another recording medium. The content stored in the recording medium is not limited to the program and may be data.

[0085]

As described above, according to the dialogue apparatus of the present invention, when the system receives an input from the user during the output of the unit output information, the unit output information during the output is changed and the user outputs the unit output information. Responding to new input from allows a natural response. This makes it easy for various types of people, such as children, adults, and elderly people who use the dialogue system, people who are not immediately familiar with the system or people who find it difficult to use. Also, the excellent effect that the system responds quickly and does not show a redundant reaction even if the user inputs while the system is uttering or gesturing Can play.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a dialogue device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing the processing of the dialog device of the present invention.

FIG. 3 is a flowchart of processing in the system output information generation unit 102.

FIG. 4 is a flowchart of processing in the system information output unit 103.

FIG. 5 is a flowchart of processing in the system output information sequential management unit 105.

FIG. 6 is a flowchart of processing in the system output information stopping unit 106.

FIG. 7 is a flowchart of processing in the system main power information changing unit 107.

FIG. 8 is a diagram showing an example of an output change pattern table 108.

FIG. 9 is a block diagram showing a configuration of a dialogue device according to a second embodiment of the present invention.

FIG. 10 is a flowchart showing processing in the system output information generation unit 903.

FIG. 11 is a flowchart showing processing in a system output information stopping unit 901.

FIG. 12 is a flowchart showing processing in the system output information changing unit 902.

FIG. 13 is a diagram showing an example of interaction between the related art and the present invention.

FIG. 14 is a block diagram showing a conventional system.

FIG. 15 is a flowchart showing processing of a conventional system.

FIG. 16 is a diagram showing an example of an output pattern table.

[Explanation of symbols]

101 user information input section 102,903 System output information generation unit 103 System information output section 104 output pattern table 105 System output information sequential management unit 106,901 System output information stop unit 107,902 System output information change unit 108 Output change pattern table

Claims (9)

[Claims] 1. A unit for recognizing at least one of a user's gesture or a user's utterance as input, and output information for responding to a dialogue with the user based on the recognition result of the input, for each unit output information. And a means for outputting the generated output information for each of the unit output information, and an interactive device for interacting with a user, in the middle of outputting the unit output information from the interactive device to the user. An interactive apparatus comprising means for changing the unit output information based on a recognition result of the new input when a new input is made. 2. The dialogue apparatus according to claim 1, wherein the unit for changing the unit output information comprises a unit for stopping the output information on the way. 3. The dialogue apparatus according to claim 1, wherein the unit for changing the unit output information comprises a table holding an output change pattern for changing the output information for the user. 4. The unit for changing the unit output information comprises a unit for sequentially managing the output information for the user, and the unit for changing the output to the user includes a recognition result of the new input and the unit output information. 4. The dialogue apparatus according to claim 1, wherein the unit output information is changed based on the output status. 5. When changing the unit output information to the user based on the recognition result of the input, when the actual output has already passed the scheduled output stop position calculated by the means for stopping the output information on the way, The means for sequentially managing the output information newly generates information on the actual stop position, and outputs the unit output according to whether or not the difference between the actual stop position and the planned output stop position exceeds a threshold value. The interactive apparatus according to claim 4, wherein the content of the information is changed. 6. A step of recognizing at least one of a user's gesture or a user's utterance as an input, and generating output information for responding to a dialogue with the user for each unit output information based on a recognition result of the input. And a step of outputting the generated output information for each of the unit output information, and a dialogue method for performing a dialogue with a user, wherein a new input is made during the output of the unit output information in the output step. And a step of recognizing the new input, and a step of changing and outputting the unit output information based on a recognition result of the new input. 7. A step of sequentially managing output information for a user, wherein the step of changing and outputting the unit output information is based on a recognition result of the new input and an output status of the output information. The interaction method according to claim 6, wherein the unit output information is changed. 8. A program for causing a computer to function as the dialog device according to claim 1. 9. A computer-readable recording medium on which the program according to claim 8 is recorded.