JP2003228449A - Interactive apparatus and recording medium with recorded program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an interactive apparatus that enables a man to conduct natural dialog by reducing the waiting time during the interaction, between machine and man. <P>SOLUTION: An interactive apparatus 10 is equipped with an audio input unit for inputting a user's voice, an action acquiring unit for acquiring actions for each body part of the user, an audio processing unit for processing audio by recognizing audio signals, an action analyzer for analyzing the actions by recognizing the actions, a dialog coordinating unit for coordinating dialogs with the users, a dialog administrative unit for administrating a progress of the dialogs and a determined result by a dialog progress determining unit, an audio control unit, an action control unit 7, and an audio output unit, as well as an action output unit. The interactive coordinating unit 200 is equipped with an voice attribute analyzer for analyzing attributes of responding voices on the basis of inputted voices, an action attribute analyzer for analyzing attributes of responding actions on the basis of inputted actions, and the dialog progress determining unit to determine the interactive progress, on the basis of the attributes of the voices or the actions, and also send an advanced message for ending of the dialog on the basis of the analyzed results by the dialog progress determining unit. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a dialogue medium for realizing a natural dialogue between a user and a machine and a recording medium on which a dialogue processing program is recorded. The present invention relates to a user interaction device and a recording medium having an interaction processing program recorded therein.

[0002]

2. Description of the Related Art In recent years, along with high-performance workstations and personal computers (hereinafter referred to as personal computers), storage media such as magneto-optical disks having a large storage capacity have become inexpensive, and are suitable for high-resolution display devices and multimedia. The price of peripheral devices has also dropped significantly. In the field of document processing, image data processing, and other fields, it is required to improve the data processing function that can adapt to the increase in the amount of information of the data to be processed. Various processing devices are being developed that are capable of performing various kinds of processing on video and moving images.

Further, voice recognition technology has been developed and is being mounted in devices such as television receivers, radio receivers, vehicle-mounted navigations, mobile phones, and personal computers. Speech recognition technology is usually built in as part of each device.
Furthermore, a technique for recognizing a moving image input from a camera is being developed. If multiple input devices, such as a microphone, camera, keyboard, mouse, pointing device such as light pen, tablet, etc., can be connected to the computer and the user can input using the input device that is most convenient for them in that aspect The interface is very easy to use. In this way, the user selects any input mode from multiple different input modes,
An interface that can be combined to convey your intention to the system is called a multimodal interface.

Conventionally, as an interface between a human and a computer, a dialogue device which responds to a voice input has been considered. It recognizes human voice
In response to this, the internal state of the system is changed, a predetermined output is performed, and a dialogue with a human is realized. In order to make the dialogue with a computer smoother, an output synthesis system in which an animation or the like responds to an input voice has been proposed.

[0005]

However, as an interface between a human being and a computer, the computer is mainly in charge of asking questions, and the user follows the task passively to proceed with the work.
Therefore, the order of the user's utterances is fixed, and no consideration is given to the easiness of the user's utterance and the naturalness of the dialogue.

When interacting with a computer, the naturalness of the interaction is lost unless the computer and the user speak in a timely manner. Japanese Unexamined Patent Publication No. 9-218770 describes a device for extracting a relationship expression in a spoken language or a non-verbal message from a user's input, and interactively controlling the expression, and a device for generating an expression requesting a relationship from the user. Has been done. Further, JP-A-62-40577 discloses that
There is described a device capable of detecting an utterance such as "what is it?" And being able to listen back during the utterance of the computer. However, the devices described in the above publications do not describe the technique of predicting the end of the dialogue by utilizing the association. Further, only the mutual response that occurs in one type of dialogue between one dialogue device and one person is described.

On the other hand, in the future, as the mechanization at the consultation counter progresses, the number of cases where a plurality of people use a plurality of interactive processing devices will increase. At this time, a queue processing method becomes a problem because a plurality of human beings use the plurality of interactive processing devices efficiently. It is wasted time for one person to go to an empty machine after seeing that one human being has left the machine after one dialogue.

The present invention has been made in view of the above problems, and in a dialogue between a machine and a human, it is possible to shorten the waiting time and realize a natural dialogue, and It is an object of the present invention to provide a recording medium recording an interactive processing program.

[0009]

According to another aspect of the present invention, there is provided a dialog device which responds to a voice input, wherein voice input means for inputting a voice from a user, and motion input means for inputting a user's action. A dialogue progress judging means for judging the progress of the dialogue with the user based on the voice input and / or the motion input, and a dialogue end notice notifying means for notifying the end of the dialogue when the dialogue progress judging means predicts the end of the dialogue. It is characterized by having and.

A dialogue apparatus of the present invention is a dialogue apparatus corresponding to a voice input, a voice input means for inputting a voice from a user, a motion input means for inputting a user's motion, the voice input and the motion. A dialogue management unit that analyzes the input and manages the dialogue with the user, and when the dialogue management unit determines the end of the dialogue, controls its own voice output and action output so as to guide the end of the dialogue. It is characterized in that it comprises a dialogue end guide means.

Further, more preferably, the dialogue progress judging means further comprises dialogue end guiding means for controlling voice output and motion output so as to guide the end of the dialogue when the end of the dialogue is predicted. May be. Further, the present invention comprises a plurality of dialogue devices according to claim 1 or 2, and further comprises dialogue end notice notifying means for notifying the end prediction timings of the plurality of dialogue devices together with the IDs of the respective dialogue devices to the outside. Is characterized by.

Further, more preferably, the dialogue progress judging means may judge the information comprehension / information sharing and the progress status of the dialogue end. As a preferred specific aspect, the voice determined by the voice feature determination means may be a joint voice, and the operation determined by the motion feature determination means may be a joint movement.

Further, more preferably, when the dialogue management means decides to end the dialogue, the dialogue end guiding means outputs the at least one of the chorus voice and the chorus operation. Alternatively, it may control the operation control means. In addition, the motion output may be a motion that displays the motion of itself or a motion of a robot having a mechanical structure.

Further, according to the present invention, in a dialogue device corresponding to a voice input for a computer, a voice input means for inputting a voice from a user, an operation input means for inputting a user's action, the voice input and Alternatively, a dialogue including a dialogue progress judging means for judging the progress of the dialogue with the user based on the operation input, and a dialogue end notice notifying means for notifying the end of the dialogue when the dialogue progress judging means predicts the end of the dialogue. It is a computer-readable recording medium in which a program for functioning as a device is recorded.

Further, according to the present invention, in a dialogue device corresponding to a voice input for a computer, a voice input means for inputting a voice from a user, an operation input means for inputting a user's action, the voice input and the voice input and A dialogue management unit that analyzes the action input and manages the dialogue with the user, and when the dialogue management unit determines the end of the dialogue, controls voice output and action output so as to guide the end of the dialogue. A computer-readable recording medium having recorded therein a program for functioning as a dialogue device including dialogue end guide means.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION Preferred embodiments of a dialogue apparatus of the present invention will be described in detail below with reference to the accompanying drawings. First, the basic idea of the present invention will be described.

The inventor has recorded and analyzed actual human interaction. For recording, an optical motion capture system was introduced, and along with voice and image data, a marker attached to the subject's body was optically captured by an infrared camera, and three-dimensional numerical data of that position was collected. As a result, it has become possible to collect multi-modal dialogue data that allows analysis of the image / sound data of the two interlocutors and the information on the marker position on the same time axis.
As a result of analyzing this dialogue data, the following was found.

(1) One speaker raises a topic, and the other speaker understands information on the topic (information understanding), or shares the same opinion on the information, and expresses emotions (information). Sharing), voices or movements tend to occur while the other party speaks. (2) When the conversation is about to end (dialogue end), no more talking about the raised topic, and when the other party's utterance ends or after the other person's utterance (other party's utterance pause period), a joint action is made to fill the gap or Voices come to occur, and at that time, the types of mutual voices tend to be "n" and "so". (3) The types of voices for sharing when understanding or sharing information are "Ah""Ah""Nnn""Hee""Eh""Eh"
Often "yes", "lie" and "really",
At the end of the dialogue, these related voices do not appear. (4) Relative voice at the end of the dialogue is rms (root mean square:
Root mean square) Amplitude values tend to be small.

The present invention is an interactive apparatus having the following functions based on the above findings.

(1) Recognizing the user's mutual voice or action occurring during or after the output of system voice or action or several frames (for example, 30 frames = 1 sec), and it is determined that the voice or action at that time is the end of the dialogue. When this is done, the next user is notified of the end of the dialogue. (2) When the system wants to end the conversation, several frames after the user's utterance or after the end (for example, 30 frames = 1)
(sec), a mutual voice or a mutual action for inducing the end of the dialogue is output to guide the end of the dialogue.

Thus, in the dialog between the machine and the user, it is possible to predict the end of the dialog and guide the end of the dialog, and in the dialog between the plurality of users and the plurality of systems, the user waiting for the turn can be performed. Waiting time is reduced by predicting that the turn is near, alleviating the frustration of waiting, and controlling the utterance or operation of the system (for example, CG (Computer Graphics) or robot) to induce the dialogue to end. It becomes possible to shorten the time, and a smooth dialogue can be realized between the user and the system.

FIG. 1 is a block diagram showing the basic configuration of a dialogue apparatus according to the first embodiment of the present invention based on the above basic idea. This is an example applied to an information processing device using a multimodal interface as a dialogue device. In FIG. 1, the dialog device 10 includes a voice input unit 101 (voice input unit) for inputting a voice signal from a user, a motion input unit 102 (motion input unit) for inputting a movement of each part of the user's body, and an input. A voice processing unit 103 for recognizing a voice signal, and an operation processing unit 1 for recognizing an input motion.
04, the dialogue adjustment unit 200 (including the dialogue progress determination unit 203) for adjusting the dialogue with the user, managing the progress of the dialogue based on the input from the user, and the dialogue progress determination unit 2
Dialog management unit 105 (dialog management means) that manages the determination result of 03, voice control unit 106 that controls the output voice, operation control unit 107 that controls the output operation, voice output unit 108 that outputs the voice, and operation Output unit 1 for outputting
09.

The dialogue adjusting unit 200 is a means for judging the progress of the dialogue based on the user's mutual relations, the voice characteristic judging unit 201 for judging the characteristic of the joint voice from the inputted voice, and the characteristic of the joint action from the inputted action. A mutual action characteristic determination unit 202 (dialogue progress determination unit), a dialogue progress determination unit 203 (a part of the dialogue progress determination unit) that determines a progress state of the dialogue based on these voice or motion features, a dialogue progress determination unit 203. It is composed of an end notifying unit 204 for notifying the next user of the end of the dialogue based on the result of the judgment, and an end notifying unit 205 for notifying the next user of the end notice, and outputs the judgment result to the dialog management unit 105. The end notice section 204
Also, the end advance notice notifying unit 205, as a whole, constitutes a dialogue end advance notice notifying unit that notifies the end of the conversation when the dialogue progress determination unit 203 predicts the end of the conversation.

FIG. 2 is a diagram showing a concrete system configuration of the dialogue apparatus 10. In FIG. 2, the dialogue device 10 is
From a CPU 1, a work memory 2 including a RAM, an input unit 3 for inputting a sound signal and a moving image signal, a database 4 for storing the input signal and a calculation processing result of the CPU 1, a liquid crystal display device (LCD) having a dot matrix configuration, and the like. Display unit or output unit 5 consisting of a robot that outputs motion shapes, SRAM (Static RAM) card that holds information written by power backup, FD, CD-
It is composed of an external storage device 7 such as a ROM and an external storage driver 6 which is a reading device of the external storage device 7.

The CPU 1 is a control unit for controlling the entire apparatus including execution of voice / moving image processing, and uses a work memory 2 storing data used for calculation in accordance with a built-in system program, and a dialogue management program. Is executed to create a processing result on the work memory 2.

The work memory 2 is a so-called working memory for temporarily storing data relating to character display, data used for calculation, calculation results, etc., and is a program used in the program processing executed by the CPU 1, voice / moving images. It stores image processing data and so on. Note that this program may be stored in a ROM (not shown) as a system program. In addition, a part of the storage area of the RAM used as a work memory is backed up by a power source, or an EEPROM (electric
It is composed of non-volatile memory such as an all erasable programmable ROM) and flash memory, and retains the setting conditions even after the power is turned off. Various setting data, processing data, and the like are stored in this storage area.

The output unit 5 is a liquid crystal display device (LCD) or a robot having a dot matrix structure, and presents voice information and operation information stored in the database 4 and calculation results of the CPU 1 by display or operation. Further, this robot may have a hardware structure having a mechanical structure, or may be synthesized and displayed on the display screen of the output unit 5 by, for example, CG.

The external storage driver 6 is a reading device of the external storage device 8 which stores the dialogue management program. External storage device 8 such as a memory card, FD, CD-ROM, etc.
Is a storage medium in which a program for realizing the speaking right management function, a voice / moving image processing program described later, and the like are recorded.

Hereinafter, the operation of the dialog device configured as described above will be described. First, the basic operation of the dialogue device 10 will be described. In FIG. 1, a voice signal from a user is input by a voice input unit 101 such as a microphone, and a movement of each body part including a user's head is input by a motion input unit 102 such as a video camera.

The input voice signal is processed by the voice processing unit 103.
The motion of each part of the user's body, which is recognized and processed by the user, is output to the dialogue management unit 105.
4 and recognition processing is performed and the result is output to the dialogue management unit 105.

The dialogue management unit 105 manages the progress of the dialogue based on the voice / motion information recognized and processed by the voice processing unit 103 and the motion processing unit 104, and determines whether or not there is a self (system) utterance (see FIG. 9 described later). (Corresponding to W1 or W2) is output to the dialogue progress determination unit 203, and the voice and action to be output by itself (system) are passed to the voice control unit 106 and the action control unit 107 at appropriate timing.

Voice control unit 106 and operation control unit 107
Controls voice and motion, and is presented by the voice output unit 108 and the motion output unit 109. The voice signal recognized by the voice processing unit 103 and the motion recognized by the motion processing unit 104 are also input to the dialogue adjusting unit 200.
The voice feature determination unit 201 analyzes the coherent voice from the input voice, and the coherence operation feature determination unit 202 measures the magnitude of the coherent action from the input action. Then, the dialogue progress judging unit 203 judges, based on the judgment results of both parties, at which stage of “information understanding / information sharing” and “dialogue end” the dialogue is.

The dialogue progress judgment unit 203 outputs the judgment result to the dialogue management unit 105, and also outputs the judgment result to the end notice unit 204 when it is judged that the stage is the “dialogue end”. The end notice unit 204 gives a notice to the next user in the waiting state of the end of the dialogue by the end notice unit 205.

Next, the operation of the dialogue adjusting section 200 will be specifically described. In the voice feature determination unit 201, the voice data of the user input from the voice input unit 101 such as a microphone is A / D-converted, and the recognition result (type of mutual relation) of the voice data recognized as mutual relation is output. The rms amplitude (power) of the data is framed (eg,
Measure every 1/30 sec). Mutual voice is generally
Appear as voices such as "Yes", "Yes", and "N". rm
The s amplitude is the average of changes in the sound pressure of a voice wave, and the loudness of the sound depends on the rms amplitude.

In the grouping motion feature determining section 202, the magnitude of the user's movement, which is input from the motion inputting section 102 such as a video camera and A / D converted in the motion processing section 104 and is recognized as the grouping motion, is framed (for example, Measure every 1/30 sec). Here, the motion input unit 102 may be a system such as a motion capture system that extracts position information of one or more parts of the human body, in which case,
The three-dimensional position information of each body part is input to the motion processing unit 104. The joint movement is generally regarded as the movement of the head in the vertical direction (Y-axis direction on the three-dimensional coordinates).

The operation of determining the progress of the dialogue in the dialogue progress determination unit 203 will be described below with reference to FIGS. As a result of analyzing the data recording the actual dialogue between humans, the following tendency was found. When we grasp the flow of dialogue in a big way, one speaker raises a topic, the other speaker understands the content of the information being talked about, and further progresses while sharing the topic with both parties. When I don't talk
They started to respond with a nod, and the topic gradually turned to the end, turning to the next topic. That is,
It is thought that the dialogue proceeds roughly from information comprehension → information sharing → end of topic.

FIGS. 3 to 6 are diagrams showing an example of the analysis result of the dialogue data. FIG. 3 shows an example of the association at the “information understanding” stage, and FIGS. 4 and 5 show “information sharing”. FIG. 6 shows an example of association at the stage, and FIG. 6 shows an example of association at the stage of “topic end”.

In each figure, (a) is voice data (rm
(s, pitch, transcription), (b) is the head (Head)
It shows the speed of movement of. As motion information,
In addition to speed, the amount of movement of each part or acceleration may be used.

As a result of analyzing the dialogue between humans in this way, the following tendency was found. (1) One speaker raises a topic and the other speaker understands the information of the topic (information understanding), or shares the same opinion about the information, and expresses emotions (information sharing).
If so, the voices or movements tend to occur during the other party's speech. For example, in FIG. 3A, while one speaker speaks “XX no-”, the conversation partner understands the information and expresses a voice together with “Ah”.
As shown in (b), the movement of the head (joint movement) occurs. In addition, one of the speakers said, "...
During the utterance, the conversation partner understands the information and expresses a joint voice "Ah", and at the same time, as shown in FIG. 3B, the movement of the head (joint movement) continues several times. In FIG. 4, while one speaker is speaking, the other party of the dialogue shares information and expresses mutual voice (FIG. 4A) and mutual movement (FIG. 4B). Similarly, in FIG. 5, the joint voice and the joint movement of information sharing can be seen. (2) When the conversation is about to end (dialogue end) because the subject is no longer being talked about, and the conversation ends, or after the other party's utterance (participant utterance pause period), the "pause" is filled. A motion or voice starts to occur, and at that time, the type of the coherent voice tends to be “n” or “so”. For example, in FIG. 6 (a), when one speaker utters "Nu", the other party of the dialogue "pauses" after the utterance.
So as to fill in, "Sona no" is added.
Also, when you say "Mezurashine", you will be depressed to fill the gap of "Sokane" in the other party's speech pause. (3) The types of mutual voice when understanding or sharing information are:
As shown in FIG. 3 to FIG. 5, “Ah” “Ah” “Nnn”
Often "he", "eh", "yes", "yes", "lie", and "really", but when the dialogue is about to end (end of dialogue), as shown in FIG. Does not appear. (4) The rms amplitude value of the coherent voice at the end of the dialogue tends to be smaller than that of the coherent voice at the time of information understanding and information sharing. For example, the connected voice at the end of the dialogue shown in FIG.
The rms amplitude value is smaller than that of the companion voice for information understanding shown in FIG. 3 and the information sharing shown in FIG.

Based on the above knowledge, the dialogue end is determined by the following procedure. 7 and 8 are flowcharts of the operation of determining the progress of the dialogue in the dialogue progress determination unit 203, and this flow is executed by the CPU 1 of FIG. In the figure, S indicates each step of the flow. FIG. 9 is a diagram for explaining the timing of system utterance and the user's mutual insertion. In the figure, W1 indicates a system utterance, and W2 indicates a system utterance end and a system utterance end.

When the dialogue progress judgment processing starts, the dialogue progress judgment unit 203 determines whether the self (system) utterance is being uttered from the dialogue management unit 105 (W1 section in FIG. 9) or at the end / after utterance (FIG. 9). W2 section) is extracted (step S101). Then, step S102.
Then, the dialogue progress judging unit 203 extracts the mutual voice section and the mutual movement section based on both the voice feature judging unit 201 and the joint motion characteristic judging unit 202, and judges the dialogue end by the following procedure.

(1) When the user's mutual voice or motion appears during the utterance of the self (system) (W1 section in FIG. 9), it is judged as "information understanding / information sharing" in step S103 and this flow is ended. To do. For example, in FIG. 9, when the user's mutual voice “V1” appears during the utterance of itself (system) (W1 section in FIG. 9), this step S10
At 3, it is judged as "information understanding / information sharing". (2) End of utterance of self (system) or after utterance (W in FIG. 9)
When the user's mutual voice or motion appears in (2 sections), it is determined in step S104 whether the mutual voice includes a voice. For example, in FIG. 9, when the user's joint voice “V2” appears at the end of the utterance of the self (system) or after the utterance (W2 section of FIG. 9), first, this step S104.
Is judged as "information understanding / information sharing", and then is judged in more detail according to the kind or prosody of the joint, or the speed of movement.

That is, when the movements are the only movements in step S104, the speed of movement is measured in step S105. When the average speed is equal to or higher than the predetermined threshold value D1, it is determined in step S107 that the dialogue is ended. Then, the present flow is ended, and when the average value of the speeds is smaller than the predetermined threshold value D1,
In step S106, "information understanding / information sharing" is determined, and this flow ends.

If there is a coherent voice in the above step S104, the kind of the voice is judged in step S108, and the coherent voice is "Ah""Ah""Nnn""Hee""Eh".
If it has a type or prosody that means “information understanding” or “information sharing”, such as “yes”, “yes”, “lie”, it is determined as “information understanding / information sharing” in step S109, and this flow ends. To do.

In step S108, if the voices have a kind or prosody that means "dialogue end" such as "n" and "so", the rms amplitude of the voice is measured in step S110, and the rms amplitude of the voice is measured. Is the predetermined threshold value D2
Is smaller than. When the average value of the rms amplitude of the voice is smaller than the predetermined threshold D2, step S
In 112, it is determined as “dialogue end” and this flow is ended, and when the average value of the rms amplitude of the voice is equal to or greater than the predetermined threshold D2, in step S111 it is determined as “information understanding / information sharing” and this flow is executed. finish.

When the determination result according to the above process flow is "information understanding / information sharing", the dialogue progress determination unit 203 outputs the determination result to the dialogue management unit 105 and ends the processing. If the above judgment result is “dialogue end”,
The dialogue progress judgment unit 203 outputs the judgment result to the dialogue management unit 105 and inputs it to the end notice unit 204. The end notice unit 204 gives a notice to the next user of the end of the dialogue, and inputs it to the end notice unit 205. In the first embodiment, it is possible to predict the end of the dialogue and to notify the next user of the end of the dialogue by catching the change in the mutual patterns in the dialogue process.

As described above, the dialogue device 1 of the present embodiment
0 is a voice input unit 10 for inputting a voice signal from the user.
1. Motion input unit 1 for inputting movements of various parts of the user's body
02, a voice processing unit 1 for recognizing an input voice signal
03, motion processing unit 10 for recognizing input motion
4, a dialogue adjusting unit 200 for adjusting the dialogue with the user, a dialogue managing unit 105 for managing the progress of the dialogue based on an input from the user, and a decision result of the dialogue progress judging unit 203, a voice control unit 106, The interaction adjustment unit 200 includes a motion control unit 107, a voice output unit 108, and a motion output unit 109, and the dialogue adjustment unit 200 determines a feature of the synched voice from the input voice, a voice feature determination unit 201, and a feature of the synched motion from the input action. The joint motion characteristic determination unit 202 for determining
A dialogue progress determination unit 203 for determining the progress state of the dialogue based on the characteristics of these voices or motions is provided, and the dialogue progress determination unit 20 is provided.
Since the next user is notified of the end of the dialogue based on the determination result of 3, it is possible to predict the end of the dialogue in the dialogue between the machine and the user, and to notify the next user who is waiting for the turn that the turn is near. Therefore, the annoyance of waiting for a turn can be alleviated, and a smooth dialogue between the user and the system can be realized.

Second Embodiment FIG. 10 is a block diagram showing the basic configuration of a dialogue apparatus according to the second embodiment of the present invention. The same components as those in FIG. 1 are designated by the same reference numerals and the description of the duplicated portions will be omitted. Figure 1
0, in the dialogue device 20 of the present embodiment, the dialogue adjusting unit 300 is configured to include the end guiding unit 206 (dialogue ending guiding means). Further, the processes of the dialogue management unit 105A (dialogue management means) and the dialogue progress determination unit 203A (dialogue progress determination means) are different from those of the first embodiment.

Hereinafter, the dialogue device 2 configured as described above.
The operation of 0 will be described. Dialogue device 10 of the first embodiment
Differences from the operation of will be described.

In the present embodiment, the dialogue management unit 105A.
Manages the progress of the dialogue based on the input from the user and manages the determination result of the dialogue progress determination unit 203A. Then, based on the judgment result of the dialogue progress judgment unit 203A, for example, when the dialogue time is not very long and the dialogue time is too long, the system side gives the dialogue end.
Is determined and the end guide unit 206 is notified of the end of the dialogue. The end guide unit 206 adds a feature for presenting a group of voices and actions for generating a group of voices for guiding the user to terminate the dialogue from the system side, and the voice control unit 10
6 and the operation control unit 107. Specifically, several frames after the user's utterance or after the end (30 frames = 1
During the second), the spit voice or the spit operation characteristic of "dialogue end" is output. Voice control unit 106, operation control unit 10
In 7, the characteristics are controlled so as to be reflected in the original voice and the action, and presented by the voice output unit 108 and the action output unit 109.

In the second embodiment, the waiting time can be shortened by controlling the utterance or operation of the system side (for example, CG or robot) to induce the dialogue to end. In addition, by predicting the end of the dialogue and inducing the system to end the dialogue, the end of the dialogue can be promoted and the waiting time can be further shortened.

Third Embodiment FIG. 11 is a block diagram showing the basic configuration of an interactive device according to the third embodiment of the present invention. The same components as those in FIGS. 1 and 10 are designated by the same reference numerals, and description of duplicated portions will be omitted.

In FIG. 11, in the dialogue device 30 of the present embodiment, the dialogue adjusting unit 400 includes the end guiding unit 206, and the dialogue progress judging unit 203B (conversation progress judging means) judges that the dialogue has ended. At this time, the dialogue progress determination unit 203
B outputs the determination result to the end notifying unit 204 and the end guiding unit 206.

As a result, when the dialogue progress judging unit 203B judges that the dialogue has ended, the system notifies the next user of the end of the dialogue and creates a system-side relationship that guides the current user to end the dialogue. It becomes possible to output.

In the third embodiment, it is possible to notify the next user of the end of the dialogue and guide the user to end the dialogue with the current user. Needless to say, it may have both the configuration of the second embodiment and the configuration of the third embodiment.

Fourth Embodiment FIG. 12 is a block diagram showing the configuration of a dialogue system according to a fourth embodiment of the present invention, which is shown in FIG. 1, FIG. 10 or FIG.
Is a dialogue system having a plurality of dialogue devices 10, 20, 30. In FIG. 12, 203C is each of the dialogue devices <1> to
<3> Dialogue progress judging unit, which is shown in FIG. 1, FIG. 10 or FIG.
It has a function similar to that of the dialogue progress determination units 203 and 203A. Further, the end notice unit 204C and the end notice unit 205C of each of the dialogue devices <1> to <3> are configured in common.

When each of the dialogue devices <1> to <3> is judged by the dialogue progress judging section 203C to be in the "dialogue end" stage, the judgment result together with the judged time is given to the end notifying section 20.
Output to 4C. The end notifying unit 204C determines that each dialogue device <
Based on the determination times sent from 1> to <3>, the end notice notifying unit 205C gives a notice to the next user in the waiting state of the ID number of the device predicted to be the “dialogue end” earliest. .

With this, in a dialogue between a plurality of users and a plurality of systems, it is possible to teach a user waiting for a turn to which dialogue apparatus to proceed to. here,
Although the example of the determination time has been described, the predicted dialogue end time of all systems or the time until the dialogue end may be notified.

The interactive device of the present invention is not limited to the above-described embodiment, and it goes without saying that various changes can be made without departing from the gist of the present invention. For example, although it can be applied to an information processing device using a multimodal interface as described above,
The present invention is not limited to this, and can be applied to all devices.

Further, the dialogue device according to each of the above-described embodiments may be incorporated as a voice / moving image processing function of a portable information terminal such as a PDA (Personal Digital Assistants) or a personal computer. Further, in each of the above-described embodiments, the name of the dialogue device is used, but this is for convenience of description, and may be, for example, a voice / moving image processing device or a multimodal interface device. Furthermore, the types of the circuit units and the like that make up the dialog device and the database are not limited to those in the above-described embodiment.

The dialog device described above is also realized by a program for operating this processing device. This program is stored in a computer-readable recording medium. In the present invention, as the recording medium, the main memory itself may be a program medium, or a program reading device is provided as an external storage device, and the program medium can be read by inserting the recording medium therein. May be. In any case, the stored program may be configured to be accessed and executed by the CPU, or in any case, the program is read and the read program is stored in a program storage area (not shown). The program may be downloaded to the computer and the program may be executed. It is assumed that this download program is stored in the main body device in advance.

Here, the program medium is a recording medium which can be separated from the main body. For example, in addition to a PC card (SRAM card), a tape system such as a magnetic tape or a cassette tape, a floppy disk (registered trademark).
Disk such as hard disk and hard disk, CD-ROM / M
Disk system of optical disk such as O / MD / DVD, card system such as IC card / optical card, or mask ROM,
It may be a medium that fixedly carries the program, including a semiconductor memory such as an EPROM, an EEPROM, a flash ROM, and the like.

Further, when a communication device capable of connecting to an external communication network is provided, it is a medium that carries the program fluidly so that the program is downloaded from the communication network via the communication device. It may be. When the program is downloaded from the communication network as described above, the download program may be stored in the main body device in advance, or may be installed from another recording medium. The content stored in the recording medium is not limited to the program and may be data.

[0064]

As described above in detail, according to the present invention, the end of the dialogue is determined from the user's co-operation and the next user is notified of the end of the dialogue. Therefore, in the dialogue between the machine and the user. , It is possible to predict the end of the dialogue and give a notice to the next user who is waiting for his turn that his turn is near, and it is possible to alleviate the frustration of waiting for his turn.

When it is desired to end the dialogue from the system side, a structure for guiding the end of the dialogue is generated to guide the end of the dialogue, so that the utterance or operation of the system side (for example, CG or robot) is controlled. Then, the waiting time can be shortened by inducing the dialogue to end.

Further, since the end of the dialogue is judged from the user's co-operation, the next user is notified of the end of the dialogue, and the co-operation for guiding the end of the dialogue is generated from the system side to guide the end of the dialogue. By predicting the end of the dialogue and inducing the system to end the dialogue, the end of the dialogue can be promoted and the waiting time can be further shortened. Therefore, the waiting time can be shortened and a natural dialogue can be realized between the user and the system.

[Brief description of drawings]

FIG. 1 is a block diagram showing a basic configuration of a dialogue device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a specific system configuration of the dialog device according to the present embodiment.

FIG. 3 is a diagram showing an example of how the result of analysis of dialogue data of the dialogue apparatus according to the present embodiment is related at the “information understanding” stage.

FIG. 4 is a diagram showing an example of how the result of analysis of dialogue data of the dialogue apparatus of the present embodiment is related at the “information sharing” stage.

FIG. 5 is a diagram showing an example of how the results of analysis of dialogue data of the dialogue device of the present embodiment are related at the “information sharing” stage.

FIG. 6 is a diagram showing an example of how the dialogue data analysis results of the dialogue device of the present embodiment are joined at the “topic end” stage.

FIG. 7 is a flowchart of an operation of determining the progress of a dialogue in the dialogue progress determination unit of the dialogue device of the present embodiment.

FIG. 8 is a flowchart of an operation of determining the progress of a dialogue in the dialogue progress determination unit of the dialogue device of the present embodiment.

FIG. 9 is a diagram for explaining the system utterance of the dialogue apparatus of the present embodiment and the timing of the user's mutual insertion.

FIG. 10 is a block diagram showing a basic configuration of an interactive device according to a second embodiment of the present invention.

FIG. 11 is a block diagram showing a basic configuration of an interactive device according to a third embodiment of the present invention.

FIG. 12 is a block diagram showing a configuration of a dialogue system according to a fourth embodiment of this invention.

[Explanation of symbols]

1 CPU 2 work memory 3 input unit 4 database 5 output unit 6 external storage driver 7 external storage device (recording medium) 10, 20, 30 dialogue device 101 voice input unit (voice input means) 102 operation input unit (motion detection means) 103 voice processing unit 104 motion processing unit 105 dialogue management unit (dialogue management means) 106 voice control unit 107 motion control unit 108 voice output unit (voice output means) 109 action output unit (action output means) 200, 300, 400 dialogue adjustment Unit 201 Voice feature determination unit 202 Coupling operation feature determination unit (dialogue progress determination unit) 203, 203A, 203B Dialogue progress determination unit (a part of the dialogue progress determination unit) 204, 204C Termination notice unit 205, 205C Termination notice unit 206 End guidance unit (dialogue end guidance means)

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) G10L 3/00 571Q

Claims (11)

[Claims] 1. A dialogue device corresponding to a voice input, wherein voice input means for inputting voice from a user, motion input means for inputting a user's motion, and based on the voice input and / or the motion input. A dialogue apparatus, comprising: dialogue progress determination means for determining the progress of a dialogue with a user; and dialogue termination notice means for notifying that the dialogue progress determination means predicts the end of the dialogue. 2. An interactive device corresponding to a voice input, a voice input means for inputting a voice from a user, a motion input means for inputting a user's motion, and analyzing the voice input and the motion input. , Dialogue management means for managing the dialogue with the user, and the dialogue management means, when the end of the dialogue is determined,
A dialogue ending device for controlling a voice output and a motion output so as to guide the end of the dialogue. 3. The dialogue progress judgment means further comprises dialogue end guide means for controlling voice output and action output so as to guide the end of the dialogue when the end of the dialogue is predicted. 1. The dialogue device according to 1. 4. A plurality of dialogue devices according to claim 1 or 2, further comprising: dialogue end advance notice notifying means for externally notifying end prediction timings of the plurality of dialogue devices together with IDs of the respective dialogue devices. Characterized dialogue device. 5. The dialogue apparatus according to claim 1, wherein the dialogue progress determination means determines information comprehension / information sharing and a progress state of dialogue end. 6. The dialogue apparatus according to claim 1, wherein the voice determined by the voice feature determination means is a continuous voice. 7. The dialogue apparatus according to claim 1, wherein the movement judged by the movement characteristic judging means is a joint movement. 8. The dialogue ending guide means, if the dialogue managing means decides to end the dialogue, the voice control means or the operation control so as to output at least one of a joint voice and a joint movement. 4. An interactive device according to claim 3, characterized in that it controls the means. 9. The dialogue device according to claim 2, wherein the action output displays the action of the user by means of an image or is expressed by the action of a robot having a mechanical structure. 10. An interactive device adapted to a computer for voice input, wherein voice input means for inputting a voice from a user, action input means for inputting a user's action, said voice input and / or said action input. Based on the above, the dialog device is provided with a dialog progress judging means for judging the progress of the dialogue with the user, and a dialogue end notice notifying means for notifying the end of the dialogue when the dialogue progress judging means predicts the end of the dialogue. A computer-readable recording medium on which a program for recording is recorded. 11. An interactive device, wherein a computer supports voice input, a voice input unit for inputting a voice from a user, a motion input unit for inputting a user's motion, the voice input and the motion input. Analyzing and managing the dialogue with the user, and the dialogue management means, when the dialogue management means decides to end the dialogue, the dialogue end for controlling its own voice output and action output so as to guide the end of the dialogue. A computer-readable recording medium having recorded therein a program for functioning as an interactive device including a guiding means.