JP2016076007A - Interactive apparatus and interactive method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an interactive apparatus and an interactive method allowing switching to the state of interaction with a user at proper timing.SOLUTION: An interactive apparatus 100 comprises detection means (detection unit 131) for detecting distance between a user and the interactive apparatus 100 and the presence of a user in a camera image, user intention determination means (user intention determination unit 132) for, on the basis of the detection result of the detection means, determining whether or not a user in a hands-free state has intention of speaking to the interactive apparatus 100, and interactive state control means (state control unit 137) for, on the basis of the determination result of the determination means, controlling the state of the interactive apparatus 100 so as to be switched to one of the state of interaction and the state of non-interaction.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a dialog device for performing a dialog with a user and a dialog method for performing a dialog between the user and the dialog device.

  In recent years, dialogue technology has been evolving to recognize voice and to understand and answer the user's natural utterance content (see, for example, Patent Document 1). By providing various sensors in a smartphone, a robot device, and the like, a dialog agent type dialog device capable of perceiving the presence of a user and performing a dialog as if interacting with a human being is being realized.

  The dialogue may be started in a state where the user does not perform a button operation or the like for transmitting the timing of utterance to the dialogue device (hereinafter referred to as “hands-free state”). In this case, the interactive device can switch from the non-interactive state to the interactive state, for example, by monitoring the microphone input and detecting the user's utterance.

JP 2002-182896 A

  However, just because a user's utterance is detected does not necessarily mean that the user has an intention to talk to the dialogue device (speaking intention). Therefore, even though the user does not have the intention to talk, there is a possibility that the dialogue apparatus that detects the user's utterance starts the dialogue at an incorrect timing and switches to the dialogue state.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an interactive apparatus and an interactive method capable of switching to an interactive state with a user at an appropriate timing.

  An interactive apparatus according to an aspect of the present invention is an interactive apparatus for performing a dialog with a user, the detecting means for detecting the distance between the user and the interactive apparatus and the presence of the user in the camera image, and the detecting means Based on the detection result, the user intention determination means for determining whether or not the user in the hands-free state has an intention to talk to the dialog apparatus, and the dialog apparatus is in the dialog state based on the determination result of the determination means And dialogue state control means for controlling the state of the dialogue device so as to switch to any one of the non-dialogue states.

  An interactive method according to an aspect of the present invention is an interactive method for performing a dialog between a user and an interactive device, wherein the interactive device has a distance between the user and the interactive device, and the presence of the user in a camera image. Detecting, based on a detection result of the detecting step, determining whether or not a user in a hands-free state has an intention to talk to the interactive device; and And controlling the state of the interactive device so that the interactive device switches to either the interactive state or the non-interactive state based on the determination result of the determining step.

  In the above interactive device or interactive method, the interactive device is controlled to switch to either the interactive state or the non-interactive state based on whether or not the user has an intention to talk to the interactive device. Thereby, the dialogue apparatus can be switched to the dialogue state at an appropriate timing according to the user's intention.

  Further, the interactive device is based on the determination result of the user state determining unit and the user state determining unit that determines whether or not the user can visually recognize the information from the interactive device based on the detection result of the detecting unit. And output control means for controlling output to the user. Thereby, when the user cannot visually recognize (view, etc.) the visual output (information) from the interactive device, the information can be transmitted to the user only by voice, for example. Further, when the user can visually recognize the visual output from the interactive device, for example, the audio output can be shortened by using both the visual output and the audio output.

  The interactive device executes a first recognition mode for continuously recognizing vocabulary contained in the user's voice in the conversation state, and recognizes only a predetermined vocabulary contained in the user's speech in the non-interactive state. Voice recognition means for executing the second recognition mode may be further included, and the user intention determination means is a predetermined voice included in the user's voice by the voice recognition means for executing the second recognition mode in the non-interactive state. When the vocabulary is recognized, it may be determined that the user has an intention to talk to the dialogue apparatus. Thereby, the dialogue apparatus can switch from the non-dialogue state to the dialogue state at an appropriate timing according to the user's intention, when the user speaks a predetermined vocabulary (keyword).

  In the first recognition mode, the voice recognition means continuously recognizes the vocabulary included in the user's voice by communicating with the outside of the dialogue apparatus and using the data processing of the server, and the second recognition mode. In the recognition mode, the voice recognition unit may recognize only a predetermined vocabulary included in the user's voice without performing communication with the outside of the dialogue apparatus. Thereby, in the first recognition mode, it is possible to perform voice recognition (server type voice recognition) capable of recognizing a large vocabulary using server data processing. Also, in the second recognition mode, for example, voice recognition can be performed while reducing power consumption compared to the first recognition mode by the amount that communication is not performed.

  The detecting unit may detect the presence of the user by detecting the user's face in the camera image. Thereby, for example, based on the positional relationship between the interactive device and the user's face, the angle of the user's face with respect to the interactive device, etc., it can be determined whether or not the user has an intention to talk to the interactive device. .

  According to the present invention, it is possible to switch to a dialog state with the user at an appropriate timing.

It is a figure which shows schematic structure of a dialogue system. It is a figure which shows the detailed structure of a dialogue apparatus. It is a figure which shows the hardware constitutions of a dialogue apparatus. It is a figure which shows an example of a response data table. It is a figure which shows another example of a response data table. It is a state transition diagram of a dialogue apparatus. It is an example of the flowchart for demonstrating the state transition of a dialogue apparatus.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant descriptions are omitted.

  FIG. 1 is a diagram illustrating an outline of a dialogue system to which a dialogue apparatus and a dialogue method according to an embodiment are applied. As shown in FIG. 1, in the dialogue system 1, a dialogue between the user 10 and the dialogue device 100 is performed.

  The interactive apparatus 100 includes an interface unit 110. In the example shown in FIG. 1, the human character 11 is displayed on a part of the interface unit 110 (a display 113 in FIG. 2 described later). The display of the humanoid character 11 allows the user 10 to interact with the interaction device 100 as if interacting with a human.

  The interactive device 100 can be connected to the server 200 via the communication network 50. Thereby, the dialogue apparatus 100 can acquire a lot of information to be used for the dialogue with the user 10 from the server 200.

  The dialog device 100 only needs to be able to perform a dialog with the user 10, and its appearance and size are not limited to the example shown in FIG. 1. For example, the interactive device 100 is suitably realized using a terminal device such as a smartphone. Such a terminal device includes various devices such as a speaker, a microphone, and various sensors necessary for interaction with the user 10, and a display serving as a contact point with the user 10, and a server via the communication network 50. This is because it is possible to communicate with 200. Further, as the interactive device 100, a physical robot imitating a human shape may be used.

  The user 10 can interact with the interaction apparatus 100 in a hands-free state. The hands-free state in this specification means a state in which an operation performed by the user 10 in contact with the interactive device 100 (for example, a button operation of the interactive device 100) is not performed. Even when the user 10 is touching something other than the interactive device 100, if the user 10 is not touching the interactive device 100, a hands-free state is set.

  The dialogue between the user 10 and the dialogue device 100 is preferably performed in a state where the user 10 is near the dialogue device 100. In FIG. 1, the positional relationship between the user 10 and the interaction device 100 that are preferable for the interaction is illustrated as a region R by a broken line. The range of the region R can be set such that the user 10 can visually recognize the character 11 displayed on the interactive device 100 and can recognize the voice from the interactive device 100. The range of such a region R is, for example, a range from several tens of centimeters to several meters from the interactive device 100. In the example illustrated in FIG. 1, the region R is set to be wide on the front side (side on which the interface unit 110 is provided) of the interactive device 100, and is set to be narrow on the side surface and back surface of the interactive device 100. That is, the region R is set to be long on the front side of the interactive device 100 and is set to be short on the side surface and back surface of the interactive device 100. When the user 10 is located in such a region R, the possibility that the user 10 can interact with the front of the interactive device 100 is increased, so that the conversation can be performed smoothly (comfortable for the user 10).

  FIG. 2 is a diagram illustrating a detailed configuration of the interactive apparatus 100. As illustrated in FIG. 2, the dialogue apparatus 100 includes an interface unit 110, a data processing unit 120, a control unit 130, a storage unit 140, and a communication unit 150.

  The interface unit 110 is a part for exchanging information with the outside of the interactive apparatus 100 (mainly the user 10 in FIG. 1). The interface unit 110 includes a camera 111, a proximity sensor 112, a display 113, a microphone 114, a speaker 115, and an operation panel 116.

  The data processing unit 120 is a part that performs data processing necessary for analysis of information input to the interface unit 110 and performs data processing necessary for generation of various information output by the interface unit 110. The data processing unit 120 includes an image processing unit 121, a sensor data processing unit 122, an output processing unit 123, a speech recognition unit 124, a speech synthesis unit 125, and an input processing unit 126.

  Hereinafter, each unit included in the interface unit 110 and the data processing unit 120 will be described.

The camera 111 images the user 10, for example. For example, the image processing unit 121 detects the position (or region) of the face of the user 10 in the camera image. Various known techniques can be used for data processing for that purpose. For example, information on various APIs (Application Program Interface) provided to Android (registered trademark) known as an OS for smartphones provided by google (registered trademark) (for example, information for obtaining, usage information) Information etc.) is listed on the following site.
http://developer.android.com/reference/android/media/FaceDetector.html

  In the interactive system 1, the camera 111 can detect that the user 10 is directed to any one of the microphone 114 that the user 10 talks to, the display 113 that outputs a response to the user 10, and the speaker 115. Installed in position.

  The distance between the interactive device 100 and the user 10 can also be detected using the camera 111. In this case, the camera 111 is installed at a position where the distance between any of the microphone 114, the display 113, and the speaker 115 and the user 10 can be detected (measured) in the interactive apparatus 100. When the distance between the interactive apparatus 100 and the user 10 is detected by the camera 111, the face area of the user 10 is detected by the above-described data processing, and the distance to the user 10 is measured from the size of the detected face area. be able to. In addition, when the interactive apparatus 100 includes two or more cameras, the user 10 can be obtained from one or more pieces of information obtained from the detection of the face area and the parallax of images captured by the two or more cameras. It is also possible to estimate the distance.

The proximity sensor 112 detects the proximity of an object within a certain distance. As the proximity sensor 112, for example, a sensor that emits infrared light (or sound waves) and detects reflected light (or reflected waves) from an object can be used. The sensor data processing unit 122 can measure the distance between the interactive device 100 and the user 10 based on the detection result of the proximity sensor 112, and determines whether the user 10 is near the interactive device 100. You can also. Various known techniques can be used for data processing for that purpose. For example, information on various APIs provided to Android, which is known as the OS for smartphones provided by google (for example, information for obtaining, usage information, etc.) is described in the following site .
http://developer.android.com/reference/android/hardware/SensorManager.html

  The display 113 displays information that the user 10 can visually recognize. The output processing unit 123 performs data processing necessary for display on the display 113. The data processing includes data processing necessary to represent the motion of the character 11 on the display 113.

The microphone 114 detects the voice of the user 10. The voice recognition unit 124 functions as a voice recognition unit that recognizes the voice of the user 10 based on the detection result of the microphone 114. The voice recognition unit 124 converts the recognized voice into a predetermined format (for example, a text format). Various known techniques can be used for data processing for recognizing speech. For example, information on various APIs provided to Android, which is known as the OS for smartphones provided by google (for example, information for obtaining, usage information, etc.) is described in the following site .
http://developer.android.com/reference/android/speech/RecognizerIntent.html

The speaker 115 emits sound that can be heard by the user 10. The voice synthesizer 125 performs various data processing for generating a voice uttered by the speaker 115. For example, the speech synthesizer 125 synthesizes various sound data, thereby converting the content (information) specified in the text format into speech. Various known techniques can be used for data processing for that purpose. For example, information on various APIs provided to Android, which is known as the OS for smartphones provided by google (for example, information for obtaining, usage information, etc.) is described in the following site .
http://developer.android.com/reference/android/speech/tts/TextToSpeech.html

In the present embodiment, the voice input method is not limited to a method that assumes continuous dialogue, but a method specialized for keyword-type voice input is also employed. When a large-scale vocabulary is required for continuous dialogue, the storage area and calculation capability of the dialogue apparatus 100 are limited. Therefore, during voice dialogue, server-connected voice recognition by communication with the server is used. On the other hand, in the keyword type voice input, a voice recognition engine that can be completed inside the dialogue apparatus 100 can be used. In this case, since communication processing with the server is not performed, it is effective in terms of battery consumption efficiency and the like. It is. For example, even with commercially available speech recognition engines, it is common for the same company to sell servers and local types of engines, and information about them (for example, information for obtaining information, usage information) Are listed on the following site.
http://www.fuetrek.co.jp/product/vgate/asr.html

  The operation panel 116 detects an operation (user operation) of the user 10. The input processing unit 126 performs necessary data processing according to the user operation based on the detection result of the operation panel 116.

  With the above configuration, the interactive apparatus 100 acquires various information from the outside (including the user 10) via the interface unit 110, and transmits various information to the outside (including the user 10), for example. be able to. Further, in the present embodiment, an interaction with the user 10 is realized by further cooperation of each element such as a control unit 130, a storage unit 140, and a communication unit 150 described later.

  The control unit 130 is a part that controls each element of the interactive apparatus 100, and includes a detection unit 131, a user intention determination unit 132, an information browsing availability determination unit 133, a dialog control unit 134, a response content determination unit 135, and output control, which will be described later. Unit 136, state control unit 137, and voice recognition control unit 138. However, the functions of the control unit 130 are not limited to those functions.

  The storage unit 140 is a part that stores various types of information necessary for a dialogue between the dialogue device 100 and the user 10. The storage unit 140 stores, for example, various data tables described later.

  The communication unit 150 is a part that communicates with the outside of the interactive device 100 (for example, the server 200 in FIG. 1). Although the communication method is not particularly limited, for example, wireless communication between the communication unit 150 and a base station (not shown), wired communication between the base station and the server 200, or the like can be used.

  Hereinafter, each unit included in the control unit 130 will be described.

  The detection unit 131 is a part (detection unit) that detects the distance between the user 10 and the interactive device 100 and the presence of the user 10 in the camera image. The distance between the user 10 and the interactive device 100 is performed using the camera 111 and the image processing unit 121, or the proximity sensor 112 and the sensor data processing unit 122. Detection of the presence of the user 10 in the camera image is performed using the camera 111, the image processing unit 121, and the like. The detection unit 131 preferably detects the presence of the user 10 by detecting the face of the user 10 in the camera image.

  The user intention determination unit 132 is a part (user intention determination unit) that determines whether the user 10 in the hands-free state has an intention to talk to the interactive device 100 based on the detection result of the detection unit 131. is there. For example, when the distance between the user 10 and the interactive device is equal to or less than a predetermined distance (for example, the user 10 is located inside the region R in FIG. 1) and the presence of the user 10 in the camera image is detected, the user The intention determination unit 132 can determine that the user 10 has an intention to talk.

  The information browsing availability determination unit 133 is a part (user state determination unit) that determines whether or not the user 10 can visually recognize information from the interactive device 100 based on the detection result of the detection unit 131. For example, the face of the user 10 faces the display 113 and the distance between the user 10 and the display 113 is within a range in which the user 10 can browse the information displayed on the display 113 (for example, the user 10 is shown in FIG. 1). The information browsing availability determination unit 133 can determine that the user 10 is in a state where the information from the interactive device 100 can be visually recognized.

  The dialogue control unit 134 controls the dialogue with the user 10. For example, the dialogue control unit 134 transmits the recognition result of the voice recognition unit 124 to the response content determination unit 135 described later. In addition, the dialogue control unit 134 transmits the response content determined by the response content determination unit 135 to the output processing unit 123, thereby transmitting the response content to the user 10 via the speaker 115, the display 113, or the like.

  The response content determination unit 135 is a part that determines the response content of the dialogue apparatus 100 for the utterance of the user 10. Various methods are conceivable for determining the response content by the response content determination unit 135. For example, the response of the dialogue apparatus 100 to a specific vocabulary (keyword) is stored in the response data table 141 of FIG. The response content can be determined according to the table 141. The response data table 141 is stored in the storage unit 140, for example.

  FIG. 4 is a diagram illustrating an example of the response data table 141. As shown in FIG. 4, the response data table 161 describes user utterances and response information in association with each other. In the example shown in FIG. 4, the user utterance "Hello", "Good morning", "we're going", for the "I'm home", the system utterance "Hello. The name of you?", "Good morning!", " Come on! ”And“ Return home ”correspond respectively.

  Thus, the interaction device 100 can interact with the user 10 by the interaction device 100 responding to the utterance of the user 10 or the like.

  Returning to FIG. 2, the output control unit 136 is a part (output control unit) that controls output to the user 10 based on the determination result of the information browsing availability determination unit 133. Specifically, the output control unit 136 transmits the response content transmitted from the dialogue control unit 134 to the user 10 in an appropriate manner according to the determination result of the information browsing availability determination unit 133. Specifically, the response content is transmitted to the user 10 by appropriately combining audio output information and visual information, and the ratio between the audio output information and the visual information is adjusted. Various methods can be considered for determining the mode of response content by the dialog control unit 134. For example, voice output information and visual output are made using the response of the dialog device 100 to a keyword and the determination result of the information browsing availability determination unit 133 as keys. Information is stored in the response data table 142 of FIG. 5 as values, and the response contents can be determined according to the response data table 142. The response data table 142 is stored in the storage unit 140, for example.

  FIG. 5 is a diagram illustrating an example of the response data table 142. As shown in FIG. 5, the response data table 142 describes the audio output information and the visual output information as values using the response contents and the information browsing availability determination result as keys.

  The response content indicates the content of information to be transmitted from the interactive apparatus 100 to the user 10. In the example shown in FIG. 5, “Tomorrow Tokyo fine 30, 18” is shown as the response content. This response content is information related to the weather, which means that tomorrow's Tokyo is sunny and the maximum temperature is expected to be 30 degrees and the minimum temperature is expected to be 18 degrees.

  The information browsing availability determination result is a flag indicating whether or not the user 10 can browse the information output to the interactive device 100. Whether or not the information can be browsed is determined by the information browsing availability determination unit 133 described above. In the example illustrated in FIG. 5, the information browsing availability determination result is expressed in two ways, “TRUE” and “FALSE”. When the information browsing availability determination result is “TRUE”, the user 10 can visually recognize the information displayed on the display 113 of the interactive apparatus 100. When the information browsing availability determination result is “FALSE”, the user 10 is in a state where the information displayed on the display 113 of the interactive apparatus 100 cannot be viewed.

  The voice output information indicates information to be transmitted to the user by voice among the response contents. Even if the audio output information has the same response content, the audio output information has different contents depending on the information browsing availability determination result. When the information browsing result determination is “TRUE”, there is less audio output information than when the information browsing availability determination result is “FALSE”. In the example illustrated in FIG. 5, when the information browsing availability determination result is “TRUE”, the voice output information is “Tomorrow is sunny”, and when the information browsing determination result is “FALSE”, the voice output information is displayed. Is said to be clear tomorrow in Tokyo, with a maximum temperature of 30 degrees and a minimum temperature of 18 degrees.

  The visual output information indicates information to be transmitted to the user by visual sense among the response contents. The visual output information does not exist when the information browsing availability determination result is “FALSE”, and exists only when the information browsing availability determination is “TRUE”. In the example illustrated in FIG. 5, when the information browsing availability determination result is “TRUE”, the visual output information is “Tokyo clear, highest temperature 30 degrees, lowest temperature 18 degrees”.

  Returning to FIG. 2, the state control unit 137 is a part (interactive state control means) that switches the state of the interactive device 100 between the interactive state and the non-interactive state. For example, when the user intention determination unit 132 determines that the user 10 has an intention to interact, the state control unit 137 switches the state of the interaction device 100 from the non-interactive state to the interactive state. The switching of the state of the interactive device 100 will be described in detail later with reference to FIGS. 6 and 7.

  The voice recognition control unit 138 continuously recognizes a vocabulary included in the voice of the user 10 and a second recognition mode that recognizes only a predetermined vocabulary (keyword) included in the voice of the user 10. Is a portion (voice recognition means) that is executed by switching. Details of the second detection mode and the second detection mode will be described later.

  Here, the hardware configuration of the interactive apparatus 100 will be described with reference to FIG. FIG. 3 is a hardware configuration diagram of the interactive apparatus 100. As shown in FIG. 3, the interactive apparatus 100 physically includes one or a plurality of CPUs (Central Processing Units) 21, a RAM (Random Access Memory) 22 and a ROM (Read Only Memory) 23 that are main storage devices. A communication module 26 that is a data transmission / reception device, an auxiliary storage device 27 such as a semiconductor memory, an input device 28 that accepts user input such as an operation panel (including operation buttons) and a touch panel, an output device 29 such as a display, and a camera. It can be configured as a computer including the imaging device 24 and the hardware of the sensor 25 such as an infrared sensor. Each function of the interactive device 100 in FIG. 2 is, for example, by loading one or more predetermined computer software on hardware such as the CPU 21 and the RAM 22 to control the communication module 26 and the input device 28 under the control of the CPU 101. This can be realized by operating the output device 29, the imaging device 24, and the sensor 25, and reading and writing data in the RAM 22 and the auxiliary storage device 27.

  With reference to FIG. 2 again, details of the first recognition mode and the second recognition mode executed by the speech recognition control unit 138 will be described.

  The first recognition mode is executed in the interactive state. In the first recognition mode, the voice recognition control unit 138 continuously recognizes the vocabulary included in the voice of the user 10. Recognizing vocabulary continuously is intended to recognize and recognize as much as possible the vocabulary contained in a series of user voices in a dialogue. Ideally, all vocabularies, i.e. all the voices of the user 10, are recognized.

  In the embodiment, in the first recognition mode, the voice recognition control unit 138 communicates with the outside of the interactive apparatus 100 and uses the data processing of the server 200 to continuously vocabulary included in the voice of the user 10. Recognize.

  The second recognition mode is executed in a non-interactive state. In the second recognition mode, the voice recognition control unit 138 recognizes only keywords included in the user's voice. Further, the user intention determination unit 132 determines that the user 10 has an intention to talk to the interactive device 100 when the voice recognition control unit 138 recognizes a keyword included in the voice of the user 10. Keyword, for example, the user utterance "Hello" in FIG. 4 described above, and the like "Good morning".

  In the embodiment, in the second recognition mode, the voice recognition control unit 138 recognizes only the keyword included in the voice of the user 10 without performing communication with the outside of the interactive device 100.

  FIG. 6 is a state transition diagram of the interactive device 100. As shown in FIG. 6, the interactive device 100 is placed in either a dialog state or a non-interaction state.

  The non-interactive state is a state where the interactive device 100 is not interacting with the user 10. In this non-interactive state, the first recognition mode is executed. In this state, communication between the dialogue apparatus 100 (the voice recognition unit 124 thereof) and the server 200 is not performed. Then, the dialogue is started when the keyword is detected in the user's voice, and the dialogue device 100 shifts to the dialogue state (AR1).

  The dialog state is a state in which the dialog device 100 is interacting with the user 10. In the dialog state, the second recognition mode is executed. In this state, communication between the interactive device 100 and the server 200 is performed. Therefore, a smooth dialogue is performed by large vocabulary recognition using the speech recognition engine of the server 200. When the dialogue ends, the dialogue device 100 shifts to a non-dialogue state (AR2).

  FIG. 7 is a flowchart for explaining the state transition of the interactive apparatus 100 (FIG. 2). The processing of this flowchart can be executed by the control unit 130 of the interactive apparatus 100 unless otherwise specified.

  First, the dialogue apparatus 100 determines whether or not it is in a dialogue state (step S1). In the case of a dialog state (step S1: YES), the dialog device 100 advances the process to step S2. When that is not right (step S1: NO), the dialogue apparatus 100 advances a process to step S8.

  If the conversation state is set in step S1 (step S1: YES), the speech recognition control unit 138 of the interaction apparatus 100 executes the first recognition mode (step S2).

  Next, the dialogue apparatus 100 determines whether or not there is no talk from the user 10 for a certain time (step S3). This determination is performed by, for example, the dialogue control unit 134. When there is no talk from the user for a certain period of time (step S3: YES), the dialogue apparatus 100 advances the process to step S4. When that is not right (step S3): NO), the dialogue apparatus 100 advances a process to step S6.

  If there is no talk from the user 10 for a predetermined time in step S3 (step S3: YES), the dialogue apparatus 100 determines whether or not the user 10 can be detected by the camera 111 (step S4). This determination is performed by the detection unit 131, for example. When the user 10 can be detected by the camera 111 (step S4: YES), the interactive apparatus 100 proceeds with the process to step S5. When that is not right (step S4: NO), the dialogue apparatus 100 advances a process to step S7.

  When the user 10 can be detected by the camera 111 in step S4 (step S4: YES), the interactive apparatus 100 has a distance from the user 10 within a predetermined range (for example, the user 10 is located inside the region R in FIG. 1). Whether or not) (step S5). When the distance to the user 10 is within the predetermined range (step S5: YES), the dialogue apparatus 100 advances the process to step S6. Otherwise (step S5: NO), the dialogue apparatus 100 proceeds with the process to step S7.

  When there is a talk from the user 10 for a certain time in step S3 (step S3: NO), or when the distance from the user is within a predetermined range in step S5 (step S5: YES), the interactive device 100 It is determined that the user 10 has an intention to talk and the conversation state is maintained (step S6). For example, the user intention determination unit 132 determines that the user 10 has an intention to talk. The process of maintaining the dialog state is performed by the dialog control unit 134, for example.

  When the user 111 cannot be detected by the camera 111 in step S4 (step S4: NO), or when the distance from the user is not within the predetermined range in step S5 (step S5: NO), the interactive device 100 talks to the user 10 It is determined that there is no intention, and a transition is made to a non-interactive state (step S7).

  On the other hand, in the non-interactive state in step S1 (step S1: NO), the voice recognition control unit 138 of the interactive apparatus 100 executes the second recognition mode (step S8).

  Next, the voice recognition control unit 138 of the interactive apparatus 100 determines whether or not a keyword has been detected (step S9). When the keyword is detected (step S9: YES), the dialogue apparatus 100 proceeds with the process to step S12. Otherwise (step S9: NO), the dialogue apparatus 100 proceeds with the process to step S10.

  When no keyword is detected in step S9 (step S9: NO), the interactive apparatus 100 determines whether or not the user 10 can be detected by the camera 111 (step S10). When the user 10 can be detected by the camera 111 (step S10: YES), the interactive apparatus 100 proceeds with the process to step S11. When that is not right (step S10: NO), the dialogue apparatus 100 advances a process to step S13.

  When the user 111 can be detected by the camera 111 in step S10 (step S10: YES), the dialogue apparatus 100 determines whether or not the distance from the user 10 is within a predetermined range (step S11). When the distance to the user 10 is within the predetermined range (step S11: YES), the dialogue apparatus 100 proceeds with the process to step S12. When that is not right (step S11: NO), the dialogue apparatus 100 advances a process to step S13.

  When the keyword is detected in step S9 (step S9: YES), or when the distance to the user 10 is within the predetermined range in step S11 (step S11: YES), the interactive device 100 has an intention to talk to the user 10 The determination is made and the state transitions to a dialog state (step S12).

  When the user 111 cannot be detected by the camera 111 in step S10 (step S10: NO), or when the distance from the user 10 is not within the predetermined range in step S11 (step S11: NO), the interactive device 100 talks to the user 10 It is determined that there is no intention and the non-interactive state is maintained (step S13).

  After the process of step S6, S7, S12 or S13 is completed, the interactive apparatus 100 returns the process to step S1 again.

  According to the flowchart shown in FIG. 7, the interaction device 100 detects the distance between the user 10 and the interaction device 100 and the presence of the user 10 in the camera image (steps S4, S5, S10, S11), and the interaction device 100. Determining whether or not the user 10 in the hands-free state has an intention to talk to the interactive device 100 based on the detection result of the detecting step (steps S6, S7, S12, S13), Steps (steps S7 and S12) for controlling the state of the interactive device 100 so that the interactive device 100 switches to either the interactive state or the non-interactive state based on the determination result of the determination step performed by the device 100. Executed.

  Next, the function and effect of the interactive device 100 will be described. The interactive device 100 detects the distance between the user and the interactive device and the presence of the user in the camera image, and the user in the hands-free state based on the detection result of the detection unit 131 Based on the user intention determination unit 132 that determines whether or not he / she has a talking intention, and the determination result of the determination unit, the dialog device 100 is switched so as to switch to either the dialog state or the non-dialog state. And a state control unit 137 for controlling the state. According to the interactive device 100, the interactive device 100 is controlled to switch to either the interactive state or the non-interactive state based on whether or not the user 10 has an intention to talk to the interactive device 100. . Thereby, the dialogue apparatus 100 can switch to the dialogue state at an appropriate timing according to the intention of the user 10.

  In addition, the interactive apparatus 100 includes an information browsing availability determination unit 133 that determines whether the user 10 can visually recognize information from the interactive apparatus 100 based on the detection result of the detection unit 131, and an information browsing availability determination. And an output control unit 136 that controls output to the user 10 based on the determination result of the unit 133. Thereby, when the user 10 cannot visually recognize (view) the visual output (information) from the interactive apparatus 100, for example, information can be transmitted to the user 10 only by voice. Further, when the user 10 can visually recognize the visual output from the interactive apparatus 100, for example, the audio output can be shortened by using both the visual output and the audio output.

  Further, the interactive device 100 executes the first recognition mode that continuously recognizes the vocabulary included in the voice of the user 10 in the dialog state, and only the predetermined vocabulary included in the voice of the user 10 in the non-interactive state. And a second recognition mode for recognizing the voice recognition control unit 138. In that case, in a non-interactive state, the user intention determination unit 132, when a predetermined vocabulary (keyword) included in the voice of the user 10 is recognized by the voice recognition control unit 138 that executes the second recognition mode, It is determined that 10 has an intention to talk to the interactive apparatus 100. Thereby, the dialogue apparatus 100 can switch from the non-dialogue state to the dialogue state at an appropriate timing according to the intention of the user 10 when the user 10 speaks the keyword.

  In the first recognition mode, the speech recognition control unit 138 continuously recognizes the vocabulary included in the speech of the user 10 by communicating with the outside of the interactive device 100 and using the data processing of the server 200. In the second recognition mode, the voice recognition control unit 138 recognizes only the keyword included in the voice of the user 10 without performing communication with the outside of the interactive device 100. Thereby, in the first recognition mode, it is possible to perform voice recognition (server type voice recognition) capable of recognizing a large vocabulary using the data processing of the server 200. Also, in the second recognition mode, for example, voice recognition can be performed while reducing power consumption compared to the first recognition mode by the amount that communication is not performed.

  Further, the detection unit 131 may detect the presence of the user 10 by detecting the face of the user 10 in the camera image. Accordingly, for example, whether or not the user 10 has an intention to talk to the interactive device 100 based on the positional relationship between the interactive device 100 and the face of the user 10, the angle of the face of the user 10 with respect to the interactive device 100, and the like. Can be determined.

  DESCRIPTION OF SYMBOLS 1 ... Dialog system, 10 ... User, 11 ... Character, 50 ... Communication network, 100 ... Dialog device, 110 ... Interface part, 111 ... Camera, 112 ... Proximity sensor, 113 ... Display, 114 ... Microphone, 115 ... Speaker, 116 ... Operation panel, 120 ... Data processing unit, 121 ... Image processing unit, 122 ... Sensor data processing unit, 123 ... Output processing unit, 124 ... Speech recognition unit, ... Speech synthesis unit 125, Input processing unit 126, 130 ... Control unit , 131 ... detection unit, 132 ... user intention determination unit, 133 ... information browsing availability determination unit, 134 ... dialogue control unit, 135 ... response content determination unit, 136 ... output control unit, 137 ... state control unit, 138 ... voice recognition Control unit, 140 ... storage unit, 150 ... communication unit, 200 ... server, R ... area.

Claims (6)

An interactive device for interacting with a user,
Detecting means for detecting a distance between the user and the interactive device and presence of the user in a camera image;
User intention determination means for determining whether or not the user in a hands-free state has an intention to talk to the interactive device based on the detection result of the detection means;
Dialog state control means for controlling the state of the dialog device so that the dialog device is switched to either a dialog state or a non-dialogue state based on the determination result of the determination means;
An interactive apparatus comprising: User state determination means for determining whether or not the user can visually recognize information from the interactive device based on a detection result of the detection means;
Output control means for controlling output to the user based on a determination result of the user state determination means;
The interactive apparatus according to claim 1, further comprising: In the interactive state, a first recognition mode for continuously recognizing the vocabulary included in the user's voice is executed, and in the non-interactive state, only a predetermined vocabulary included in the user's voice is recognized. Voice recognition means for executing a recognition mode of
Further comprising
In the non-dialogue state, the user intention determination unit is configured such that when a predetermined vocabulary included in the user's voice is recognized by the voice recognition unit that executes the second recognition mode, the user transmits to the dialogue apparatus. The interactive apparatus according to claim 1, wherein the interactive apparatus determines that the person has an intention to speak. In the first recognition mode, the voice recognition means continuously recognizes a vocabulary included in the user's voice by communicating with the outside of the interactive device and using server data processing,
4. The dialogue apparatus according to claim 3, wherein in the second recognition mode, the voice recognition unit recognizes only a predetermined vocabulary included in the voice of the user without performing communication with the outside of the dialogue apparatus. .   The interactive device according to claim 1, wherein the detection unit detects the presence of the user by detecting a face of the user in a camera image. A dialogue method for carrying out a dialogue between a user and a dialogue device,
The interactive device detecting the distance between the user and the interactive device and the presence of the user in a camera image;
A step of determining whether or not the user in a hands-free state has an intention to talk to the interactive device based on a detection result of the detecting step;
The interactive device controlling the state of the interactive device so that the interactive device is switched to either the interactive state or the non-interactive state based on the determination result of the determining step;
Including interactive methods.

Images