JP2016071192A - Interaction device and interaction method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an interaction device and an interaction method that can start an interaction between a system and a user at an appropriate timing.SOLUTION: An interaction device 100 includes a detection part 130 that can execute a plurality of detection modes for detecting a timing to start an interaction with a user in a hands-free state, and a switching part 140 that switches between the detection modes of the detection part 130 according to the presence/absence of information to be transferred from the interaction device 100 to the user.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a dialog device for performing a dialog with a user and a dialog method for performing a dialog between the user and the dialog device.

  In recent years, dialogue technology has evolved that recognizes and answers voice recognition and natural utterances of users. By providing various sensors in a smartphone, a robot device, and the like, a dialog agent type dialog device capable of perceiving the presence of a user and performing a dialog as if interacting with a human being is being realized.

“What is Pepper (registered trademark)”, SOFTBANK CORP., [Online], [searched on September 11, 2014], Internet (http://www.softbank.jp/robot/products/)

  The dialogue (interaction) between the dialogue apparatus and the user can be started by the user's button operation, but the dialogue can be started without the button operation. For example, the interactive device can speak to the user and start the conversation by detecting the user's voice. In addition, the dialogue apparatus can detect the user's face and the like with a camera, and can speak to the user and start a dialogue only when the user is nearby. In this case, the user can start a dialogue with the dialogue device in a state where no button operation or the like of the dialogue device is performed (hereinafter referred to as “hands-free state”).

  However, just because the user speaks near the dialog device does not necessarily mean that the user has an intention to interact with the dialog device (interaction intention). For this reason, even though the user does not have an intention to interact, it is determined that the user's utterance is a conversation to the interaction device, and the interaction device may start an interaction at an incorrect timing.

  A robotic voice dialogue system such as “Pepper” (see Non-Patent Document 1) implements a mechanism for starting a dialogue when a user's face is found by combining a face detection technique using a camera. However, this system can solve the problem that the user talks from the system when the user is not nearby, but cannot solve the problem that the system erroneously starts a dialogue in response to surrounding sounds.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an interactive apparatus and an interactive method capable of starting an interaction with a user at an appropriate timing.

  An interactive apparatus according to an aspect of the present invention is an interactive apparatus for performing a dialog with a user, and is capable of executing a plurality of detection modes for detecting timing for starting a dialog with a user in a hands-free state. And a switching unit that switches the detection mode of the detection unit according to the presence or absence of information to be transmitted from the interactive device to the user.

  A dialog method according to an aspect of the present invention is a dialog method for performing a dialog between a user and a dialog device, and detects a timing at which the dialog device starts a dialog with a user in a hands-free state. Detecting a timing at which a dialog is started using any one of a plurality of detection modes for detecting, and a step in which the dialog device detects whether there is information to be transmitted from the dialog device to the user And switching the detection mode used in.

  In the above interactive apparatus or interactive method, the timing for starting an interaction with the user is detected using a detection mode switched according to the presence or absence of information to be transmitted from the interactive apparatus to the user. Accordingly, it is possible to flexibly change a criterion such as whether or not to start the dialogue and to start the dialogue at an appropriate timing.

  The plurality of detection modes include a first detection mode for detecting the user's intention to interact and a second detection mode for detecting the user's intention to interact and detecting that the user is located within a predetermined range from the interaction device. When there is no information to be transmitted from the interactive device to the user, the switching unit switches the detection unit to the first detection mode, and there is information to be transmitted from the interactive device to the user. Alternatively, the detection unit may be switched to the second detection mode. Thus, when there is no information to be transmitted to the user, the first detection mode can be used to start the conversation only when the user has an intention to interact. Therefore, for example, it is possible to prevent a conversation from being started in response to a conversation between the user and another person. Also, when there is information to be transmitted to the user, the second detection mode is used to start the dialogue with the user when the user is located within the predetermined range (nearly), thereby ensuring the information. It can be communicated to the user. Therefore, the dialogue with the user can be started at an appropriate timing.

  Further, in the first detection mode, the detection unit detects at least one of a predetermined keyword in the user's voice, a predetermined operation of the user, and a contact with the user's interactive device. May be detected. Thereby, for example, a dialogue can be started with a call to a dialogue device by a user's voice utterance, a user's action (gesture), or a user's contact action with the dialogue device.

  In the second detection mode, the detection unit detects the presence of a user detected by a camera image, the presence of a user detected by a human sensor, the presence of a user detected by a user operation of a device other than the interactive device, By detecting at least one of the presence of the user inferred based on time and illuminance, the user's intention to interact may be detected and the user may be located within a predetermined range from the interaction device. Accordingly, it is possible to clearly detect the presence of the user and start a dialog with the user at a more appropriate timing.

  In addition, the interactive device guides the user within the predetermined range when there is information whose priority to be transmitted from the interactive device to the user is higher than a predetermined level and the user is located outside the predetermined range from the interactive device. An output control unit that outputs the expression of Thereby, a user can be guide | induced to the vicinity of a dialog apparatus and a dialog can be started at an early stage. Therefore, high priority information to be transmitted to the user can be transmitted to the user at an early stage.

  In addition, the interactive device further includes an output control unit that outputs an expression for transmitting the presence of the information to the user when there is information whose priority to be transmitted from the interactive device to the user is less than a predetermined level. May be. Thereby, the presence of information can be informed to the user and the user's interest can be aroused. Therefore, the possibility that the user has an intention to interact increases.

  According to the present invention, it is possible to start a conversation at an appropriate timing.

It is a figure which shows schematic structure of a dialogue system. It is a figure which shows the detailed structure of a dialogue apparatus. It is a figure which shows the hardware constitutions of a dialogue apparatus. It is a figure which shows an example of the transmission information data table. It is a figure which shows an example of a response data table. It is a figure which shows the operation example of a character. It is a state transition diagram of a dialogue apparatus. It is an example of the flowchart for demonstrating the state transition of a dialogue apparatus.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant descriptions are omitted.

  FIG. 1 is a diagram illustrating an outline of a dialogue system to which the dialogue apparatus and the dialogue method according to the embodiment are applied. As shown in FIG. 1, in the dialogue system 1, a dialogue between the user 10 and the dialogue device 100 is performed.

  The interactive apparatus 100 includes an interface unit 110. In the example shown in FIG. 1, the human character 11 is displayed on a part of the interface unit 110 (a display 115 in FIG. 2 described later). The display of the humanoid character 11 allows the user 10 to interact with the interaction device 100 as if interacting with a human.

  The interactive device 100 can be connected to the server 200 via the communication network 50. Thereby, the dialogue apparatus 100 can acquire a lot of information to be used for the dialogue with the user 10 from the server 200.

  The dialog device 100 only needs to be able to perform a dialog with the user 10, and its appearance and size are not limited to the example shown in FIG. 1. For example, the interactive device 100 is suitably realized using a terminal device such as a smartphone. Such a terminal device includes various devices such as a speaker, a microphone, and various sensors necessary for interaction with the user 10, and a display serving as a contact point with the user 10, and a server via the communication network 50. This is because it is possible to communicate with 200. Further, as the interactive device 100, a physical robot imitating a human shape may be used.

  The user 10 can interact with the interaction apparatus 100 in a hands-free state. The hands-free state in this specification means a state in which an operation performed by the user 10 in contact with the interactive device 100 (for example, a button operation of the interactive device 100) is not performed. Even when the user 10 is touching something other than the interactive device 100, if the user 10 is not touching the interactive device 100, a hands-free state is set.

  The dialogue between the user 10 and the dialogue device 100 is preferably performed in a state where the user 10 is near the dialogue device 100. In FIG. 1, the positional relationship between the user 10 and the interaction device 100 that are preferable for the interaction is illustrated as a region R by a broken line. The range of the region R can be set such that the user 10 can visually recognize the character 11 displayed on the interactive device 100 and can recognize the voice from the interactive device 100. The range of such a region R is, for example, a range from several tens of centimeters to several meters from the interactive device 100. In the example illustrated in FIG. 1, the region R is set to be wide on the front side (side on which the interface unit 110 is provided) of the interactive device 100, and is set to be narrow on the side surface and back surface of the interactive device 100. That is, the region R is set to be long on the front side of the interactive device 100 and is set to be short on the side surface and back surface of the interactive device 100. When the user 10 is located in such a region R, the possibility that the user 10 can interact with the front of the interactive device 100 is increased, so that the conversation can be performed smoothly (comfortable for the user 10).

  FIG. 2 is a diagram illustrating a detailed configuration of the interactive apparatus 100. As shown in FIG. 2, the dialogue apparatus 100 includes an interface unit 110, a data processing unit 120, a detection unit 130, a switching unit 140, a control unit 150, a storage unit 160, a transmission information determination unit 165, And a communication unit 170.

  The interface unit 110 is a part for exchanging information with the outside of the interactive apparatus 100 (mainly the user 10 in FIG. 1). The interface unit 110 includes a camera 111, a human sensor 112, an infrared sensor 113, an illuminance sensor (inference sensor) 114, a display 115, a microphone 116, a speaker 117, and an operation panel 118.

  The data processing unit 120 is a part that performs data processing necessary for analysis of information input to the interface unit 110 and performs data processing necessary for generation of various information output by the interface unit 110. The data processing unit 120 includes an image processing unit 121, a sensor data processing unit 122, an output processing unit 123, a speech recognition unit 124, a speech synthesis unit 125, and an input processing unit 126.

  Hereinafter, each unit included in the interface unit 110 and the data processing unit 120 will be described.

The camera 111 images the user 10, for example. For example, the image processing unit 121 detects the position (or region) of the face of the user 10 in the camera image. Various known techniques can be used for data processing for that purpose. For example, information on various APIs (Application Program Interface) provided to Android (registered trademark) known as an OS for smartphones provided by google (registered trademark) (for example, information for obtaining, usage information) Information etc.) is listed on the following site.
http://developer.android.com/reference/android/media/FaceDetector.html

Furthermore, the image processing unit 121 can also detect a specific action of the user 10 (for example, a gesture such as raising a hand) based on the image data captured by the camera 111. For example, products having a function of detecting the user's posture and the like are publicly known at the following sites, and the technology is applied to the interactive device 100, or the products and the interactive device 100 can communicate with each other. By using the information output from those products, the operation of the user 10 can be detected.
http://www.microsoft.com/en-us/kinectforwindows/

The human sensor 112 detects the approach of an object (person). As the human sensor 112, for example, a sensor that emits infrared light (or sound waves) and detects reflected light (or reflected waves) from an object can be used. The sensor data processing unit 122 can determine whether the user 10 is near the interactive device 100 based on the detection result of the human sensor 112. Various known techniques can be used for data processing for that purpose. For example, information on various APIs provided to Android, which is known as the OS for smartphones provided by google (for example, information for obtaining, usage information, etc.) is described in the following site .
http://developer.android.com/reference/android/hardware/SensorManager.html

  The infrared sensor 113 detects infrared rays. For example, when the user 10 operates a remote controller to operate a television or the like near the interactive device 100, the infrared sensor 113 detects infrared rays emitted from the remote controller. The sensor data processing unit 122 determines whether or not the user 10 is near the interactive apparatus 100 based on the detection result of the infrared sensor 113. For example, when the infrared sensor 113 detects infrared rays, it is determined that the user 10 is near the interactive apparatus 100.

The illuminance sensor 114 detects ambient brightness (illuminance). The sensor data processing unit 122 infers (determines) whether the user 10 is near the interactive device 100 based on the detection result of the illuminance sensor 114 and the current time (time zone). For example, when high illuminance is detected by the illuminance sensor 114 at night, it is determined that the user 10 is near the interactive apparatus 100. In this way, the illuminance sensor 114 can function as an inference sensor that infers the presence of the user 10. The current time can be obtained by using a timer (not shown) function included in the interactive apparatus 100, for example. Various known techniques can be used for data processing for detecting illuminance. For example, information on various APIs provided to Android, which is known as the OS for smartphones provided by google (for example, information for obtaining, usage information, etc.) is described in the following site . Note that the sensor data processing unit 122 can perform the above inference based on the illuminance (brightness) in the image captured by the camera 111 and the current time, instead of the detection result of the illuminance sensor 114.
http://developer.android.com/reference/android/hardware/SensorManager.html

  The display 115 displays information that the user 10 can visually recognize. The output processing unit 123 performs data processing necessary for display on the display 115. For example, the data processing necessary to represent the action of the character 11 on the display 115 is performed by the output processing unit 123. The display 115 can also be a touch panel so that the contact of the user 10 with the interactive device 100 can be detected.

The microphone 116 detects the voice of the user 10. The voice recognition unit 124 recognizes the voice of the user 10 based on the detection result of the microphone 116. The voice recognition unit 124 converts the recognized voice into a predetermined format (for example, a text format). Various known techniques can be used for data processing for recognizing speech. For example, information on various APIs provided to Android, which is known as the OS for smartphones provided by google (for example, information for obtaining, usage information, etc.) is described in the following site .
http://developer.android.com/reference/android/speech/RecognizerIntent.html

The speaker 117 emits sound that can be heard by the user 10. The voice synthesizer 125 performs various data processing for generating a voice emitted from the speaker 117. For example, the speech synthesizer 125 synthesizes various sound data, thereby converting the content (information) specified in the text format into speech. Various known techniques can be used for data processing for that purpose. For example, information on various APIs provided to Android, which is known as the OS for smartphones provided by google (for example, information for obtaining, usage information, etc.) is described in the following site .
http://developer.android.com/reference/android/speech/tts/TextToSpeech.html

  The operation panel 118 detects an operation (user operation) of the user 10. The input processing unit 126 performs necessary data processing according to the user operation based on the detection result of the operation panel 118. In the case where the display 115 is a touch panel, necessary data processing corresponding to the user operation is also performed by detecting that the user 10 has touched (tapped) the character 11 displayed on the display 115.

  With the above configuration, the interactive apparatus 100 acquires various information from the outside (including the user 10) via the interface unit 110, and transmits various information to the outside (including the user 10), for example. be able to. In the present embodiment, each element such as a detection unit 130, a switching unit 140, a control unit 150, a storage unit 160, a transmission information determination unit 165, and a communication unit 170, which will be described later, further collaborates with the user 10. Dialogue with is realized.

  The detection unit 130 is a part that detects timing for starting a conversation with the user 10 in the hands-free state. The detection unit 130 is configured to be able to execute a plurality of detection modes. The plurality of detection modes include a first detection mode for detecting the user 10's intention to interact, and a user 10's intention to interact and the user 10 within a predetermined range from the interaction device 100 (for example, inside the region R in FIG. 1). 2nd detection mode which detects that it is located in. Details of the first detection mode and the second detection mode will be described later.

  The switching unit 140 is a part that switches the detection mode of the detection unit 130. The switching unit 140 switches the detection mode of the detection unit 130 according to the presence / absence of information to be transmitted from the interactive device 100 to the user 10. The presence / absence of information to be transmitted to the user 10 is determined with reference to the storage unit 160 described later. When there is no information to be transmitted from the interactive apparatus 100 to the user 10, the switching unit 140 switches the detection unit 130 to the first detection mode. On the other hand, when there is information to be transmitted from the interactive device 100 to the user 10, the switching unit 140 switches the detection unit 130 to the second detection mode.

  The control unit 150 is a part that controls each element of the dialogue apparatus 100 and includes a dialogue control unit 151 and an output control unit 152 described later. However, the function of the control unit 150 is not limited to the functions of the dialogue control unit 151 and the output control unit 152.

  The storage unit 160 is a part that stores information to be transmitted from the interactive apparatus 100 to the user 10. Here, an example of information stored in the storage unit 160 will be described with reference to FIG.

  FIG. 4 is a diagram illustrating an example of a data table of information stored in the storage unit 160. As shown in FIG. 4, the transmission information data table 160a describes the positive transmission flag, the transmission information, and the evaluation information in association with each other.

  The positive transmission flag indicates that the transmission information is information that should be positively transmitted from the interactive apparatus 100 to the user 10. In the example illustrated in FIG. 4, the positive transmission flag is expressed in two ways: “true” and “false”. The transmission information whose active transmission flag is “true” has a higher priority to be transmitted from the interactive device 100 to the user 10 than the transmission information whose active transmission flag is “false”. For example, by quantifying the priority for each piece of transmission information, the positive transmission flag is set to “true” for transmission information whose priority is equal to or higher than a predetermined value (predetermined level), and the transmission information whose priority is lower than a predetermined level is positive. The transmission flag can be “false”.

  The transmission information is information transmitted from the interactive apparatus 100 to the user 10 by voice. In the example shown in FIG. 4, as transmission information, “Tomorrow is likely to rain. Don't forget your umbrella!”, “It will be a thunderstorm in the afternoon. Let's evacuate indoors when thunder starts!”, “ The animation “@@” starts at 21:00. ”,“ It looks like celebrity ## has been arrested. ”

  The evaluation information is information for transmitting the presence of transmission information to the user 10. The evaluation information may exist when the positive transmission flag is “false”. When the priority of the transmission information is less than a predetermined level, the evaluation information is used as an expression for transmitting the presence of the transmission information to the user 10 instead of the transmission information itself. In the example shown in FIG. 4, the animation “@@” starts from the transmission information “21:00”. Let's turn on the TV. ”And“ # Entertainer ## seems to have been arrested. ”, The evaluation information“ I'm looking forward to every week ”and“ Is it really unbelievable ”are associated respectively.

  Returning to FIG. 2 again, the control unit 150 can acquire information to be transmitted from the interactive apparatus 100 to the user 10 from the storage unit 160. Specifically, the control unit 150 acquires transmission information whose active transmission flag (FIG. 4) is “true” as information to be transmitted from the interactive apparatus 100 to the user 10.

  The transmission information determination unit 165 newly generates information to be transmitted from the interactive apparatus 100 to the user 10 and causes the storage unit 160 to store the information. Specifically, the transmission information determination unit 165 generates the positive transmission flag, the transmission information, and the evaluation information as necessary in FIG. 4, and updates the storage unit 160 (the transmission information data table 160a). For example, the transmission information determination unit 165 acquires the precipitation probability of the next day using an external weather forecast API at a fixed time. Then, if it exceeds 80%, information (text) “Tomorrow is likely to rain. Don't forget your umbrella!” Is additionally stored in the storage unit 160 as transmission information. When you receive other important emails, add the text “Your email seems to have come from Mr. A. Did you read it?”, The preset broadcast time of the program, and the website that posted the program information It is possible to implement various means to notify external events such as confirming from the site information and adding information when it is near the broadcast time. The positive transmission flag is set to “true” or “false” depending on the acquisition source. For example, “true” is set for a weather forecast or e-mail, and “false” is set for program information or news. The evaluation information can also be realized by setting a fixed wording associated with the information acquisition source. For example, in the case of program information, words such as “I'm looking forward to every week”, and in the case of news information, “really?

  The communication unit 170 is a part that communicates with the outside of the interactive apparatus 100 (for example, the server 200 in FIG. 1). Although the communication method is not particularly limited, for example, wireless communication between the communication unit 170 and a base station (not shown), wired communication between the base station and the server 200, or the like can be used. Via the communication unit 170, the transmission information determination unit 165 can acquire information from outside the interactive apparatus 100 and generate a large amount of transmission information.

  Here, the hardware configuration of the interactive apparatus 100 will be described with reference to FIG. FIG. 3 is a hardware configuration diagram of the interactive apparatus 100. As shown in FIG. 3, the interactive apparatus 100 physically includes one or a plurality of CPUs (Central Processing Units) 21, a RAM (Random Access Memory) 22 and a ROM (Read Only Memory) 23 that are main storage devices. A communication module 26 that is a data transmission / reception device, an auxiliary storage device 27 such as a semiconductor memory, an input device 28 that accepts user input such as an operation panel (including operation buttons) and a touch panel, an output device 29 such as a display, and a camera. It can be configured as a computer including hardware of the imaging device 24 and various sensors 25 such as an infrared sensor. Each function of the interactive device 100 in FIG. 2 is, for example, by loading one or more predetermined computer software on hardware such as the CPU 21 and the RAM 22 to control the communication module 26 and the input device 28 under the control of the CPU 101. This can be realized by operating the output device 29, the imaging device 24, and the various sensors 25, and reading and writing data in the RAM 22 and the auxiliary storage device 27.

  The details of the first detection mode and the second detection mode executed by the detection unit 130 will be described with reference to FIG. 2 again.

  In the first detection mode, the detection unit 130 detects the user's intention to interact. Specifically, in the first detection mode, the detection unit 130 detects at least one of a predetermined keyword in the voice of the user 10, a predetermined operation of the user 10, and a contact of the user 10 with the interactive device 100. Thus, the user's 10 intention to interact is detected.

  Detection of a predetermined keyword in the voice of the user 10 is performed using the microphone 116, the voice recognition unit 124, and the like. Detection of a predetermined operation of the user 10 is performed using the camera 111, the image processing unit 121, and the like. Detection of contact of the user 10 with the interactive device 100 is performed using the display 115, the operation panel 118, the input processing unit 126, and the like. As described above, when a physical robot imitating a human shape is used as the interactive device 100, the physical interaction of the user 10, such as stroking the robot head or hitting the shoulder, is performed. It is possible to detect the user 10's intention to interact.

  When detecting a predetermined keyword in the voice of the user 10, for example, "Hello", is set in advance specific keywords such as "Good morning", the speech contents of the user recognized by the speech recognition unit 124 When the keyword matches (or includes the keyword), it can be determined that the user 10 has an intention to interact.

The voice input method may be a method specialized for keyword-type voice input, which is different from that assumed for continuous dialogue. When a large-scale vocabulary is required, a server connection may be required for voice recognition, assuming a client with limited storage area and computing power, such as a mobile communication terminal. For keyword-type speech input, it is effective in terms of battery consumption efficiency and the like to use a speech recognition engine that can be completed inside the terminal instead of such means. For example, even with commercially available speech recognition engines, it is common for the same company to sell servers and local types of engines, and information about them (for example, information for obtaining information, usage information) Are listed on the following site.
http://www.fuetrek.co.jp/product/vgate/asr.html

  The detection part 130 can detect the timing (dialogue start opportunity) which starts the dialogue with the user 10 in the hands-free state by executing the first detection mode. As described above, when there is no information to be transmitted from the interactive apparatus 100 to the user 10, the switching unit 140 switches the detection unit 130 to the first detection mode. Specifically, when there is no transmission information whose active transmission flag (FIG. 4) is “true”, the switching unit 140 determines that there is no information to be transmitted from the interactive device 100 to the user 10 and detects it. The unit 130 is switched to the first detection mode. For this reason, in the first detection mode, only the dialog start opportunity (user-initiated dialog start opportunity) when the user has an intention to interact is detected.

  When the user-initiated dialogue start opportunity is detected, the dialogue device 100 responds to the user 10's intention to dialogue, whereby the dialogue between the user 10 and the dialogue device 100 is started. The dialogue is performed by the dialogue control unit 151 controlling each element of the dialogue apparatus 100. For example, when the utterance of the user 10 is detected as a user-initiated dialogue start opportunity, the dialogue apparatus 100 starts the dialogue by performing the utterance in response to the utterance of the user 10. The method of determining the response content of the interactive device 100 is not particularly limited. For example, the response from the interactive device 100 to a specific utterance (such as a predetermined keyword in the user's voice) of the user 10 is shown in FIG. A response can be determined according to the response data table 161 stored in the data table 161. The response data table 161 is stored in the storage unit 160, for example.

  FIG. 5 is a diagram illustrating an example of the response data table 161. As shown in FIG. 5, the response data table 161 describes user utterances and response information in association with each other. In the example shown in FIG. 5, the user utterance "Hello", "Good morning", "we're going", for the "I'm home", the system utterance "Hello. The name of you?", "Good morning!", " Come on! ”And“ Return home ”correspond respectively.

  Thus, the interaction between the user 10 and the interaction device 100 can be started when the interaction device 100 responds to the user 10's intention to interact (such as an utterance). When the user 10's intention to interact is other than utterance, for example, when the user 10 performs a predetermined operation, the user 10 touches the interactive device 100, a response data table corresponding to them is created. Good.

  In the second detection mode, in addition to the operation in the first detection mode (detecting the user 10's intention to interact), the detection unit 130 further detects that the user 10 is within a predetermined range (for example, the region R) from the interaction device 100. ) Is detected. Specifically, in the second detection mode, the detection unit 130 detects the presence of the user 10 detected by the image (camera image) of the camera 111, the presence of the user 10 detected by the human sensor 112, and other than the interactive device. By detecting at least one of the presence of the user detected by the user operation of the device, the presence of the user 10 inferred based on the time and the illuminance, the user's 10 intention to interact is detected, and the user 10 It is detected that the position is within a predetermined range from 100.

  The presence of the user 10 detected by the camera image is detected using the camera 111, the image processing unit 121, and the like. The presence of the user 10 detected by the human sensor 112 is detected using the human sensor 112 and the sensor data processing unit 122. Detection of the presence of a user detected by a user operation of a device other than the interactive device is performed using, for example, the infrared sensor 113 and the sensor data processing unit 122. Detection of the user 10 inferred based on the time and the illuminance is performed using the illuminance sensor 114, the sensor data processing unit 122, and the like.

  The detection unit 130 can also detect a dialog start trigger by executing the second detection mode. As described above, when there is information to be transmitted from the interactive apparatus 100 to the user 10, the switching unit 140 switches the detection unit 130 to the second detection mode. Specifically, when there is transmission information whose active transmission flag (FIG. 4) is “true”, the switching unit 140 determines that there is information to be transmitted from the interactive device 100 to the user 10 and detects it. The unit 130 is switched to the second detection mode. Also in the second detection mode, the user-initiated dialogue start trigger is detected as in the first detection mode. Further, in the second detection mode, a dialog start opportunity (system-initiated dialog start opportunity) when the user 10 is located within a predetermined range (for example, inside the region R) from the dialog device 100 is detected.

  The processing that is executed when a user-initiated dialogue start trigger is detected in the second detection mode is the same as in the first detection mode described above.

  When a system-initiated dialogue start trigger is detected in the second detection mode, the dialogue is started by transmitting information from the dialogue device 100 to the user 10. Specifically, the transmission information in which the positive transmission flag (FIG. 4) is “true” is transmitted to the user 10. In the case of the transmission information data table 160a shown in FIG. 4, the system utterances with the positive transmission flag “true” “It seems to rain tomorrow. Don't forget your umbrella!” And “It will be a thunderstorm in the afternoon. “When the thunder starts to sound, let's evacuate outdoors!” Is transmitted from the interactive device 100 to the user 10.

  As described above, the communication between the user 10 and the dialog device 100 can be started when information is transmitted from the dialog device 100 to the user 10. When the transmission information is transmitted to the user 10, the transmission information is deleted from the storage unit 160. The deletion of the transmission information may be performed by the control unit 150 or the transmission information determination unit 165.

  As described above, the dialogue is performed by the dialogue control unit 151 appropriately controlling each element of the dialogue apparatus 100. Here, the transmission information determination unit 165 can update the information stored in the storage unit 160 even during the conversation. The storage unit 160 may be updated continuously or at predetermined intervals. By updating the storage unit 160 during the dialogue, information to be transmitted from the dialogue device 100 to the user 10 can be supplemented to enhance the dialogue. When transmission information whose active transmission flag is “true” is generated by updating the storage unit 160, the information is transmitted from the interactive device 100 to the user 10. On the other hand, when there is no information to be transmitted from the interactive device 100 to the user 10, the interactive device 100 waits for an utterance from the user.

  When it is determined that there is no information to be transmitted from the interactive device 100 to the user 10 and that there is no utterance from the user 10 to the interactive device 100, the interaction between the user 10 and the interactive device 100 ends. After that, the dialogue apparatus 100 waits for the opportunity to start the dialogue again. Specifically, the detection unit 130 executes the first detection mode or the second detection mode to detect the timing for starting the conversation. At the end of the dialogue, it can be determined that there is no utterance from the user 10 to the dialogue device 100 based on whether the utterance of the user 10 does not exist for a certain period of time, for example. At this time, information such as whether or not the user 10 is near the interactive apparatus 100, such as the presence or absence of the user 10 (the face area) in the camera image, may be considered.

  Next, the output control unit 152 will be described. The output control unit 152 controls the interactive device 100 so that the interactive device 100 outputs a predetermined expression to the user 10.

  When there is information whose priority level to be transmitted from the interactive device 100 to the user 10 is a predetermined level or more and the user 10 is located outside the predetermined range (for example, the region R) from the interactive device 100, the output control unit 152 The interactive apparatus 100 is controlled to output an expression for guiding the user 10 within a predetermined range (for example, inside the region R shown in FIG. 1).

  In this case, the output control unit 152 has a function of changing the action of the character 11 so as to visually indicate that when there is one or more transmission information whose true transmission flag (FIG. 4) is “true” (active Functional communication information presence / absence notification function). For example, the motion of the character 11 is changed to a motion of waving as shown in FIG. In this case, information is not forcibly transmitted from the interactive device 100 to the user 10 by voice, but the user 10 is guided near the interactive device 100 by the action of the character 11 entering the field of view of the user 10. Can do.

  Further, when there is information whose priority to be transmitted from the interactive device to the user is less than a predetermined level, the output control unit 152 outputs an expression for transmitting the presence of the information to the user. 100 is controlled.

  In this case, the output control unit 152 visually indicates that there is no transmission information with the information transmission flag “true” and there is one or more transmission information with the information transmission flag “false”. The function of changing the action of the character 11 so as to indicate (non-positive transmission information evaluation presentation function) is provided. For example, the action of the character 11 is changed to the action of watching TV as shown in FIG. In addition, although not shown in figure, the evaluation (evaluation information of FIG. 4) with respect to it is also displayed. In this case, it is unknown whether the information is useful enough for the user 10 to be forcibly transmitted to the user 10, but the user 10 is informed of the presence of information that the user 10 may be interested in. be able to. Note that the evaluation information may be transmitted to the user 10 by voice instead of displaying the evaluation information. In that case, it is preferable to transmit the evaluation information by a method that does not require much attention to the user 10, such as reducing the volume compared to that during normal dialogue.

  As other actions of the character 11, in the example shown in FIG. 6C, the character 11 shows a talking action. This operation is performed during a dialogue between the user 10 and the dialogue apparatus 100.

  FIG. 7 is a state transition diagram of the interactive apparatus 100. As shown in FIG. 7, the interactive apparatus 100 is placed in any one of a SLEEP state, a WAIT state, and a DIALOGUE state.

  The SLEEP state indicates an initial state of the interactive apparatus 100. In this SLEEP state, only the user-initiated dialog start trigger is detected as the dialog start trigger. That is, in the SLEEP state, the detection unit 130 executes the first detection mode. This state assumes that the user 10 has a normal life such as watching TV or talking with his / her family, and does not actively start a conversation. The action of the character 11 is also displayed (FIG. 6 (a)) such as watching a television to clearly show this. At this time, if there is at least one piece of transmission information whose active transmission flag (FIG. 4) is “false”, the character 11 shows an action for transmitting the evaluation information.

  When the user 10's intention to interact is detected in the SLEEP state, the conversation is started and the interactive apparatus 100 shifts to the DIALOG state (AR1). On the other hand, in the SLEEP state, when the transmission information determination unit 165 updates the storage unit 160 (for example, periodically), and generates one or more transmission information with the “true” transmission flag (FIG. 4), The dialogue apparatus 100 shifts to the WAIT state (AR2).

  In the WAIT state, the user-initiated dialog start opportunity is detected as in the SLEEP state, and further, the system-initiated dialog start opportunity is also detected. That is, in the WAIT state, the detection unit 130 executes the second detection mode. The character 11 shows an action (FIG. 6B) such as waving his hand so as to visually express that there is content to be transmitted to the user 10. When it is detected that the user 10 is present near the interactive device 100, the interactive operation is started, and the interactive device 100 shifts to the DIALOG state (AR3).

  In the DIALOGUE state, a dialogue between the user 10 and the dialogue device 100 is in progress. During the conversation, if there is transmission information whose active transmission flag (FIG. 4) is “true”, the content is transmitted as speech from the interactive apparatus 100 to the user 10. Further, the dialogue apparatus 100 recognizes the speech of the user 10 and responds (reply using the response information in FIG. 5). The character 11 shows an interactive action (FIG. 6C). When the dialogue ends, the dialogue device 100 shifts to the WAIT state (AR4).

  When the dialogue is ended and the WAIT state is reached, in most cases, the transmission information whose active transmission flag (FIG. 4) is “true” is deleted after being transmitted to the user 10 by the previous dialogue. There is no transmission information whose active transmission flag is “true”, and the interactive apparatus 100 shifts to the SLEEP state (AR5).

  FIG. 8 is a flowchart for explaining the state transition of the interactive apparatus 100 (FIG. 2). The processing of this flowchart can be executed by the control unit 150 of the interactive apparatus 100 unless otherwise specified.

  First, the dialogue apparatus 100 determines the current state (step S1). In the case of the SLEEP state, the process proceeds to step S2. In the WAIT state, the process proceeds to step S6. In the case of the DIALOGUE state, the process proceeds to step S10.

  In the case of the SLEEP state, the detection unit 130 of the interaction device 100 determines whether or not the user 10 has an intention to interact (step S2). When the user has an intention to interact (step S2: YES), the interaction apparatus 100 proceeds with the process to step S4. When that is not right (step S2: NO), the dialogue apparatus 100 advances a process to step S3.

  If the user has no intention to interact in step S2 (step S2: NO), the interaction apparatus 100 determines whether there is information to be transmitted to the user 10 (step S3). This determination is performed by the switching unit 140, for example. If there is information to be transmitted to the user 10 (step S3: YES), the dialogue apparatus 100 proceeds with the process to step S5. When that is not right (step S3: NO), the dialogue apparatus 100 returns a process to step S1 again.

  If the user has an intention to interact in step S2 (step S2: YES), the interaction apparatus 100 transitions to the DIALOGUE state (step S4). Thereafter, the interactive apparatus 100 returns the process to step S1 again.

  In the WAIT state, the detection unit 130 of the interaction device 100 determines whether or not the user has an intention to interact (step S6). If the user has an intention to interact (step S6: YES), the interaction apparatus 100 proceeds with the process to step S4. When that is not right (step S6: NO), the dialogue apparatus 100 advances a process to step S7.

  When the user 10 has no intention to interact in step S6 (step S6: NO), the detection unit 130 of the interaction apparatus 100 determines whether or not the user 10 is nearby (step S7). When the user 10 is nearby (step S7: YES), the dialogue apparatus 100 proceeds with the process to step S4. When that is not right (step S7: NO), the dialogue apparatus 100 advances a process to step S7.

  When the user 10 is not nearby in step S7 (step S7: NO), the dialogue apparatus 100 determines whether there is any information to be transmitted to the user 10 (step S8). This determination is performed by the switching unit 140, for example. If there is no information to be transmitted to the user 10 (step S8: YES), the dialogue apparatus 100 advances the process to step S9. When that is not right (step S8: NO), the dialogue apparatus 100 returns a process to step S1 again.

  If there is no information to be transmitted to the user 10 in step S8 (step S8: YES), the dialogue apparatus 100 shifts to the SLEEP state (step S9). Thereafter, the interactive apparatus 100 returns the process to step S1 again.

  In the case of the DIALOGUE state, the dialogue apparatus 100 determines whether or not the dialogue with the user 10 has ended (step S10). This determination is made by, for example, the dialogue control unit 151. When the dialogue is finished (step S10: YES), the dialogue device 100 advances the process to step S5. When that is not right (step S10: NO), the dialogue apparatus 100 returns a process to step S1 again.

  Although not shown in FIG. 8, it is assumed that the information in the storage unit 160 is updated in a timely manner. By updating the storage unit 160, the determination result in whether or not there is information to be transmitted to the user (steps S3 and S8) can be different.

  According to the flowchart shown in FIG. 8, a plurality of detection modes (a first detection mode and a second detection mode) for detecting the timing when the interactive device 100 starts a conversation with the user 10 in the hands-free state. The step (step S2, S3, S6, S7) which detects the timing which starts a dialog using any detection mode among these, and the presence or absence of the information which the dialog apparatus 100 should transmit to the user 10 from the dialog apparatus 100 In response to this, steps (S5, S9) of switching the detection mode used in the detecting step are executed.

  Next, the function and effect of the interactive device 100 will be described. The dialogue apparatus 100 includes a detection unit 130 configured to be able to execute a plurality of detection modes for detecting timing for starting a dialogue with the user 10 in a hands-free state, and information to be transmitted from the dialogue apparatus 100 to the user. And a switching unit 140 that switches the detection mode of the detection unit 130 according to the presence or absence of. Thereby, the timing which starts the dialog with the user 10 is detected using the detection mode switched according to the presence or absence of information to be transmitted from the dialog device 100 to the user 10. By flexibly changing the criteria such as whether or not to start the dialogue, the dialogue can be started at an appropriate timing.

  The plurality of detection modes include a first detection mode for detecting the user 10's intention to interact, and detecting the user 10's intention to interact and the user 10 being located within a predetermined range (inside the region R) from the interaction device 100. The switching unit 140 switches the detection unit 130 to the first detection mode when there is no information to be transmitted from the interactive device 100 to the user 10, and the interactive device When there is information to be transmitted from 100 to the user 10, the detection unit 130 is switched to the second detection mode. Thus, when there is no information to be transmitted to the user 10, the first detection mode can be used to start the conversation only when the user 10 has an intention to interact. Therefore, for example, it is possible to prevent a conversation from being started in response to a conversation between the user 10 and another person. When there is information to be transmitted to the user 10, the second detection mode is used to start a dialog with the user 10 when the user 10 is located (close) within the predetermined range. Can be reliably transmitted to the user. Therefore, the dialogue with the user 10 can be started at an appropriate timing.

  Further, in the first detection mode, the detection unit 130 detects at least one of a predetermined keyword in the user's voice, a predetermined operation of the user, and a contact with the user's interactive device, whereby the user's interaction is detected. Detect intention. Thereby, for example, a dialogue can be started with a call to a dialogue device by a user's voice utterance, a user's action (gesture), or a user's contact action on the dialogue device. For example, by providing a keyword type voice input detection function (keyword detection), it is possible to answer a call from the user 10 to the dialog device 100 by voice utterance. By providing the gesture input function (detection of motion), it is possible to give the dialog device 100 an opportunity to start a dialog by a gesture such as beckoning without speaking. By providing a physical interaction sensing function (contact detection), it is possible to start a dialogue by physical interaction.

  In the second detection mode, the detection unit 130 is detected by the presence of the user 10 detected by the camera image, the presence of the user 10 detected by the human sensor 112, or a user operation of a device other than the interactive device 100. By detecting at least one of the presence of the user 10 and the presence of the user 10 inferred based on the time and the illuminance, the user's intention to interact is detected and the user is located within a predetermined range from the interaction device 100. Detect what to do. Thereby, the presence of the user 10 can be clearly detected, and the dialogue with the user 10 can be started at a more appropriate timing. For example, an image processing function (detection by a camera image) makes it possible to talk with the user 10 clearly in front of the interactive device 100 as an opportunity. By the detection by the human sensor 112, it is possible to talk even when it cannot be detected by image processing such as at night. By detecting the peripheral device operation, it is possible to talk to the user 10 even when the dialogue apparatus 100 is not directly facing. In addition, by providing an inference type detection means, the presence of the user 10 can be detected without using special image processing or sensors, and the accuracy of the speaking timing can be determined by using the detection results of the sensors in combination. Can be improved.

  Further, the dialogue apparatus 100 determines the user 10 when the priority level to be transmitted from the dialogue apparatus 100 to the user 10 is higher than a predetermined level and the user 10 is located outside the predetermined range from the dialogue apparatus 100. It further includes an output control unit 152 that outputs an expression for guiding within the range. Thereby, the user 10 can be guided to the vicinity of the dialogue apparatus 100 and the dialogue can be started early. Therefore, high priority information to be transmitted to the user 10 can be transmitted to the user 10 at an early stage. That is, when there is information to be transmitted to the user 10, the character motion control function of the output control unit 152 visually appeals the information to prompt the user to approach the interactive device 100, and naturally It is possible to start a dialogue.

  When there is information whose priority to be transmitted from the interactive apparatus 100 to the user 10 is less than a predetermined level, the output control unit 152 outputs an expression for transmitting the presence of the information to the user 10. Thereby, the presence of information can be notified to the user 10, and the user's 10 interest can be aroused. Therefore, the possibility that the user 10 has an intention to interact increases. That is, when there is information that may be beneficial to the user 10, the interaction of the dialogue apparatus 100 itself is presented, and the presence of the information is indirectly presented to the user 10, thereby arousing the interest of the user 10. It is possible to guide to dialogue.

  DESCRIPTION OF SYMBOLS 1 ... Dialog system, 10 ... User, 11 ... Character, 50 ... Communication network, 100 ... Dialog device, 110 ... Interface part, 111 ... Camera, 112 ... Human sensor, 113 ... Infrared sensor, 114 ... Illuminance sensor (inference sensor) , 115 ... Display, 116 ... Microphone, 117 ... Speaker, 118 ... Operation panel, 120 ... Data processing unit, 121 ... Image processing unit, 122 ... Sensor data processing unit, 123 ... Output processing unit, 124 ... Voice recognition unit, ... voice synthesis unit 125, input processing units 126, 130 ... detection unit, 140 ... switching unit, 150 ... control unit, 151 ... dialogue control unit, 152 ... output control unit, 160 ... storage unit, 165 ... transmission information determination unit 165 , 170 ... communication section, 200 ... server, R ... area.

Claims (7)

An interactive device for interacting with a user,
A detection unit configured to be able to execute a plurality of detection modes for detecting timing for starting a conversation with the user in a hands-free state;
A switching unit that switches a detection mode of the detection unit according to the presence or absence of information to be transmitted from the interactive device to the user;
An interactive apparatus comprising: The plurality of detection modes of the detection unit are:
A first detection mode for detecting the user's intention to interact;
A second detection mode for detecting the user's intention to interact and detecting that the user is within a predetermined range from the interaction device;
The switching unit switches the detection unit to the first detection mode when there is no information to be transmitted from the interactive device to the user, and when there is information to be transmitted from the interactive device to the user. The interactive device according to claim 1, wherein the detection unit is switched to the second detection mode.   In the first detection mode, the detection unit detects at least one of a predetermined keyword in the user's voice, a predetermined operation of the user, and contact of the user with the interactive device. The dialogue apparatus according to claim 2, wherein the dialogue intention of the user is detected.   In the second detection mode, the detection unit detects the presence of the user detected by a camera image, the presence of the user detected by a human sensor, and a user operation of a device other than the interactive device. By detecting at least one of the presence of the user, the presence of the user inferred based on time and illuminance, the user's intention to interact is detected and the user is located within a predetermined range from the interaction device The interaction device according to claim 2, wherein the interactive device is detected. In order to guide the user into the predetermined range when there is information whose priority level to be transmitted from the interactive device to the user is equal to or higher than a predetermined level and the user is located outside the predetermined range from the interactive device An output control unit that outputs an expression of
The interactive apparatus according to any one of claims 2 to 4, further comprising: An output control unit that outputs an expression for transmitting the presence of the information to the user when there is information whose priority level to be transmitted from the interactive device to the user is less than a predetermined level;
The interactive apparatus according to any one of claims 2 to 4, further comprising: A dialogue method for carrying out a dialogue between a user and a dialogue device,
Detecting the timing for starting the dialogue by using any one of a plurality of detection modes for detecting the timing when the dialogue device starts a dialogue with the user in a hands-free state; ,
Switching the detection mode used in the detecting step according to whether the interactive device has information to be transmitted from the interactive device to the user;
Including interactive methods.

Images